Page 178 - Artificial Intelligence in the Age of Neural Networks and Brain Computing

168 CHAPTER 8 The New AI: Basic Concepts, and Urgent Risks

FIGURE 8.4
Key slide from first year “grant results” report from Ng and LeCun to COPN/NSF.

much bigger impact on university culture than the earlier success of neural networks
in equally challenging real-world applications, such as vehicle control and missile
interception [4] whose leaders do not work so hard to make their best technology
well-known.

2.3 BACKPROPAGATION: THE FOUNDATION WHICH MADE THIS
POSSIBLE
When I began work in the neural network field in the 1960s, it was widely “known”
that neural networks were an old discredited heresy. Anyone still working in that
field was generally treated as a crackpot. The theoretical rules of the scientific
method [15] and the rational quest for truth are generally not upheld so strictly
for the work of those labeled as crackpots, regardless of the logic of the work.
The belief that artificial neural networks (ANNs) could never do anything useful
(let alone explain how brains do so) was solidified by a seminal book from Marvin
Minsky [16], who is deeply revered for his insight in traditional AI to this day. The
most powerful argument against ANNs in that book had two parts: (1) the observa-
tion that ANNs could not even solve a minimal example of a classification task, the
XOR problem (see Fig. 8.5A), unless they had at least one hidden layer (see
Fig. 8.5B); and (2) even after years of effort, no one had found a reasonably effective
way to train such a network to perform such classification tasks.
Amari had previously mentioned the possibility of trying steepest descent
methods to train such networks, but dismissed them as unworkable and did not
give solutions to the obvious problems [18].