Page 179 - Artificial Intelligence in the Age of Neural Networks and Brain Computing

2. Brief History and Foundations of the Deep Learning Revolution 169

FIGURE 8.5
(A) the XOR task; (B) Example of 3-layer ANN with one hidden layer [17].

Circa 1970, when I was a student at Harvard, I found a way to solve these
problems, and suggested to Minsky that we could share the credit by writing a joint
paper. There were two key elements to my solution: (1) replacing the old “Threshold
Logic Unit” model of the neuron with a new differentiable model of the neuron; and
(2) using a new way to calculate derivatives by a new kind of dynamic feedback
procedure, which seemed intuitively obvious and which actually was a translation
into mathematics of Freud’s theory of psychodynamics.
Minsky stated that he could not get away with using such a model of the neuron,
since it was holy writ in computational neuroscience that the brain is a modern
digital device relying on ones and zeroesdspikesdfor its computations. But then
I showed them some actual time-series recordings from the brain, shown in
Fig. 8.6, similar to much of what we still see today:

FIGURE 8.6
Recordings from actual neurons [19] and some updated sources.