Page 175 - Artificial Intelligence in the Age of Neural Networks and Brain Computing

2. Brief History and Foundations of the Deep Learning Revolution 165

FIGURE 8.2
Four of the three to six most serious ultimate threats of human extinction, as depicted at
www.werbos.com/IT_big_picture.pdf.

posted an excellent story on what is happening now in the mainstream of that indus-
try, describing many of the key players [10]. The largest technical conference on the
real cutting edge of AI research in those companies within the last 12 months was
held in Barcelona, a research symposium organized by Juergen Schmidhuber; the
website for that symposium [11] is a good source of further information.
Not so long after that conference, I am grateful to have had a chance to be one of
the first people invited to the long-term futures conference of one of the most
important leading companies from outside the company. They expressed the general
sentiment of the industry: “How was it that Google saw all this first and got the jump
on us so completely?” And in more detail: “The new AI based on deep learning is
remaking the world here and now. We need to jump to CNN and RNN, the next big
thing.” At the time, I assumed that “CNN” meant Cellular Neural Networks, a very
important breakthrough on the computer hardware side [12]; however, they meant
Convolutional Neural Networks, a particular type of mathematical neural network
design which could be implemented either as a program for an ordinary PC, as a
program for special hardware like Game Processing Units (GPU) which offer
much more throughput in implementing neural networks, or even as an actual
chip architecture. Here I will use the acronyms CeNN and CoNN for cellular and
Convolutional Neural Networks, respectively, to avoid confusion. The company’s
interest in Recurrent Neural Networks (RNN) was recent, seen as “the next new
big thing.”
The term “deep learning” is one of those cultural terms which is defined
mainly by what it means to a large group of people. Generally, it means some
combination of neural network learning exploiting three tools new to much of