Page 176 - Artificial Intelligence in the Age of Neural Networks and Brain Computing

166 CHAPTER 8 The New AI: Basic Concepts, and Urgent Risks

the world: (1) CoNNs; (2) neural networks with many layers (deep, after all); (3)
autoencoder or bottleneck networks. But all three of these design capabilities
were used and known many decades ago. The deep learning revolution was not about
a new underlying technology, but about a cultural revolution, an appreciation of
what the tools could do, and development of new groups and procedures for using
them.
The core of the “new AI” is the idea of complex systems developed by learning
rather than programming. For the first few years of the deep learning revolution, the
market was dominated by the tensor flow open software package developed by
Google. Fig. 8.3 depicts the core page of the tensor flow system, the page which
addresses how the learning is done for whatever network design or data the user
may choose:
The core of the learning is based on optimizer routines which rely on gradients,
on vectors of derivatives, which tell the optimizer whether changing any parameter
of the system leads to more error or less error.
Must gradients really be at the core of any complex system for learning or for
general purpose optimization in a nonlinear world? Many, many theorists have hoped
to avoid them, and many attempts have been made, but at the end of the day gradients
are central to any really effective system for these purposes [13]. Competitors to
Google, like Amazon Cloud Services, have been attracting more market share lately,
especially as computer hardware is deployed which exploits the special parallel
structure of all these neural network designs, but the effective and exact calculation
of gradients by the general method we call “backpropagation” remains at the very
center here, as it was at the center of the first rebirth of the neural network field.

FIGURE 8.3
Core page of Google open tensor flow system.