Page 315 - Artificial Intelligence in the Age of Neural Networks and Brain Computing

308 CHAPTER 15 Evolving Deep Neural Networks

FIGURE 15.7
Top: Four good captions. The model is able to abstract about ambiguous images and even
describe drawings, along with photos of objects in context. Bottom: Four bad captions.
When it fails, the output of the model still contains some correct sense of the image.

networks further, it would be possible to identify top performers more reliably, and
final performance would likely improve.
In order to speed up evolution, other techniques may be developed. For instance,
it may be possible to seed the population with various state-of-the-art architectures
and modules, instead of having to rediscover them during evolution.
Significantly more computational resources are likely to become available in the
near future. Already cloud-based services such as Amazon Web Services offer GPU
computation with a reasonable cost, and efforts to harness idle cycles on gaming
center GPUs are underway. At Sentient, for example, a distributed AI computing
system called DarkCycle is being built that currently utilizes 2M CPUs and 5000
GPUs around the world, resulting in a peak performance of 9 petaflops, on par
with the fastest supercomputers in the world. Not many approaches can take advan-
tage of such power, but evolution of deep learning neural networks can. The search
space of different components and topologies can be extended, and more hyperpara-
meters can be optimized. While cloud services such as AutoML provide some such
functionality, and benefit from the increasing computing power as well, they
currently represent brute-force approaches that are unlikely to scale as well. Given
the results in this paper, the evolutionary approach is likely to discover designs that