Page 305 - Artificial Intelligence in the Age of Neural Networks and Brain Computing

298 CHAPTER 15 Evolving Deep Neural Networks

FIGURE 15.1
A visualization of how CoDeepNEAT assembles networks for fitness evaluation. Modules
and blueprints are assembled together into a network through replacement of blueprint
nodes with corresponding modules. This approach allows evolving repetitive and deep
structures seen in many successful recent DNNs.

(Fig. 15.1) visualization of how CoDeepNEAT assembles networks for fitness eval-
uation. Modules and blueprints are assembled together into a network through
replacement of blueprint nodes with corresponding modules. This approach allows
evolving repetitive and deep structures seen in many successful recent DNNs.
DNN. Both graphs and the nodes in them are evolved. During fitness evaluation,
the modules and blueprints are combined to create a larger assembled network
(Fig. 15.1). Each node in the blueprint is replaced with a module chosen randomly
from the species to which that node points. If multiple blueprint nodes point to the
same module species, then the same module is used in all of them. The assembled
networks are evaluated the a manner similar to DeepNEAT, but the fitnesses of the
assembled networks are attributed back to blueprints and modules as the average
fitness of all the assembled networks containing that blueprint or module.
CoDeepNEAT can evolve repetitive modular structure efficiently. Furthermore,
because small mutations in the modules and blueprints often lead to large changes
in the assembled network structure, CoDeepNEAT can explore more diverse and
deeper architectures than DeepNEAT. An example application to the CIFAR10
domain is presented next.

3.3 EVOLVING DNNs IN THE CIFAR-10 BENCHMARK

In this experiment, CoDeepNEAT was used to evolve the topology of a CNN to
maximize its classification performance on the CIFAR-10 dataset, a common im-
age classification benchmark. The dataset consists of 50,000 training images and
10,000 testing images. The images consist of 32 32 color pixels and belong to