Page 309 - Artificial Intelligence in the Age of Neural Networks and Brain Computing

302 CHAPTER 15 Evolving Deep Neural Networks

FIGURE 15.3
The best-performing LSTM variant after 25 generations of neuroevolution. It includes a
novel skip connection between the two memory cells, resulting in 5% improvement over
the vanilla LSTM baseline. Such improvements are difficult to discover by hand;
CoDeepNEAT with LSTM-specific mutation searches for them automatically.

5. APPLICATION CASE STUDY: IMAGE CAPTIONING FOR THE
BLIND
In a real-world case study, the vision and language capabilities of CoDeepNEAT
were combined to build a real-time online image captioning system. In this applica-
tion, CoDeepNEAT searches for architectures that learn to integrate image and text
representations to produce captions that blind users can access through existing
screen readers. This application was implemented for a major online magazine web-
site. Evolved networks were trained with the open source MSCOCO image
captioning dataset [46], along with a new dataset collected for this website.