Page 234 - Artificial Intelligence in the Age of Neural Networks and Brain Computing

3. Deep Architectures and Learning 225

FIGURE 11.3
Stacked autoencoders architecture. The first autoencoder (AE 1 ) maps the input instance
x into a compressed representation h 1 (coding operation) which is used to reconstruct the
input data (decoding operation). After training AE 1 , the code h 1 is used as input to train
AE 2 , providing the code vector h 2 and so on. The procedure is repeated for all AE n
autoencoders. The compressed representations h 1 , h 2 . h n form the stacked architecture
(SAE) which is typically fine-tuned using conventional back propagation algorithm.

the visual cortex of the brain and have been widely applied in image and speech
recognition. A CNN includes an automatic feature extractor (composed by multiple
stages of convolution and pooling layers) and a standard MLP-NN which processes
the features learned before for classification tasks (Fig. 11.4) [7]. The convolutional

FIGURE 11.4
CNN architecture. It includes a sequence of convolution (CONV) and pooling (POOL) layers
followed by a standard fully connected neural network. In the convolutional layer the input
map convolves with K filters (or kernels), providing K feature maps. After applying a
nonlinear activation function (sigmoidal or a rectified linear unit) to each feature map, the
pooling layer is performed. The features learned are the input of a fully connected neural
network followed by a softmax layer, which performs the classification tasks.