Page 155 - Handbook of Deep Learning in Biomedical Engineering Techniques and Applications

144 Chapter 5 Depression discovery in cancer communities using deep learning

4.1.1 Variants of convolutional neural network
Several variants of CNN can be applied for text classification:
• Static-CNN model: In static model, word vectors are of fixed
length and are pretrained vectors from any word embedding
model. To make a static model [84], words are kept static
and other parameters of the model can be learned.
• Nonstatic-CNN model: This is same static CNN model except
the fact that it required fine-tuned parameters, and this model
also outperforms [84] over static models.
• Random-CNN model: This is the basic model where all the
words are initialized randomly and later modified during
training phase. Initialization of word vectors has no impact
on the performance of the model [85]; it can be given as tuned
or nontuned. The performance depends on the learning rate.
• Multichannel CNN model: In multichannel model, more than
one input channel can be used for processing different n-gram
word vectors [82]. For example, for each 4-grams, 6-grams, and
8-grams, an input channel is to be created for the text process-
ing. Same filter is applied to all the channels, and gradients will
also be backpropagated only through one of the channels. In
multichannel at the output layer, max pooling is applied to
conclude the output. This function produces a single vector,
and the sequence of output vector is not considered. Hence,
the model is able to fine-tune one set of vectors while keeping
the other static.

4.2 Recurrent neural network
Unlike CNN models, the RNN model can work on the variable
length of sequence input and perform well in finding the
emotional drifts and negations if any exist in the sentence. The
reason behind this is that RNNs are networks with loops in
them, allowing information to persist that allows for remem-
bering values over different time durations. RNN model consists
of a sequence of nodes organized in successive layers, and sample
of a node is shown in Fig. 5.7.

Figure 5.7 Sample of a single node of RNN model. RNN, recurrent neural network.