Page 154 - Handbook of Deep Learning in Biomedical Engineering Techniques and Applications

Chapter 5 Depression discovery in cancer communities using deep learning 143

Figure 5.6 Sentence convolution with k ¼ 2.

where 4 is the concatenation operator. In general, let w i:iþj
refer to the concatenation of words w i ; w iþ1 ; .; w iþj , and the
output is shown in Fig. 5.6. After the word vector representation,
k
the CNN applies the filter f on the V to a window of size h words
to produce the new low-dimensional features using the hyperbol-
ic tangent function as shown in Eq. (5.2):
n
e i ¼ f ðf :x i:iþh 1 þ bÞ (5.2)
where e i is a feature generated from the nonlinear function f n
and b is a bias.
This filter is applied to each possible window of words to pro-
duce the feature map represented as shown in Eq. (5.3):

e ¼½e 1 ; e 2 .e n hþ1 Š (5.3)
k
where e ˛V . The outcome produced by the convolution layer is
the feature e where a multiple number of [82] filters can be
applied to remove the noisy data from the text. After the feature
generation, the next layer max pooling is applied on the features
[83] to extract the global abstract features. The idea behind is to
find the most important features with higher value for each
feature map such as shown in Eq. (5.4).
b e ¼ maxðeÞ (5.4)
This function results the features of length f, which is equiva-
lent to the length of the filter used in the model. These features
form the penultimate layer and are passed to a fully connected
softmax layer whose output is the probability distribution over
labels.