Page 87 - Handbook of Deep Learning in Biomedical Engineering Techniques and Applications

Chapter 3 Application, algorithm, tools directly related to deep learning 75

Figure 3.10 Movement of the kernel.

3.3 Recurrent neural network
Recurrent neural network (RNN) is a type of neural network
where the output from previous step is fed as input to the
current step. In traditional neural networks, all the inputs and
outputs are independent of each other, but in some cases when
it is required to predict the next word of a sentence, the previous
words are necessary; hence, there is a need to recognize the
previous words. Thus RNN came into existence, which has solved
issue with the use of a hidden layer. The main and most important
feature of RNN is hidden state, which dwell upon some informa-
tion about a sequence [17]. The basic structure of RNN is shown in
Fig. 3.13.

3.3.1 How recurrent neural network works
The working of an RNN can be described with the help of the
following example:
Example:
Suppose a deeper network consists of one input layer, three
hidden layers, and one output layer. Then not like other neural
networks, each hidden layer will have its own set of weights
and their biases. The value for hidden layer is 1; then the weights
and biases are w1 and b1, w2 and b2 for second hidden layer, and
w3 and b3 for third hidden layer. This means that each of these
layers is independent of each other, i.e., they do not memorize
any other previous outputs [18]. The process of RNN with input
and output layer is shown in Fig. 3.14[30].