Page 64 - Neural Network Modeling and Identification of Dynamical Systems

52 2. DYNAMIC NEURAL NETWORKS: STRUCTURES AND TRAINING METHODS

applied to classification, regression, and system methods, see [10]. Reinforcement learning meth-
identification problems. ods are presented in the books [45–48].
If a training data set is not known beforehand, We need to mention that the actual goal of
but rather presented sequentially one example the neural network supervised learning is not
at a time, and a neural network is expected to to achieve a perfect match of predictions with
operate and learn simultaneously, then it is said the training data, but to perform highly accurate
to perform incremental learning.Additionally,if predictions on the independent data during the
the environment is assumed to be nonstationary, network operation, i.e., the network should be
i.e., the desired response to some input may vary able to generalize. In order to evaluate the gen-
eralization ability of a network, we split all the
over time, then the training data set becomes
available experimental data into training set and
inconsistent and a neural network needs to per-
test set. The model learns only on the training set,
form adaptation. In this case, we face a stability-
and then it is evaluated on an independent test
plasticity dilemma: if the network lacks plastic-
set. Sometimes, yet another subset is reserved –
ity, then it cannot rapidly adapt to changes; on the so-called validation set, which is used to select
the other hand, if it lacks stability, then it forgets the model hyperparameters (such as the number
the previously learned data.
of layers or neurons).
Another variation of supervised learning is
active learning, which assumes that the neural
network itself is responsible for the data set ac- 2.2.1 Overview of the Neural Network
quisition. That is, the network selects a new in- Training Framework
put and queries an external system (for example,
Suppose that the network parameters are rep-
some sensor) for the desired outputs that corre- resented by a finite-dimensional vector W ∈ R .
n w
spond to this input. Hence, a neural network is The supervised learning approach implies a
expected to “explore” the environment by inter-
minimization of an error function (also called
acting with it and to “exploit” the obtained data
objective function, loss function, or cost func-
by minimizing some objective. In this paradigm,
tion), which represents the deviation of actual
finding a balance between exploration and ex-
network outputs from their desired values. We
ploitation becomes an important issue. Reinforce- ¯ n w
define a total error function E : R → R to be a
ment learning takes the idea of active learning sum of individual errors for each of the training
one step further by assuming that the external examples, i.e.,
system cannot provide the network with exam-
P
ples of desired behavior – instead, it can only (p)
¯
score the previous behavior of the network. This E(W) = E (W). (2.25)
approach is usually applied to intelligent control i=1
and decision making problems. The error function (2.25) is to be minimized with
In this book, we cover only the supervised respect to neural network parameters W.Thus,
learning approach and focus on the modeling we have an unconstrained nonlinear optimiza-
and identification problem for dynamical sys- tion problem:
tems. Section 2.3.1 treats the training methods
¯
for static neural networks with applications to minimize E(W). (2.26)
W
function approximation problems. These meth-
ods constitute the basis for dynamic neural In order for the minimization problem to
network training algorithms, discussed in Sec- make sense, we require the error function to be
tion 2.3.3. For a discussion of unsupervised bounded from below.