Page 201 - Neural Network Modeling and Identification of Dynamical Systems

192 5. SEMIEMPIRICAL NEURAL NETWORK MODELS OF CONTROLLED DYNAMICAL SYSTEMS

Algorithm 1 Simple homotopy continuation 5.5 OPTIMAL DESIGN OF
training algorithm for semiempirical ANN EXPERIMENTS FOR
model (5.2). SEMIEMPIRICAL ANN-BASED
¯
Require: δ, δ, τ min , τ MODELS
1: a ∼ U(W)
2: w ← a
The indirect approach to acquisition of train-
3: τ ← 0 ing data sets for ANN-based models of dynam-
4: while τ< 1 and τ > τ min do
ical systems, described in Section 2.4.3, can also
5: ˜ τ ← min{τ + τ,1}
benefit from theoretical knowledge of the sim-
6: ˜ w ← LM(E,a,w,τ)
ulated system. Recall that we need to design a
¯
7: if ˜ w − w < δ then
set of reference maneuvers that maximize the
8: w ← ˜ w
resulting training set representativeness. Such a
τ
9: τ ←˜
set of maneuvers might be designed manually
10: if ˜ w − w <δ then
by an expert in the specific domain, although
11: τ ← 2 τ this procedure is quite time consuming and the
12: end if results tend to be suboptimal. Methods for au-
13: else tomation of this procedure constitute the sub-
1
14: τ ← τ
2 ject of study for optimal design of experiments
15: end if [41]. Classical theory of optimal design of exper-
16: end while
iments is mostly dedicated to linear regression
models. Extensions of this theory to active selec-
tion of most informative training examples for
function (5.45) with respect to parameters w, feedforward neural networks were suggested
while keeping τ fixed. It uses the current pa- in [42,43]. More recently, these results were ex-
rameter values as initial guess. The Levenberg– tended to active selection of controls that pro-
Marquardt method is denoted by LM in the algo- vide most informative training examples for re-
rithm description. The continuation algorithm current neural networks [44]; however, the pri-
also involves some form of step length adapta- mary focus is on the greedy optimization with
one-step-ahead prediction horizon. All of the
tion, whereby if the norm of model parameters
change exceeds δ, the predictor step length τ is abovementioned methods alternate between the
¯
decreased and the corrector step is reevaluated. following three steps: search for the most infor-
mative training examples to include in the data
Conversely, if the norm of model parameters set guided by the current model estimate; acqui-
change does not exceed δ, the step length is in-
sition of the selected training examples; retrain-
creased. The initial guess for parameter values a
ing or adaptation of the model using the new
is picked at random.
training set. Since this approach relies on the
Note that a conceptually similar approach of
specific form of the model and involves model
solving a series of problems with increasing pre- training after inclusion of each training exam-
diction horizon was suggested in [37–40], and ple, it is better suited for online adaptation of the
it has proved to be highly successful. Results of existing model rather than the design of a new
computational experiments with this algorithm model from scratch.
for training of semiempirical ANN-based mod- In this section we discuss an approach to
els of a maneuverable F-16 aircraft motion are the optimal design of reference maneuvers for
presented in Chapter 6. semiempirical neural network–based models of