Page 204 - Neural Network Modeling and Identification of Dynamical Systems

5.5 OPTIMAL DESIGN OF EXPERIMENTS FOR SEMIEMPIRICAL ANN-BASED MODELS 195
Algorithm 2 Active CMA-ES.
Require: E : R n θ → R objective function to be minimized
+
Require: θ ∈ R n θ initial guess for parameter vector
Require: σ ∈ R >0 initial step length
Require: λ 2 population size
) *
λ
1: μ ← number of individuals subject to recombination
4
4
2: c c ← learning rate for search path p c
n θ +4
3: c σ ← c c learning rate for search path p σ
1
4: d σ ← 1 + damping factor for step length
c σ
2
5: c cov ← √ learning rate for C based on the search history
(n θ + 2) 2
4μ−2
6: c μ ← learning rate for C based on the current population
2
(n θ +12) +4μ
7: C ← I initial guess for covariance matrix
8: p σ = 0 initial value for search path
9: p c = 0

√ n θ +1
← 2 2 M [ N(0,I) ]
10: χ n θ
n θ
2
11: repeat
12: C = BD(BD) T eigendecomposition of the covariance matrix
13: for i = 1,...,λ do
14: ζ ∼ N(0,I)
i
2
+
+
15: ν i ← θ + σBDζ i ν i ∼ N(θ ,σ C)
16: E i ← E(ν i )
17: end for
18: ζ 1,...,λ ← argsort(E 1,...,λ ) sort ζ according to objective function values E(ν i )
i
μ
1 (
19: ¯ ζ ← ζ i
μ
i=1
−
20: θ ← θ +
−
+
21: θ ← θ + σBD ¯ ζ
√
22: p σ ← (1 − c σ )p σ + μc σ (2 − c σ )B ¯ ζ
√
23: p c ← (1 − c c )p c + μc c (2 − c c )BD ¯ ζ
& '
μ (
λ
(
T
T
24: C ← (1 − c cov )C + c cov p c p + c μ BD 1 μ ζ ζ − μ 1 ζ ζ T (BD) T
c
i i
i i
i=1 i=λ−μ+1
p σ −χ n θ
25: σ ← σ exp
d σ χ n θ
+
−
26: until θ − θ >ε
the general case, but experimentally confirmed Another important aspect of the effective
for many real-world problems. In the follow- training set is the choice of weights for contribu-
ing we present a pseudocode for the basic ver- tions of individual training examples to the error
sion of the Active CMA-ES algorithm (see Algo- function (so-called error weights). Note that the
rithm 2). situation when the values of inputs for ¯n train-