Page 188 - Neural Network Modeling and Identification of Dynamical Systems

5.3 SEMIEMPIRICAL ANN-BASED MODEL DERIVATIVES COMPUTATION 179
2
∂ e(˜y, ˆ x,w) ∂ ˆ g(ˆ x,w) T ∂ ˆ g(ˆ x,w) Similarly, the second-order sensitivities are rep-
=
∂w 2 ∂w ∂w resented by the solution of the following initial
n y 2 value problem for i = 1,...,n x :
∂ ˆ g i (ˆ x,w)
− ω i ˜y i − ˆ g i (ˆ x,w) ,
∂w 2
2
i=1 ∂ ˆ x i (0,w)
2
∂ e(˜y, ˆ x,w) ∂ ˆ g(ˆ x,w) T ∂ ˆ g(ˆ x,w) ∂w 2 = 0,
=
∂w∂ ˆ x ∂w ∂ ˆ x 2
d ∂ ˆ x i (t,w)
n y 2
∂ ˆ g i (ˆ x,w) dt ∂w 2
− ω i ˜y i − ˆ g i (ˆ x,w) ,
∂w∂ ˆ x
2ˆ
i=1 ∂ f i (ˆ x(t,w),u(t),w)
2
∂ e(˜y, ˆ x,w) ∂ ˆ g(ˆ x,w) T ∂ ˆ g(ˆ x,w) = ∂w 2
=
∂ ˆ x 2 ∂ ˆ x ∂ ˆ x 2ˆ
∂ f i (ˆ x(t,w),u(t),w) ∂ ˆ x(t,w)
n y 2 +
∂ ˆ g i (ˆ x,w) ∂w∂ ˆ x ∂w

− ω i ˜y i − ˆ g i (ˆ x,w) .
∂ ˆ x 2
T
2ˆ
i=1 ∂ ˆ x(t,w) ∂ f i (ˆ x(t,w),u(t),w)
(5.15) +
∂w ∂ ˆ x∂w
T 2ˆ
Recall that according to Schwarz’s theorem on ∂ ˆ x(t,w) ∂ f i (ˆ x(t,w),u(t),w) ∂ ˆ x(t,w)
+
equality of mixed partials, ∂w ∂ ˆ x 2 ∂w
n x
∂f i (ˆ x(t,w),u(t),w) ∂ ˆ x j (t,w)
ˆ 2
T
2
2
∂ e(˜y, ˆ x,w) ∂ e(˜y, ˆ x,w) + .
= . ∂ ˆ x j ∂w 2
∂ ˆ x∂w ∂w∂ ˆ x j=1
(5.17)
Also, the individual trajectory error function
gradient (5.12) depends on first-order sensitivi-
A computationally cheaper Gauss–Newton
ties ∂ ˆ x(t,w) , and the Hessian (5.13) additionally Hessian approximation may be obtained by
∂w
depends on second-order sensitivities discarding the second-order terms in (5.13),
i.e.,
2
n x
∂ ˆ x i (t,w)
.
∂w 2 i=1 2 ¯ t 2
∂ E(w) ∂ e(˜y(t), ˆ x(t,w),w)
≈
Differentiating the initial value problem for ∂w 2 ∂w 2
0
the semiempirical model (5.2) with respect to pa-
2
rameters w yields the initial value problem for ∂ e(˜y(t), ˆ x(t,w),w) ∂ ˆ x(t,w)
+
the first-order sensitivities, i.e., ∂w∂ ˆ x ∂w
T 2
∂ ˆ x(0,w) ∂ ˆ x(t,w) ∂ e(˜y(t), ˆ x(t,w),w)
= 0, +
∂w ∂w ∂ ˆ x∂w
T
2
ˆ
d ∂ ˆ x(t,w) ∂f(ˆ x(t,w),u(t),w) ∂ ˆ x(t,w) ∂ e(˜y(t), ˆ x(t,w),w) ∂ ˆ x(t,w)
= + dt,
dt ∂w ∂w ∂w ∂ ˆ x 2 ∂w
ˆ
∂f(ˆ x(t,w),u(t),w) ∂ ˆ x(t,w) (5.18)
+ .
∂ ˆ x ∂w
(5.16) as well as in (5.15), i.e.,