Page 71 - Applied Probability

3. Newton’s Method and Scoring
54
can be singular or nearly so. To cure this ill, Marquardt suggested
substituting
m

t
=
A n
i=1 w i dµ i (θ n ) dµ i (θ n )+ λI
for it and iterating according to
m
t
= θ n + A −1 w i [x i − µ i (θ n )]dµ i (θ n ) . (3.20)
θ n+1
n
i=1
Prove that the increment ∆θ n = θ n+1 − θ n proposed in equation
(3.20) minimizes the criterion
m
1 2 λ 2
w i [x i − µ i (θ n ) − dµ i (θ n )∆θ n ] + ∆θ n .
2
2 2
i=1
8. Consider the quadratic function
1 t 21
L(θ)= −(1, 1)θ − θ θ
2 11
2
defined on R . Compute the iterates of the quasi-Newton scheme
= θ n + A −1 dL(θ n ) t
θ n+1
n

10
t
starting from θ 1 =(0, 0) and A 1 = − and using Davidon’s
01
update (3.9).
9. For symmetric matrices A and B, define A 0 to mean that A is
nonnegative definite and A B to mean that A − B 0. Show that
A B and B C imply A C. Also show that A B and B A
imply A = B. Thus, induces a partial order on the set of symmetric
matrices.
10. In the notation of Problem 9, demonstrate that two positive definite
matrices A =(a ij ) and B =(b ij ) satisfy A B if and only they satisfy
B −1 A −1 .If A B, then prove that det A ≥ det B,tr A ≥ tr B, and
t
t
a ii ≥ b ii for all i. (Hints: A B is equivalent to x Ax ≥ x Bx for all
vectors x. Thus, A B if and only if I A −1/2 BA −1/2 if and only if
all eigenvalues of A −1/2 BA −1/2 are ≤ 1.)
t
11. Let X =(X 1 ,... ,X m ) follow a multinomial distribution with n
trials and m categories. If the success probability for category i is θ i