Page 53 - Applied Probability

2. Counting Methods and the EM Algorithm
36
From the initial estimates α 0 = .3, µ 01 =1.0, and µ 02 =2.5, compute
via the EM algorithm the maximum likelihood estimates ˆ α = .3599,
ˆ µ 1 =1.2561, and ˆ µ 2 =2.6634. Note how slowly the EM algorithm
converges in this example.
9. In the EM algorithm, demonstrate the identity
∂ n ∂ n
Q(θ | θ )| θ=θ n = L(θ )
∂θ i ∂θ i
n
for any component θ i of θ at any interior point θ of the parameter
domain. Here L(θ) is the loglikelihood of the observed data Y .
10. Suppose that the complete data in the EM algorithm involve N
binomial trials with success probability θ per trial. Here N can be
random or fixed. If M trials result in success, then the complete data
M
likelihood can be written as θ (1 − θ) N−M c, where c is an irrelevant
constant. The E step of the EM algorithm amounts to forming
Q(θ | θ n )=E(M | Y, θ n )ln θ +E(N − M | Y, θ n)ln(1 − θ)+ ln c.
The binomial trials are hidden because only a function Y of them is
directly observed. Show in this setting that the EM update is given
by either of the two equivalent expressions

E(M | Y, θ n )
=
θ n+1
E(N | Y, θ n )
θ n (1 − θ n ) d
= θ n + L(θ n ),
E(N | Y, θ n ) dθ
where L(θ) is the loglikelihood of the observed data Y [10, 16]. (Hint:
Use Problem 9.)
11. As an example of the hidden binomial trials theory sketched in Prob-
lem 10, consider a random sample of twin pairs. Let u of these pairs
consist of male pairs, v consist of female pairs, and w consist of op-
posite sex pairs. A simple model to explain these data involves a
random Bernoulli choice for each pair dictating whether it consists
of identical or nonidentical twins. Suppose that identical twins oc-
cur with probability p and nonidentical twins with probability 1 − p.
Once the decision is made as to whether the twins are identical or
not, then sexes are assigned to the twins. If the twins are identical,
one assignment of sex is made. If the twins are nonidentical, then two
independent assignments of sex are made. Suppose boys are chosen
with probability q and girls with probability 1 − q. Model these data
as hidden binomial trials. Using the result of Problem 10, give the
EM algorithm for estimating p and q. What other problems from this
chapter involve hidden binomial trials?