Page 307 - Fundamentals of Probability and Statistics for Engineers

290 Fundamentals of Probability and Statistics for Engineers

Analogous results are obtained when population X is discrete. Furthermore,
^
the distribution of tends to a normal distribution as n becomes large.
^
This important result shows that MLE is consistent. Since the variance
given by Equation (9.104) is equal to the Crame ´r–Rao lower bound, it is
^
efficient as n becomes large, or asymptotically efficient. The fact that MLE
is normally distributed as n !1 is also of considerable practical interest as
probability statements can be made regarding any observed value of a max-
imum likelihood estimator as n becomes large.
Let us remark, however, these important properties are large-sample proper-
ties. Unfortunately, very little can be said in the case of a small sample size; it
may be biased and nonefficient. This lack of reasonable small-sample proper-
ties can be explained in part by the fact that maximum likelihood estimation is
based on finding the mode of a distribution by attempting to select the true
parameter value. Estimators, in contrast, are generally designed to approach
the true value rather than to produce an exact hit. Modes are therefore not as
desirable as the mean or median when the sample size is small.
^
Property 9.2: invariance property. It can be shown that, if is the MLE of ,
^

then the MLE of a function of , say g( ), is g( ), where g( ) is assumed to
represent a one-to-one transformation and be differentiable with respect to .
^
This important invariance property implies that, for example, if is the
MLE of the standard deviation in a distribution, then the MLE of the
^ 2
2 c 2
variance , ,is .
Let us also make an observation on the solution procedure for solving like-
lihood equations. Although it is fairly simple to establish Equation (9.99) or
Equations (9.100), they are frequently highly nonlinear in the unknown estimates,
and close-form solutions for the MLE are sometimes difficult, if not impossible,
to achieve. In many cases, iterations or numerical schemes are necessary.
Example 9.15. Let us consider Example 9.9 again and determine the MLEs of
2

m and . The logarithm of the likelihood function is
n
1 X 2 1 1
2
ln L ˆ x j m† n ln n ln 2 : 9:105†
2 2 2 2
jˆ1
Let 1 ˆ m , and 2 ˆ 2 , as before; the likelihood equations are
n
q ln L ˆ 1 X x j 1 †ˆ 0;
^
^ ^
q 1 2 jˆ1
n
q ln L 1 X ^ 2 n
ˆ x j 1 † ˆ 0:
^ ^ 2 ^
q 2 2 2 jˆ1 2 2

TLFeBOOK