Page 277 - Applied Statistics And Probability For Engineers

c07.qxd 5/15/02 10:18 M Page 235 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH114 FIN L:Quark Files:

7-3 METHODS OF POINT ESTIMATION 235

To illustrate the “large-sample” or asymptotic nature of the above properties, consider the
2
maximum likelihood estimator for , the variance of the normal distribution, in Example 7-9.
It is easy to show that

n 1
2
E1 ˆ 2 2
n
The bias is

n 1 2
2
2
2
2
E1 ˆ 2
n n
2
Because the bias is negative, ˆ 2 tends to underestimate the true variance . Note that the
bias approaches zero as n increases. Therefore, ˆ 2 is an asymptotically unbiased estimator
2
for .
We now give another very important and useful property of maximum likelihood
estimators.

The Invariance
ˆ
ˆ
ˆ
Property Let , , p , k be the maximum likelihood estimators of the parameters ,
1
2
1
, p , . Then the maximum likelihood estimator of any function h( , , p , )
k
1
k
2
2
ˆ
ˆ
ˆ
of these parameters is the same function h1 , , p , 2 of the estimators
1
2
k
ˆ
ˆ
ˆ
, , p , k .
2
1
2
EXAMPLE 7-10 In the normal distribution case, the maximum likelihood estimators of and were ˆ X
n
2
2
and ˆ g i 1 1X X 2 n . To obtain the maximum likelihood estimator of the function
i
2
2
h1 , 2 2 , substitute the estimators and ˆˆ 2 into the function h, which yields

1 n 2 1 2
2
n a
ˆ 2 ˆ c 1X i X 2 d
i 1
Thus, the maximum likelihood estimator of the standard deviation is not the sample
standard deviation S.
Complications in Using Maximum Likelihood Estimation
While the method of maximum likelihood is an excellent technique, sometimes complications
arise in its use. For example, it is not always easy to maximize the likelihood function because
the equation(s) obtained from dL 1 2 d 0 may be difficult to solve. Furthermore, it may
not always be possible to use calculus methods directly to determine the maximum of L( ).
These points are illustrated in the following two examples.
EXAMPLE 7-11 Let X be uniformly distributed on the interval 0 to a. Since the density function is f 1x2 1 a
for 0 x a and zero otherwise, the likelihood function of a random sample of size n is
n 1 1
L1a2
q a n
i 1 a