Page 284 - Applied statistics and probability for engineers

262 Chapter 7/Point Estimation of Parameters and Sampling Distributions

Properties of the Maximum Likelihood Estimator
As noted previously, the method of maximum likelihood is often the estimation method that
we prefer because it produces estimators with good statistical properties. We summarize these
properties as follows.

Properties of a
Maximum Likelihood Under very general and not restrictive conditions when the sample size n is large and
Estimator if Θ is the maximum likelihood estimator of the parameter θ,
ˆ
ˆ
ˆ
(1) Θ is an approximately unbiased estimator for θ[ ( )]E Θ ≃ θ,
ˆ
(2) The variance of Θ is nearly as small as the variance that could be obtained with
any other estimator.
ˆ
(3) Θ has an approximate normal distribution.

Properties 1 and 2 essentially state that the maximum likelihood estimator is approximately
an MVUE. This is a very desirable result and, coupled with the facts that it is fairly easy to
obtain in many situations and has an asymptotic normal distribution (the “asymptotic” means
“when n is large”), explains why the maximum likelihood estimation technique is widely
used. To use maximum likelihood estimation, remember that the distribution of the population
must be either known or assumed.
To illustrate the “large-sample” or asymptotic nature of these properties, consider the maxi-
2
mum likelihood estimator for σ , the variance of the normal distribution, in Example 7-13.
It is easy to show that
)
E( ˆ σ = n −1 σ 2
2
n
The bias is
E( ˆ σ 2 )− σ = n −1 σ − σ = −È 2
2
2
2
n n
2
2
Because the bias is negative, ˆ σ tends to underestimate the true variance σ . Note that the bias
2
2
approaches zero as n increases. Therefore, ˆ σ is an asymptotically unbiased estimator for σ .
We now give another very important and useful property of maximum likelihood estimators.
Invariance Property
ˆ
ˆ
ˆ
…
1 ,
Let Θ Θ 2 , , Θ k be the maximum likelihood estimators of the parameters θ 1 , θ 2 , …,
θ k . Then the maximum likelihood estimator of any function h( , θ … θ k ) of these
,
,
θ 1
2
ˆ ˆ , ,Θ ˆ ˆ … ˆ
ˆ
…
1 ,
parameters is the same function h(Θ 1 ,Θ 2 k ) of the estimators Θ Θ 2 , , Θ k .
Example 7-14 In the normal distribution case, the maximum likelihood estimators of μ and σ were ˆ μ = X and
2
n
2
X
ˆ σ = Σ (X − ) 2
i =1 i /n, respectively. To obtain the maximum likelihood estimator of the function
h ( μ , σ ) = σ = σ, substitute the estimators ˆ μ and ˆ σ into the function h, which yields
2
2
2
n
2
ˆ σ = σ = ⎡1 ∑ (X i − ) ⎤ ⎥ 1 2 /
2
ˆ
X
⎢
⎣ n i =1 ⎦
Conclusion: The maximum likelihood estimator of the standard deviation σ is not the sample standard deviation S.