Page 283 - Fundamentals of Probability and Statistics for Engineers

266 Fundamentals of Probability and Statistics for Engineers
9.2.2 MINIMUM VARIANCE

^
It seems natural that, if ˆ h(X 1 , X 2 ,..., X n ) is to qualify as a good estimator

for , not only its mean should be close to true value but also there should be a
^

good probability that any of its observed values will be close to . This can be
^
achieved by selecting a statistic in such a way that not only is unbiased but
also its variance is as small as possible. Hence, the second desirable property is
one of minimum variance.
^

Definition 9.1. let be an unbiased estimator for . It is an unbiased

minimum-variance estimator for if, for all other unbiased estimators of
from the same sample,
^

varf g varf g; 9:24†
for all .
Given two unbiased estimators for a given parameter, the one with smaller
variance is preferred because smaller variance implies that observed values of
the estimator tend to be closer to its mean, the true parameter value.
Example 9.1. Problem: we have seen that X obtained from a sample of size n
is an unbiased estimator for population mean m. Does the quality of X improve
as n increases?
Answer: we easily see from Equation (9.5) that the mean of X is independent
of the sample size; it thus remains unbiased as n increases. Its variance, on the
other hand, as given by Equation (9.6) is

2
varfXgˆ ; 9:25†
n

which decreases as n increases. Thus, based on the minimum variance criterion,
the quality of X as an estimator for m improves as n increases.

Example 9.2. Part 1. Problem: based on a fixed sample size n, is X the best

estimator for m in terms of unbiasedness and minimum variance?
Approach: in order to answer this question, it is necessary to show that the
variance of X as given by Equation (9.25) is the smallest among all unbiased
estimators that can be constructed from the sample. This is certainly difficult to
do. However, a powerful theorem (Theorem 9.2) shows that it is possible to
determine the minimum achievable variance of any unbiased estimator
obtained from a given sample. This lower bound on the variance thus permits
us to answer questions such as the one just posed.

TLFeBOOK