Page 284 - Fundamentals of Probability and Statistics for Engineers

Parameter Estimation 267

Theorem 9.2: the Crame ´ r–Rao inequality. Let X 1 , X 2 ,... , X n denote a sample

of size n from a population X with pdf f (x; ), where is the unknown param-
^
eter, and let ˆ h(X 1 , X 2 ,..., X n ) be an unbiased estimator for . Then, the
^
variance of satisfies the inequality
)) 1
qlnf X; † 2
^
varf g nE ; 9:26†
q
if the indicated expectation and differentiation exist. An analogous result with

p(X ; ) replacing f (X ; ) is obtained when X is discrete.
Proof of Theorem9.2:the joint probability density function (jpdf) of X 1 , X 2 , . . .,
and X n is, because of their mutual independence, f x 1 ; ) f x 2 ; ) f x n ; ). The
^
^
mean of statistic , ˆ h X 1 , X 2 , .. . , X n ), is
^
Ef gˆ Efh X 1 ; X 2 ; ... ; X n †g;
^
and, since is unbiased, it gives
Z 1 Z 1
ˆ h x 1 ; ... ; x n † f x 1 ; † f x n ; † dx 1 dx n : 9:27†
1 1
Another relation we need is the identity:
1
Z
1 ˆ f x i ; † dx i ; i ˆ 1; 2; ... ; n: 9:28†
1
Upon differentiating both sides of each of Equations (9.27) and (9.28) with

respect to , we have
" #
1 1 n
Z Z
X 1 q f x j ; †
1 ˆ h x 1 ; ... ; x n † f x 1 ; † f x n ; † dx 1 dx n
1 1 jˆ1 f x j ; † q
9:30†
" #
Z 1 Z 1 n
X q ln f x j ; †
ˆ h x 1 ; ... ; x n † f x 1 ; † f x n ; † dx 1 dx n ;
1 1 jˆ1 q
1
Z q f x i ; †
0 ˆ dx i
1 q
9:30†
Z 1 q ln f x i ; †
ˆ f x i ; † dx i ; i ˆ 1; 2; ... ; n:
1 q

TLFeBOOK