Page 397 - Probability and Statistical Inference

374 7. Point Estimation

7.5.3 A Generalization of the Cramér-Rao Inequality
In the statement of the Cramér-Rao inequality, Theorem 7.5.1, the random
variables X , ..., X do not necessarily have to be iid. The inequality holds with
1
n
minor modifications even if the X’s are not identically distributed but they are
independent.
Let us suppose that we have iid real valued observable random variables X ,
j1
..., X , from a population with the common pmf or pdf f (x; θ) where X ’s and
j
jn
j
X ’s are independent for all j ≠ l = 1, ..., k, x ∈ χ ⊆ ℜ. Here, we have the
l
unknown parameter θ ∈ Θ ⊆ ℜ. We denote X = (X , ..., X , ..., X , ..., X )
1n1
k1
11
knk
and X = (x , ..., x , ..., x , ..., x ) with x ∈ . Let
χ n
11 1n1 k1 knk
us pretend that we are working with the pdf’s and hence the expectations of
functions of the random variables would be written as appropriate multiple
integrals. In the case of discrete random variables, one will replace the inte-
grals by the corresponding finite or infinite sums, as the case may be. Let us
denote the information content of X by I (θ) which is calculated as E
j1 X3 ?
[( [logf (X;θ)]) ]. We now state a generalized version of the Theorem 7.5.1.
2
j
Its proof is left as Exercise 7.5.16.
χ
Standing Assumptions: Let us assume that the support does not in-
volve θ and the first partial derivative of f (x;θ), j = 1, ..., k with respect to θ
j
and the integrals with respect to X are interchangeable.
Theorem 7.5.4 Suppose that T = T(X) is an unbiased estimator of a real
valued parametric function T(θ), that is E (T) = T(θ) for all θ ∈ Θ. Assume
θ
also that T (θ), denoted by T′(θ), exists and is finite for all θ ∈ Θ. Then, for
all θ ∈ Θ, under the standing assumptions we have:
The expression on the rhs of the inequality in (7.5.18) is called the Cramér-
Rao lower bound (CRLB) as before.
Example 7.5.13 Let X , ..., X be iid Exponential(θ) so that their common
1n1
11
-1 -x/θ
pdf is given by f (x;θ) = θ e with the unknown parameter θ ∈ Θ = (0, ∞) and
1
χ = (0, ∞). Also, suppose that X , ..., X are iid Gamma(α, θ), that is, their
2n2
21
-1 -x/θ α-1
common pdf is given by f (x; θ) = {θ Γ(α)} e x with the same unknown
α
2
χ
parameter θ and = (0, ∞), but α(> 0) is assumed known. Let us assume that
the X ’s are independent of the X ’s. In an experiment on reliability and survival
2
1
analyses, one may have a situation like this where a combination of two or more
statistical models, depending on the same unknown parameter θ, may be ap-
propriate. By direct calculations we obtain I (θ) = θ and I (θ) = αθ . That
-2
-2
X1 X2