Page 156 - Elements of Distribution Theory

P1: JZP
052184472Xc05 CUNY148/Severini May 24, 2005 17:53

142 Parametric Families of Distributions

Some distribution theory for exponential families
The importance of exponential family distributions lies in the way in which the parameter
of the model interacts with the argument of the density or frequency function in the model
function. For instance, if p(y; θ)isof the form (5.1) and θ 0 and θ 1 are two elements of the
parameter space, then log[p(y; θ 1 )/p(y; θ 0 )] is a linear function of T (y) with coefficients
depending on θ 0 ,θ 1 :
p(y; θ 1 ) T
log = A(θ 0 ) − A(θ 1 ) + [c(θ 1 ) − c(θ 0 )] T (y).
p(y; θ 0 )
This type of structure simplifies certain aspects of the distribution theory of the model,
particularly those aspects concerned with how the distributions change under changes in
parameter values. The following lemma gives a relationship between expectations under
two different parameter values.

Lemma 5.2. Let Y denote a random variable with model function of the form
T
exp{η T (y) − k(η)}h(y), y ∈ Y,
where η ∈ H.
Fix η 0 ∈ H and let g : Y → R. Then
T
E[g(Y); η] = exp{k(η 0 ) − k(η)}E[g(Y)exp{(η − η 0 ) T (Y)}; η 0 ]
for any η ∈ H such that
E[|g(Y)|; η] < ∞.

Proof. The proof is given for the case in which Y has an absolutely continuous distribution.
Suppose E[|g(Y)|; η] < ∞; then the integral

g(y)p(y; η) dy
Y
exists and is finite. Note that
p(y; η) p(Y; η)

g(y)p(y; η) dy = g(y) p(y; η 0 ) dy = E g(Y) ; η 0 ;
Y Y p(y; η 0 ) p(Y; η 0 )
The result now follows from the fact that

p(Y; η) T
= exp{k(η 0 ) − k(η)} exp{(η − η 0 ) T (Y)}.
p(Y; η 0 )

Consider a random variable Y with model function of the form
T
exp{c(θ) T (y) − A(θ)}h(y);
this function can be written as the product of two terms, the term given by the exponential
function and h(y). Note that only the first of these terms depends on θ and that term depends
on y only through T (y). This suggests that, in some sense, the dependence of the distribution
of Y on θ is primarily through the dependence of the distribution of T (Y)on θ. The following
two theorems give some formal expressions of this idea.