Page 151 - Elements of Distribution Theory

P1: JZP
052184472Xc05 CUNY148/Severini May 24, 2005 17:53

5.3 Exponential Family Models 137

Then
p n+1 (Y 1 ,..., Y n , y; θ 1 ) p n+1 (Y 1 ,..., Y n , y; θ 0 )

∞
E(X n+1 |Y 1 ,..., Y n ; θ 0 ) = dy
−∞ p n+1 (Y 1 ,..., Y n , y; θ 0 ) p n (Y 1 ,..., Y n ; θ 0 )
p n+1 (Y 1 ,..., Y n , y; θ 1 )
∞

= dy
p n (Y 1 ,..., Y n ; θ 0 )
−∞
p n (Y 1 ,..., Y n ; θ 1 )
=
p n (Y 1 ,..., Y n ; θ 0 )
= X n .
Since (X 1 ,..., X n )isa function of (Y 1 ,..., Y n ),
E(X n+1 |X 1 ,..., X n ; θ 0 ) = E[E(X n+1 |X 1 ,..., X n , Y 1 ,..., Y n ; θ 0 )|X 1 ,..., X n ; θ 0 ]
= E[E(X n+1 |Y 1 ,..., Y n ; θ 0 )|X 1 ,..., X n ; θ 0 ]
= E[X n |X 1 ,..., X n ; θ 0 ] = X n .
It follows that X 1 , X 2 ,... is a martingale.

5.3 Exponential Family Models
Many frequently used families of distributions have a common structure. Consider a family
d
of disributions on R , {P(·; θ): θ ∈ }, such that each distribution in the family is either
absolutely continuous or discrete with support not depending on θ.For each θ, let p(·; θ)
denote either the density function or frequency function corresponding to P(·; θ). The family
of distributions is said to be an m-parameter exponential family if each p(·; θ) may be written
T
p(y; θ) = exp{c(θ) T (y) − A(θ)}h(y), y ∈ Y (5.1)
m
m
d
where Y ⊂ R , c : → R , T : Y → R , A : → R, and h : Y → R .Itis important
+
to note that the representation (5.1) is not unique; for example, we may replace c(θ)by
c(θ)/2 and T (y)by2T (y).
Example 5.9 (Normal distributions). Let Y denote a random variable with a normal dis-
tribution with mean µ, −∞ <µ< ∞ and standard deviation σ> 0; then Y has density
function
1 1 2
√ exp − (y − µ) , −∞ < y < ∞.
σ (2π) 2σ 2
Hence, θ = (µ, σ) and = R × R . This density may be written
+
1 2 µ 1 µ 1
2
exp − y + y − − log σ , y ∈ R.
2σ 2 σ 2 2 σ 2 (2π) 2 1
2
This is of the form (5.1) with T (y) = (y , y),

1 µ
c(θ) = − 2 , 2 ,θ = (µ, σ),
2σ σ
2
2
A(θ) = µ /(2σ ) − log σ, h(y) = (2π) − 1 2 , and Y = R. Hence, this is a two-parameter
exponential family distribution.