Page 165 - Elements of Distribution Theory

P1: JZP
052184472Xc05 CUNY148/Severini May 24, 2005 17:53

5.5 Regression Models 151

Then we may write

Y j = µ(x j ,θ) + j , j = 1,..., n
where j is simply Y j − µ(x j ,θ). If, for each j, the minimal support of the distribution of
j does not depend on the values of x j and θ,we call the model an additive error model.
By construction, 1 ,..., n are independent random variables each with mean 0 and with
a distribution depending on θ. Often additional assumptions are made about the distribution
of 1 ,..., n .For instance, it may be reasonable to assume that 1 ,..., n are identically
distributed with a specific distribution.

Example 5.21 (Linear models). Let Y 1 ,..., Y n denote independent scalar random vari-
ables such that each Y j follows an additive error model of the form
Y j = µ(x j ,θ) + j .

Suppose further that θ may be written (β, σ), where β is a vector and σ> 0isa scalar such
that µ(x j ; θ)isa linear function of β,

µ(x j ,θ) = x j β
and the distribution of j depends only on σ, the standard deviation of the distribution of
Y j . Thus, we may write
Y j = x j β + σz j , j = 1,..., n

where z 1 ,..., z n have known distributions. A model of this type is called a linear model;
when z 1 ,..., z n are assumed to be standard normal random variables, it is called a normal-
theory linear model.

Example 5.22 (Linear exponential family regression models). Let Y 1 ,..., Y n denote
independent scalar random variables such that Y j has an exponential family distribution
with model function of the form
T
p(y; λ j ) = exp λ T (y) − k(λ j ) h(y),
j
as discussed in Section 5.3. Suppose that
λ j = x j β, j = 1,..., n

where, as above, x 1 ,..., x n are fixed covariates and β is an unknown parameter. Hence, the
density, or frequency function, of Y 1 ,..., Y n is

n n n
T
T
exp β x T (y j ) − k(x j β) h(y j ).
j
j=1 j=1 j=1
This is called a linear exponential family regression model;itis also a special case of a
generalized linear model.
For instance, suppose that Y 1 ,..., Y n are independent Poisson random variables such
that Y j has mean λ j with log λ j = x j β. Then Y 1 ,..., Y n has frequency function

n 1
n
n

exp β T x j y j − exp(x j β) , y j = 0, 1, 2,... ; j = 1,..., n.
y j !
j=1 j=1 j=1