Page 164 - Elements of Distribution Theory

P1: JZP
052184472Xc05 CUNY148/Severini May 24, 2005 17:53

150 Parametric Families of Distributions

Suppose that λ is distributed according to a normal distribution with mean µ and standard
deviation τ. Note that X 1 has characteristic function
1 2 2

E[exp(it X 1 )] = E{E[exp(it X 1 |λ)]}= E exp itλ − σ t
2
;

1 2 2 2
= exp itµ − (τ + σ )t
2
it follows that X 1 is distributed according to a normal distribution with mean µ and variance
2
2
τ + σ . Clearly, X 2 has the same marginal distribution as X 1 .
However, X 1 and X 2 are no longer independent. Since
2
2
2
E(X 1 X 2 ) = E[E(X 1 X 2 |λ)] = E(λ ) = τ + µ ,
it follows that
Cov(X 1 , X 2 ) = τ 2
2
2
2
and, hence, that the correlation of X 1 and X 2 is τ /(τ + σ ). Hence, although, conditionally
on λ, X 1 and X 2 are independent, marginally they are dependent random variables.
The distribution of (X 1 , X 2 )is called the bivariate normal distribution; its properties
will be considered in detail in Chapter 8.

5.5 Regression Models
d
Consider a parametric model on Y ⊂ R , P ={P(·; λ): λ ∈ }. Suppose that Y 1 ,..., Y n
are independent random variables such that, for each j = 1,..., n, the distribution of Y j is
the element of P corresponding to a parameter value λ j . Hence, Y 1 ,..., Y n are independent,
but are not necessarily identically distributed.
Let x 1 , x 2 ,..., x n denote a known sequence of nonrandom vectors such that, for each
j = 1,..., n, there exists a function h such that

λ j = h(x j ; θ)
for some θ in a set . Thus, the distribution of Y j depends on the value of x j , along with the
value of θ and the function h. The vectors x 1 ,..., x n are known as covariates or explana-
tory variables; the random variables Y 1 ,..., Y n are called the response variables. The
response variables and covariates are sometimes called dependent and independent vari-
ables, respectively; however, those terms will not be used here, in order to avoid confusion
with the concept of independence, as discussed in Section 2.2.
In a regression model, the function h is known, while θ is an unknown parameter. Interest
generally centers on the relationship between Y j and x j , j = 1,..., n,asexpressed through
the function h.Regression models are very widely used in statistics; the goal of this section
is to present a few examples illustrating some of the regression models commonly used.

Example 5.20 (Additive error models). Suppose that each distribution in P has a finite
mean. Let

µ(x j ,θ) = E(Y j ; θ).