Page 253 - Elements of Distribution Theory

P1: JZP
052184472Xc08 CUNY148/Severini May 24, 2005 17:54

8.2 Multivariate Normal Distribution 239

Let (λ 1 , e 1 ),..., (λ d , e d ) denote the eigenvalue–eigenvector pairs of , λ 1 ≥ λ 2 ≥ ··· ≥
λ d ,so that
T
T
= λ 1 e 1 e + ··· + λ d e d e .
1
d
Note we may write a = c 1 e 1 + ··· c d e d for some scalar constants c 1 , c 2 ,..., c d ; since
T
2
2
a a = 1, it follows that c +· · · + c = 1.
1 d
Hence,
T
2
2
a a = λ 1 c + ··· + λ d c ,
1
d
2 2 2 2
which is maximized, subject to the restriction c +· · · + c = 1, by c = 1, c = ··· =
1 d 1 2
2
T
c = 0. That is, the variance of a X is maximized by taking a to be the eigenvector
d
T
corresponding to the largest eigenvalue of ; a X is called the first principal component
of X.
Example8.4 (Themultivariatenormaldistributionasatransformationmodel). Consider
d
the class of multivariate normal distributions with mean vector µ ∈ R and covariance
matrix , where is an element of the set of all d × d positive-definite matrices; we will
denote this set by C d .
d
For A ∈ C d and b ∈ R let
(A, b)X = AX + b.
d
Consider the set of transformations G of the form (A, b) with A ∈ C d and b ∈ R . Since
(A 1 , b 1 )(A 0 , b 0 )X = A 1 (A 0 X + b 0 ) + b 1 = A 1 A 0 X + A 1 b 0 + b 1 ,
define the operation
(A 1 , b 1 )(A 0 , b 0 ) = (A 1 A 0 , A 1 b 0 + b 1 ).
It is straightforward to show that G is a group with respect to this operation. The identity
element of the group is (I d , 0) and the inverse operation is given by
−1
−1
(A, b) −1 = (A , −A b).
If X has a multivariate normal distribution with mean vector µ and covariance matrix
, then, by Theorem 8.1, (A, b)X has a multivariate normal distribution with mean vector
T
Aµ + b and positive definite covariance matrix A A . Clearly, the set of all multivariate
d
normal distributions with mean vector µ ∈ R and covariance matrix ∈ C d is invariant
with respect to G.
As discussed in Section 5.6, G may also be viewed as acting on the parameter space of
d
the model C d × R ; here
T
(A, b)( , µ) = (A A , Aµ + b).
Density of the multivariate normal distribution
For the case in which the covariance matrix is positive-definite, it is straightforward to
derive the density function of the multivariate normal distribution.

Theorem 8.2. Let X be a d-dimensional random vector with a multivariate normal dis-
tribution with mean µ and covariance matrix .If | | > 0 then the distribution of X is