Page 54 - Introduction to Statistical Pattern Recognition

36 Introduction to Statistical Pattern Recognition

n
d2 = (@Y)'Q(@Y) = Y'QTQ@Y = Y'AY = zh;y? , (2.124)
i=l
where the hi's are the eigenvalues of Q. If these eigenvalues are all positive, then
d is positive, unless Y is a zero vector. From the relation between Y andX, we see
that d must be positive for all nonzeroX as well. Therefore, Q is positive definite.
When Q is a covariance or autocorrelation matrix, the hi's are the variances
or second order moments after the orthonormal transformation to diagonalize Q.
Therefore, all hi's should be positive for both cases, and both covariance and auto-
correlation matrices are positive definite.

Trace

Theorem The trace of Q is the summation of all eigenvalues and is invariant
under any orthonormal transformation. That is,
n
trQ = . (2.125)
i=l

Proof First for general rectangular matrices A,,,,, and B,,,,, ,

(2.126)

because

(2.127)

where aij and bji are the components of A,,,, and B,,,,, . Using (2.126),
n
zhi = trh = tr(@Q@) = tr(QWD') = trQ . (2.128)
i=l
As we proved before, the eigenvalues are invariant under any orthonormal
transformation. Therefore, any function of eigenvalues is also invariant.
When Q is a covariance or autocorrelation matrix, the above theorem states
that the summation of the variances or second order moments of individual vari-
ables is invariant under any orthonormal transformation.