Page 53 - Introduction to Statistical Pattern Recognition

2 Random Vectors and their Properties 35

2.4 Various Properties of Eigenvalues and Eigenvectors

As we saw in the diagonalization processes, the eigenvalues and eigenvec-
tors of symmetric matrices play an important role. In this section, we review vari-
ous properties of eigenvalues and eigenvectors, which will simplify discussions in
later chapters. Most of the matrices we will be dealing with are covariance and
autocorrelation matrices, which are symmetric. Therefore, unless specifically
stated, we assume that matrices are symmetric, with real eigenvalues and eigen-
vectors.

Orthonormal Transformations

Theorem An eigenvalue matrix A is invariant under any orthonormal linear
transformation.
Proof Let A be an orthonormal transformation matrix and let it satisfy
ATA =I or =A-' . (2.1 19)

By this transformation, Q is converted to A TQA [see (2.63)]. If the eigenvalue and
eigenvector matrices of A 'QA are A and Q,,
QT(ATQA)Q, = A , (2.120)

(AQ,)TQ(AcD) = A . (2.121)

Thus, A and A@ should be the eigenvalue and eigenvector matrices of Q. This
transformation matrix A Q, satisfies the orthonormal condition as

= Q,'A'AQ,
(AQ,~(AQ,) = aTcg =I . (2.122)

Positive Definiteness

Theorem If all eigenvalues are positive, Q is a positive definite matrix.

Proof Consider a quadratic form

d2 =XTQX. (2.123)

We can rewrite X as @Y, where Q, is the eigenvector matrix of Q. Then