Page 49 - Introduction to Statistical Pattern Recognition

2 Random Vectors and their Properties 31

Simultaneous Diagonalization

We can diagonalize two symmetric matrices El and C2 simultaneously by a
linear transformation. The process is as follows:

(I) First, we whiten C, by
y = @-”2@Tx (2.93)
where 0 and @ are the eigenvalue and eigenvector matrices of E I as

El@=@@ and @‘@=I. (2.94)
Then, C, and C2 are transformed to

@-1/*@TZ, QQ-1’2 = (2.95)

@-l/2@TC2@Q-’/2 = K (2.96)
In general, K is not a diagonal matrix.

(2) Second, we apply the orthonormal transformation to diagonalize K.
That is,

Z=VY, (2.97)
where ‘i’ and A are the eigenvector and eigenvalue matrices of K as

KY=YA and YTY==I. (2.98)
As shown in (2.92), the first matrix I of (2.95) is invariant under this transforma-
tion. Thus,
Y’IY =YTY =I, (2.99)

=
YJ~KY A. (2.100)
Thus, both matrices are diagonalized. Figure 2-3 shows a two-dimensional
example of this process. The combination of steps (1) and (2) gives the overall
transformation matrix w-”~Y’.

Alternative approach: The matrices O@-”2\y and A can be calculated
directly from X, and C2 without going through the two steps above. This is done
as follows:

Theorem We can diagonalize two symmetric matrices as