Page 58 - Introduction to Statistical Pattern Recognition

40 Introduction to Statistical Pattern Recognition

Thus, (U@)nx,,, and A,,,,,,, are the m eigenvectors and eigenvalues of
n
S = (UUT)nx,,/m. The other (n - m) eigenvalues are all zero and their eigenvectors
are indefinite. The advantage of this calculation is that only an m xm matrix is
used for calculating m eigenvalues and eigenvectors. The matrix (U@),,,,,
represents orthogonal vectors but not orthonormal ones. In order to obtain ortho-
normal vectors Vi, we have to divide each column vector of (U(D)f,ml by (mhi)1'2
as
1
or
v. = - V,,, = -(U@II-"~)~~,,, , (2.149)
UQi
' (mhi)1'2
112
because, from (2.147),

Near-singular matrix: in many pattern recognition problems, n may be
very large, for example 100. However, only a few eigenvalues, such as 10, are
dominant, so that

h, + . . . + h, a1 + . . . + hp (k <<n) . (2.151)
This means that in a practical sense we are handling X (or S ) with rank k, even
though the mathematical rank of C is still n. Therefore, it is very inefficient to use
an n x n matrix to find k eigenvalues and eigenvectors, even when we have a sam-
ple size greater than n. In addition to this inefficiency, we face some computa-
tional difficulty in handling a large, near-singular matrix. For example, let us con-
sider the calculation of X-' or I C I. The determinant 1 Z I is n:=, hi and (n - k)
S;'sareveryclose tozero. If we haven = 100,k= 10, and hl + . . . +hIO=0.90ut
ofhl+ ...+ hloo=l, IZlbecomes

nfo hi x n!O0 h. = n!' hi x (0.1190)90 E n,!:,hi x
/=I
J=II
J
r=l
fortheassumptionofhI1 =hl2 = . . . =Aloo.