Page 57 - Introduction to Statistical Pattern Recognition

2 Random Vectors and their Properties 39

hI#O, hZ= ...= hn=O, (2.141)

-
ih;=h - tr(ATMMTA) = tr(M*AA*M) = M7X-'M , (2.142)
I
i=l
where AA * = C-' is obtained from (2. IOl), and tr(MTZ-'M) = MrC-'M because
MTZ-l M is a scalar. Thus,

IS1 = lZl(1 +MYM). (2.143)

Small sample size problem: When only m samples are available in an n-
A
dimensional vector space with m < n, the sample autocorrelation matrix S is calcu-
lated from the samples as

(2.144)

That is, is a function of m or less linearly independent vectors. Therefore, the
rank of ,? should be m or less. The same conclusion can be obtained for a sample
,.
covariance matrix. However, the (Xi - M)'s are not linearly independent, because
A
they are related by E? (Xi - M) = 0. Therefore, the rank of a sample covariance
I =I
matrix is (m - I) or less. This problem, which is called a small sample size prob-
lem, is often encountered in pattern recognition, particularly when n is very large.
For this type of problem, instead of calculating eigenvalues and eigenvectors from
an n x n matrix, the following procedure is more efficient [9].
Let XI, . . . , X, (m c n) be samples. The sample autocorrelation matrix of
these samples is

(2.145)

where U,,, is called a sample matrix and defined by
u = [XI . * Xnilnxm . (2.146)
.
Instead of using the n x n matrix i,lx, of (2.143, let us calculate the eigenvalues
and eigenvectors of an m x m matrix (U TU)mml as
1
-(~~~)mmtQmmi = Q'nixniAmxni . (2.147)
m
Multiplying U into (2.147) from the left side, we obtain