Page 222 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

LINEAR FEATURE EXTRACTION 211

corresponding to the D largest eigenvalues are collected in U D , being an
N D submatrix of U. Then, the linear feature extraction becomes:

T
W ¼ U 1=2 V T ð6:49Þ
D
The feature space defined by y ¼ Wz can be thought of as a linear
subspace of the measurement space. This subspace is spanned by the D
row vectors in W. The performance measure associated with this feature
space is:

D 1
X
J INTER=INTRA ðWzÞ¼
i ð6:50Þ
i¼0

Example 6.3 Feature extraction based on inter/intra distance
Figure 6.9(a) shows the within-scattering and between-scattering of
Example 6.1 after simultaneous decorrelation. The within-scattering
has been whitened. After that, the between-scattering is rotated such
that its ellipse is aligned with the axes. In this figure, it is easy to see
which axis is the most important. The eigenvalues of the between-
scatter matrix are
0 ¼ 56:3 and
1 ¼ 2:8, respectively. Hence, omit-
ting the second feature does not deteriorate the performance much.
The feature extraction itself can be regarded as an orthogonal
projection of samples on this subspace. Therefore, decision bound-
aries defined in the feature space correspond to hyperplanes orthog-
onal to the linear subspace, i.e. planes satisfying equations of the type
Wz ¼ constant.

A characteristic of linear feature extraction based on J INTER/INTRA is that
the dimension of the feature space found will not exceed K 1, where K
is the number of classes. This follows from expression (6.7), which
shows that S b is the sum of K outer products of vectors (of which one
vector linearly depends on the others). Therefore, the rank of S b cannot
exceed K 1. Consequently, the number of nonzero eigenvalues of S b
cannot exceed K 1 either. Another way to put this into words is that
the K conditional means m span a (K 1) dimensional linear subspace
k
N
in R . Since the basic assumption of the inter/intra distance is that
within-scattering does not convey any class information, any feature
extractor based on that distance can only find class information within
that subspace.