Page 201 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

190 FEATURE EXTRACTION AND SELECTION

A performance measure more suited to express the separability of
classes is the ratio between interclass and intraclass distance:

J INTER traceðS b Þ
¼ ð6:9Þ
J INTRA traceðS w Þ

This measure possesses some of the desired properties of a performance
measure. In Figure 6.2, the numerator, trace(S b ), measures the area of
the ellipse associated with S b . As such, it measures the fluctuations of the
conditional expectations around the overall expectation, i.e. the fluctu-
ations of the ‘signal’. The denominator, trace(S w ), measures the area of
the ellipse associated with S w . As such, it measures the fluctuations due
to noise. Therefore, trace(S b )/trace(S w ) can be regarded as a ‘signal-
to-noise ratio’.
Unfortunately, the measure of (6.9) oversees the fact that the ellipse
associated with the noise can be quite large, but without having a large
intersection with the ellipse associated with the signal. A large S w can be
quite harmless for the separability of the training set. One way to correct
this defect is to transform the measurement space such that the within-
scattering becomes white, i.e. S w ¼ I. For that purpose, we apply a linear
operation to all measurement vectors yielding feature vectors y ¼ Az n .
n
In the transformed space the within- and between-scatter matrices
T
T
become: AS w A and AS b A , respectively. The matrix A is chosen such
T
that AS w A ¼ I.
T
The matrix A can be found by factorization: S w ¼ V V where L is a
diagonal matrix containing the eigenvalues of S w , and V a unitary matrix
containing the corresponding eigenvectors; see appendix B.5. With this
T
factorization it follows that A ¼ 1/2 V . An illustration of the process
T
is depicted in Figure 6.2. The operation V performs a rotation that
aligns the axes of S w . It decorrelates the noise. The operation 1/2 scales
the axes. The normalized within-scatter matrix corresponds with a circle
with unit radius. In this transformed space, the area of the ellipse
associated with the between-scatter is a useable performance measure:

1 T 1

J INTER=INTRA ¼ traceð V S b V Þ
2
2
1 T
¼ traceðV V S b Þ ð6:10Þ
1
¼ traceðS S b Þ
w
This performance measure is called the inter/intra distance. It meets all
of our requirements stated above.