Page 224 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

6.4 REFERENCES 213

load license_plates.mat % Load dataset
figure; clf; show(z); % Display it
J ¼ fisherm(z,0); % Calculate criterion values
figure; clf; plot(J, ‘r.-’); % and plot them
w ¼ fisherm(z,24,0.9); % Calculate the feature extractor
figure; clf; show(w); % Show the mappings as images

The two-class case, Fisher’s linear discrimant

1 T T
m
m
m
m
m
m
m
m
S b ¼ N 1 ð^ m ^ mÞð^ m ^ mÞ þ N 2 ð^ m ^ mÞð^ m ^ mÞ
2
1
2
1
N S ð6:51Þ
m m m m T
¼ ð^ m ^ m Þð^ m ^ m Þ
2
1
1
2
where is a constant that depends on N 1 and N 2 . In the transformed space,
T
m
m
T
m
m
S b becomes 1/2 V (^ m ^ m )(^ m ^ m ) V 1/2 . This matrix has only one
1 2 1 2
m
m
m
T 1
m
eigenvector with a nonzero eigenvalue
0 ¼ (^ m ^ m ) S b (^ m ^ m ).
1
2
1
2
m
T
m
The corresponding eigenvector is u ¼ 1/2 V (^ m ^ m ). With that, the
2
1
feature extractor evolves into (see (6.49)):
1
T
W ¼ u V T

2
T

1 1
2 T m m V T ð6:52Þ

2
¼ V ð^ m ^ m Þ
2
1
T 1
m
m
¼ð^ m ^ m Þ S w
2
1
This solution – known as Fisher’s linear discriminant (Fisher, 1936) – is
similar to the Bayes classifier for Gaussian random vectors with class-
independent covariance matrices; i.e. the Mahalanobis distance clas-
sifier. The difference with expression (2.41) is that the true covariance
matrix and the expectation vectors have been replaced with estimates
from the training set. The multi-class solution (6.49) can be regarded as
a generalization of Fisher’s linear discriminant.
6.4 REFERENCES
Bhattacharyya, A., On a measure of divergence between two statistical populations
defined by their probability distributions. Bulletin of the Calcutta Mathematical
Society, 35, 99–110, 1943.
Chernoff, H., A measure of asymptotic efficiency for tests of a hypothesis based on the
sum of observations. Annals of Mathematical Statistics, 23, 493–507, 1952.