Page 55 -

2.4 Principal Components 41

variance of one feature if the total variance was equally distributed. For the cork
stoppers data this would correspond to retaining only the first 2 eigenvalues. It can
also be useful to inspect a plot of the eigenvalues as illustrated in Figure 2.17. A
criterion based on such a plot (called scree lest) suggests discarding the
eigenvalues starting where the plot levels off, which, in this case, would amount to
retaining only the first 4 eigenvalues.
Principal components analysis as a dimensionality reduction method must be
applied with caution, for the follow~ng reasons:

- Principal components are linear trunsfbrmations of the original features,
therefore, reduction to significant principal components may not appropriately
reflect non-linearities that may be present in the data (see e.g. chapter eight of
Bishop, 1995).
- Principal components with negligible contribution to the overall variance may
nevertheless provide a crucial contribution to pattern discrimination. By
discarding such components we may inadvertently impair the classification or
regression performance (idem).
- It is usually difficult to attach any semantic meaning to principal components.
Such semantic meaning, provided by the original features, is often useful when
developing and applying classification or regression solutions.

Although principal components analysis suffers from these shortcomings there
is still something to be gained from it since it provides the designer with desirable
low dimensional representations of the original data, to be explored further in the
next chapter. It also provides meaningful estimates of the intrinsic data
dimensionality, which can sonlehow serve as reference for more sophisticated
feature selection methods, to be explained in chapter four.

2.5 Feature Assessment

Assessing the discriminative capability of features is an important task either in the
initial phase of a PR project or in a more advanced phase, for example, when
choosing between alternative feature sets. This assessment is usually of great help
in guiding subsequent project phases by giving some insight in what is going on in
the feature space, e.g., concerning class separability.
The feature assessment task is usually conducted in the following sequence of
subtasks:

1. Graphic inspection
2. Distribution model assessment
3. Statistic inference tests

We will illustrate now these subtasks using the cork stoppers data. We assume
that the reader is familiar with the statistical techniques used, which are described