Page 151 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

140 SUPERVISED LEARNING

(a) (b)

bolts 1
nuts
1 scrap 0.8
measure of eccentricity 0.8 measure of eccentricity 0.6
rings

0.6
0.4
0.4
0.2
0.2
0
0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
measure of six-fold rotational symmetry measure of six-fold rotational symmetry

Figure 5.1 Training sets. (a) Labelled. (b) Unlabelled

The chapter starts with a section on the representation of training sets.
In Sections 5.2 and 5.3 two approaches to supervised learning are dis-
cussed: parametric and nonparametric learning. Section 5.4 addresses
the problem of how to evaluate a classifier empirically. The discussion
here is restricted to classification problems only. However, many tech-
niques that are useful for classification problems are also useful for
estimation problems. Especially Section 5.2 (parametric learning) is
useful for estimation problems too.

5.1 TRAINING SETS

The set of samples is usually called the training set (or: learning data or
design set). The selection of samples should occur randomly from the
population. In almost all cases it is assumed that the samples are i.i.d.,
independent and identically distributed. This means that all samples are
selected from the same population of objects (in the simplest case, with
equal probability). Furthermore, the probability of one member of the
population being selected is not allowed to depend on the selection of
other members of the population.
Figure 5.1 shows scatter diagrams of the mechanical parts application
of Chapter 2. In Figure 5.1(a) the samples are provided with a label
carrying the information of the true class of the corresponding object.
There are several methods to find the true class of a sample, e.g. manual
inspection, additional measurements, destructive analysis, etc. Often,