Page 153 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

142 SUPERVISED LEARNING

dat ¼ [ 0.1 0.9 ; 0.3 0.95 ; 0.2 0.7 ];
lab ¼ { ‘class 1’, ‘class 2’, ‘class 3’ };
z ¼ dataset(dat,lab);
% Method (5.3):
[nlab,lablist] ¼ getnlab(z); % Extract the numeric labels
[m,k,c] ¼ getsize(z); % Extract number of classes
for i ¼ 1:c
T{i} ¼ seldat(z,i);
end;

5.2 PARAMETRIC LEARNING

The basic assumption in parametric learning is that the only unknown
factors are parameters of the probability densities involved. Thus,
learning from samples boils down to finding the suitable values of these
parameters. The process is analogous to parameter estimation discussed
in Chapter 3. The difference is that the parameters in Chapter 3
describe a physical process whereas the parameters discussed here are
parameters of the probability densities of the measurements of the
objects. Moreover, in parametric learning a set of many measurement
vectors is available rather than just a single vector. Despite these two
differences, the concepts from Chapter 3 are fully applicable to the
current chapter.
Suppose that z n are the samples coming from a same class ! k . These
samples are repeated realizations of a single random vector z. An alter-
native view is to associate the samples with single realizations coming
from a set of random vectors with identical probability densities. Thus, a
training set T k consists of N k mutually independent, random vectors z n .
The joint probability density of these vectors is

N k
Y
j! k ; a k Þ¼ pðz n j! k ; a k Þ ð5:5Þ
pðz 1 ; z 2 ; .. . ; z N k
n¼1
a k is the unknown parameter vector of the conditional probability
density p(zj! k , a k ). Since in parametric learning we assume that the form
of p(zj! k , a k ) is known (only the parameter vector a k is unknown),
the complete machinery of Bayesian estimation (minimum risk, MMSE
estimation, MAP estimation, ML estimation) becomes available to find
estimators for the parameter vector a k : see Section 3.1. Known concepts
to evaluate these estimators (bias and variance) also apply. The next
subsections discuss some special cases for the probability densities.