Page 8 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

CONTENTS vii

5.2.2 Gaussian distribution, covariance matrix
unknown 144
5.2.3 Gaussian distribution, mean and covariance
matrix both unknown 145
5.2.4 Estimation of the prior probabilities 147
5.2.5 Binary measurements 148
5.3 Nonparametric learning 149
5.3.1 Parzen estimation and histogramming 150
5.3.2 Nearest neighbour classification 155
5.3.3 Linear discriminant functions 162
5.3.4 The support vector classifier 168
5.3.5 The feed-forward neural network 173
5.4 Empirical evaluation 177
5.5 References 181
5.6 Exercises 181

6 Feature Extraction and Selection 183
6.1 Criteria for selection and extraction 185
6.1.1 Inter/intra class distance 186
6.1.2 Chernoff–Bhattacharyya distance 191
6.1.3 Other criteria 194
6.2 Feature selection 195
6.2.1 Branch-and-bound 197
6.2.2 Suboptimal search 199
6.2.3 Implementation issues 201
6.3 Linear feature extraction 202
6.3.1 Feature extraction based on the
Bhattacharyya distance with Gaussian
distributions 204
6.3.2 Feature extraction based on inter/intra
class distance 209
6.4 References 213
6.5 Exercises 214

7 Unsupervised Learning 215
7.1 Feature reduction 216
7.1.1 Principal component analysis 216
7.1.2 Multi-dimensional scaling 220
7.2 Clustering 226
7.2.1 Hierarchical clustering 228
7.2.2 K-means clustering 232