Page 14 -

xii

14.5.2 Technique: Decision Trees and Random Forests . . . . . . . . 448
14.5.3 Labeling Pixels . . . . . . . . . . . . . . . . . . . . . . . . . . 450
14.5.4 Computing Joint Positions . .. ... .. ... .. .. ... . 453
14.6 Notes . . . . . . .. .. .. ... .. .. ... .. ... .. .. ... . 453

15 Learning to Classify 457
15.1 Classification, Error, and Loss . . . . . . . . . . . . . . . . . . . . . . 457
15.1.1 Using Loss to Determine Decisions . . . . . . . . . . . . . . . 457
15.1.2 Training Error, Test Error, and Overfitting . . . . . . . . . . 459
15.1.3 Regularization . . . . .. .. .. ... .. ... .. .. ... . 460
15.1.4 Error Rate and Cross-Validation . . . . . . . . . . . . . . . . 463
15.1.5 Receiver Operating Curves . . . . . . . . . . . . . . . . . . . 465
15.2 Major Classification Strategies . .. .. ... .. ... .. .. ... . 467
15.2.1 Example: Mahalanobis Distance ... .. ... .. .. ... . 467
15.2.2 Example: Class-Conditional Histograms and Naive Bayes . . 468
15.2.3 Example: Classification Using Nearest Neighbors . . . . . . . 469
15.2.4 Example: The Linear Support Vector Machine . . . . . . . . 470
15.2.5 Example: Kernel Machines . . . . . . . . . . . . . . . . . . . 473
15.2.6 Example: Boosting and Adaboost . . . . . . . . . . . . . . . 475
15.3 Practical Methods for Building Classifiers . . . . . . . . . . . . . . . 475
15.3.1 Manipulating Training Data to Improve Performance . . . . . 477
15.3.2 Building Multi-Class Classifiers Out of Binary Classifiers . . 479
15.3.3 Solving for SVMS and Kernel Machines . . . . . . . . . . . . 480
15.4 Notes . . . . . . .. .. .. ... .. .. ... .. ... .. .. ... . 481

16 Classifying Images 482
16.1 Building Good Image Features . . . . . . . . . . . . . . . . . . . . . 482
16.1.1 Example Applications . . . . . . . . . . . . . . . . . . . . . . 482
16.1.2 Encoding Layout with GIST Features . . . .. .. .. ... . 485
16.1.3 Summarizing Images with Visual Words . . . . . . . . . . . . 487
16.1.4 The Spatial Pyramid Kernel . . . . . . . . . . . . . . . . . . . 489
16.1.5 Dimension Reduction with Principal Components . . . . . . . 493
16.1.6 Dimension Reduction with Canonical Variates . . . . . . . . 494
16.1.7 Example Application: Identifying Explicit Images . . . . . . 498
16.1.8 Example Application: Classifying Materials . . . . . . . . . . 502
16.1.9 Example Application: Classifying Scenes . . . . . . . . . . . . 502
16.2 Classifying Images of Single Objects . . ... .. ... .. .. ... . 504
16.2.1 Image Classification Strategies . ... .. ... .. .. ... . 505
16.2.2 Evaluating Image Classification Systems . . . . . . . . . . . . 505
16.2.3 Fixed Sets of Classes . .. .. .. ... .. ... .. .. ... . 508
16.2.4 Large Numbers of Classes . . . . . . . . . . . . . . . . . . . . 509