Introduction to Pattern Recognition (eBook)

Front Cover 1
Title Page 4
Copyright Page 5
Table of Contents 6
Preface 10
Chapter 1. Classifiers Based on Bayes Decision Theory 14
1.1 Introduction 14
1.2 Bayes Decision Theory 14
1.3 The Gaussian Probability Density Function 15
1.4 Minimum Distance Classifiers 19
1.4.1 The Euclidean Distance Classifier 19
1.4.2 The Mahalanobis Distance Classifier 19
1.4.3 Maximum Likelihood Parameter Estimation of Gaussian pdfs 20
1.5 Mixture Models 24
1.6 The Expectation-Maximization Algorithm 26
1.7 Parzen Windows 32
1.8 k-Nearest Neighbor Density Estimation 34
1.9 The Naive Bayes Classifier 35
1.10 The Nearest Neighbor Rule 38
Chapter 2. Classifiers Based on Cost Function Optimization 42
2.1 Introduction 42
2.2 The Perceptron Algorithm 43
2.2.1 The Online Form of the Perceptron Algorithm 46
2.3 The Sum of Error Squares Classifier 48
2.3.1 The Multiclass LS Classifier 52
2.4 Support Vector Machines: The Linear Case 56
2.4.1 Multiclass Generalizations 61
2.5 SVM: The Nonlinear Case 63
2.6 The Kernel Perceptron Algorithm 71
2.7 The AdaBoost Algorithm 76
2.8 Multilayer Perceptrons 79
Chapter 3. Data Transformation: Feature Generation and Dimensionality Reduction 92
3.1 Introduction 92
3.2 Principal Component Analysis 92
3.3 The Singular Value Decomposition Method 97
3.4 Fisher’s Linear Discriminant Analysis 100
3.5 The Kernel PCA 105
3.6 Laplacian Eigenmap 114
Chapter 4. Feature Selection 120
4.1 Introduction 120
4.2 Outlier Removal 120
4.3 Data Normalization 121
4.4 Hypothesis Testing: The t-Test 124
4.5 The Receiver Operating Characteristic Curve 126
4.6 Fisher’s Discriminant Ratio 127
4.7 Class Separability Measures 130
4.7.1 Divergence 131
4.7.2 Bhattacharyya Distance and Chernoff Bound 132
4.7.3 Measures Based on Scatter Matrices 133
4.8 Feature Subset Selection 135
4.8.1 Scalar Feature Selection 136
4.8.2 Feature Vector Selection 137
Chapter 5. Template Matching 150
5.1 Introduction 150
5.2 The Edit Distance 150
5.3 Matching Sequences of Real Numbers 152
5.4 Dynamic Time Warping in Speech Recognition 156
Chapter 6. Hidden Markov Models 160
6.1 Introduction 160
6.2 Modeling 160
6.3 Recognition and Training 161
Chapter 7. Clustering 172
7.1 Introduction 172
7.2 Basic Concepts and Definitions 172
7.3 Clustering Algorithms 173
7.4 Sequential Algorithms 174
7.4.1 BSAS Algorithm 174
7.4.2 Clustering Refinement 175
7.5 Cost Function Optimization Clustering Algorithms 181
7.5.1 Hard Clustering Algorithms 181
7.5.2 Nonhard Clustering Algorithms 197
7.6 Miscellaneous Clustering Algorithms 202
7.7 Hierarchical Clustering Algorithms 211
7.7.1 Generalized Agglomerative Scheme 212
7.7.2 Specific Agglomerative Clustering Algorithms 213
7.7.3 Choosing the Best Clustering 216
Appendix 222
References 228
Index 230