Visual Perception for Manipulation and Imitation in Humanoid Robots (eBook)

Foreword 6
Preface 8
Contents 10
1 Introduction 14
1.1 Motivation and Scope 14
1.2 Contribution 15
1.3 Outline 17
2 State of the Art in Object Recognition and Pose Estimation 19
2.1 Appearance-Based Methods 19
2.1.1 Global Approaches 20
2.1.1.1 Grayscale Correlation 20
2.1.1.2 Moments 21
2.1.1.3 The Viola-Jones Object Detector 22
2.1.1.4 Color Coocurrence Histograms 25
2.1.2 Local Approaches 28
2.1.2.1 Calculation of Feature Points 28
2.1.2.2 Matching Interest Points 29
2.1.2.3 Scale Invariant Feature Transform (SIFT) 31
2.1.2.4 Maximally Stable Extremal Regions 35
2.1.2.5 Object Recognition and Pose Estimation Frameworks Using Local Features 37
2.2 Model-Based Methods 39
2.2.1 Edge-Based Object Tracking 39
2.2.1.1 RAPiD Tracker 40
2.2.1.2 POSIT 44
2.2.1.3 2D-3D Tracking 46
2.2.1.4 Tracking Using Particle Filtering 47
2.2.2 Edge-Based Object Recognition 47
2.2.2.1 Generalized Hough Transform 47
2.2.2.2 Geometric Hashing 50
Translation 51
Similarity Transformation 51
Affine Transformation 52
2.2.2.3 Recognition Based on Perceptual Organization 52
2.2.3 Pose Estimation Based on Matched Feature Points 53
2.2.4 Object Recognition Based on 3D Point Clouds 55
2.2.5 Hybrid Approaches 57
2.3 Comparison 57
3 State of the Art in Human Motion Capture 60
3.1 VICON 61
3.2 Systems Using a Search Method 62
3.3 Systems Using a Minimization Method 63
3.3.1 Minimization Method for Articulated Objects 63
3.3.2 Systems Using a 3D-3D Minimization Method 66
3.3.3 Systems Using a 2D-3D Minimization Method 69
3.4 Systems Based on Particle Filtering 70
3.5 Pose Estimation Based on Silhouettes 73
3.6 Comparison 75
4 Fundamentals of Image Processing 78
4.1 Camera Model 78
4.1.1 Coordinate Systems 79
4.1.2 Intrinsic Camera Parameters of the Linear Mapping 80
4.1.3 Extrinsic Camera Parameters 81
4.1.4 Distortion Parameters 81
4.1.5 Overview 84
4.2 Segmentation 85
4.2.1 Thresholding 85
4.2.2 Background Subtraction 86
4.2.3 Color Segmentation 87
4.3 Correlation Methods 88
4.3.1 General Definition 89
4.3.2 Non-normalized Correlation Functions 90
4.3.3 Normalized Correlation Functions 90
4.4 Homography 92
4.4.1 General Definition 93
4.4.2 Least Squares Computation of Homography Parameters 93
4.5 Principal Component Analysis 95
4.5.1 Mathematical Definition 95
4.5.2 Eigenspace 96
4.5.3 Application 96
4.6 Particle Filtering 97
4.7 RANSAC 100
5 Guiding Principles 101
5.1 Ways of Using Calibrated Stereo Camera Systems 101
5.2 Eye Movements 102
5.3 Rectification of Stereo Image Pairs 103
5.4 Undistortion of Images 104
5.5 Grasp Execution 104
5.6 Imitiation Learning 106
6 Stereo-Based Object Recognition and Pose Estimation System 108
6.1 Recognition and Pose Estimation Based on the Shape 108
6.1.1 Problem Definition 110
6.1.2 Basic Approach 111
6.1.2.1 Region Processing Pipeline 112
6.1.2.2 Recognition 113
6.1.2.3 6-DoF Pose Estimation 114
6.1.2.4 Convenient Acquisition of Training Views 115
6.1.3 Orientation Correction 116
6.1.4 Position Correction 117
6.1.5 Increasing Robustness 121
6.1.6 Summary of the Algorithm 124
6.2 Recognition and Pose Estimation Based on Texture 125
6.2.1 Accuracy Considerations 126
6.2.2 Feature Calculation 134
6.2.3 Recognition and 2D Localization 138
6.2.3.1 Hough Transform 139
6.2.3.2 Homography Estimation 142
6.2.4 6-DoF Pose Estimation 145
6.2.5 Occlusions 150
6.2.6 Increasing Robustness 151
6.2.7 Runtime Considerations 153
6.2.8 Summary of the Algorithm 154
7 Stereo-Based Markerless Human Motion Capture System 155
7.1 Problem Definition 156
7.2 Human Upper Body Model 156
7.2.1 Kinematic Model 156
7.2.2 Geometric Model 156
7.3 General Particle Filtering Framework for Human Motion Capture 158
7.3.1 Edge Cue 159
7.3.2 Region Cue 161
7.3.3 Fusion of Multiple Cues 161
7.4 Cues in the Proposed System 162
7.4.1 Edge Cue 162
7.4.2 Distance Cue 162
7.4.3 Cue Comparison 163
7.4.4 Using a Calibrated Stereo System 169
7.5 Image Processing Pipeline 170
7.6 Hand/Head Tracking 172
7.7 Hierarchical Search 177
7.8 Fusing the Edge Cue and the Distance Cue 178
7.9 Adaptive Noise 181
7.10 Adaptive Shoulder Position 183
7.11 Incorporating Inverse Kinematics 186
7.12 Summary of the Algorithm 190
8 Software and Interfaces 192
8.1 Integrating Vision Toolkit 192
8.1.1 Implementation 192
8.1.2 The Class CByteImage 193
8.1.3 Implementation of Graphical User Interfaces 193
8.1.4 Connection of Image Sources 194
8.1.5 Integration of OpenCV 195
8.1.6 Integration of OpenGL 196
8.1.7 Camera Calibration and Camera Model 196
8.2 Master Motor Map 197
8.2.1 Specification 198
8.2.2 Framework and Converter Modules 200
8.2.3 Conversion to the Master Motor Map 202
Algorithm 22. 202
Algorithm 23. 203
8.3 Interfaces 203
8.3.1 Object Recognition and Pose Estimation 203
8.3.2 Human Motion Capture 204
9 Evaluation 205
9.1 Recognition and Pose Estimation System Based on the Shape 206
9.1.1 Accuracy 206
9.1.2 Real-World Experiments 213
9.1.3 Runtime 215
9.2 Recognition and Pose Estimation System Based on Texture 217
9.2.1 Accuracy 217
9.2.2 Real-World Experiments 227
9.2.3 Runtime 227
9.3 Markerless Human Motion Capture System 230
9.3.1 Automatic Initialization 230
9.3.2 Real-World Experiments and Accuracy 231
9.3.3 Runtime 234
10 Conclusion 237
10.1 Contribution 237
10.2 Example Applications 239
10.3 Discussion and Outlook 240
A Mathematics 243
A.1 Singular Value Decomposition 243
A.2 Pseudoinverse 243
A.2.1 Using the Regular Inverse 243
A.2.2 Using the Singular Value Decomposition 244
A.3 Linear Least Squares 244
A.3.1 Using the Normal Equation 244
A.3.2 Using the QR Decomposition 245
A.3.3 Using the Singular Value Decomposition 245
A.3.4 Homogeneous Systems 245
A.4 Functions for Rotations 245
B File Formats 248
B.1 Camera Parameters 248
List of Figures 250
List of Tables 256
List of Algorithms 257
References 258
Index 269