Digital Video Processing

A. Murat Tekalp is a Professor at Koc University, Istanbul, Turkey, and is a Fellow of IEEE and a member of Academia Europaea and Turkish Academy of Sciences. He received a Ph.D. degree in electrical, computer, and systems engineering from Rensselaer Polytechnic Institute (RPI), Troy, New York, in 1984. He was with the Eastman Kodak Company, Rochester, New York, from 1984 to 1987, and was a professor at the University of Rochester, Rochester, New York, from 1987 to 2005, where he was named Distinguished University Professor. His research interests are in the area of digital image and video processing, image and video compression, and video networking.   Professor Tekalp received the TUBITAK Science Award (highest scientific award in Turkey) in 2004. He is a former Chair of the IEEE Technical Committee on Image and Multidimensional Signal Processing, and a Founding Member of IEEE Technical Committee on Multimedia Signal Processing. He was appointed as the Technical Program Co-Chair for IEEE ICASSP 2000 in Istanbul, Turkey; the General Chair of the IEEE International Conference on Image Processing (ICIP) at Rochester, New York, in 2002; and Technical Program Co-Chair of EUSIPCO 2005 in Antalya, Turkey.

Preface xvii

About the Author xxv

 

Chapter 1: Multi-Dimensional Signals and Systems 1

1.1 Multi-Dimensional Signals 2

1.2 Multi-Dimensional Transforms 8

1.3 Multi-Dimensional Systems 20

1.4 Multi-Dimensional Sampling Theory 30

1.5 Sampling Structure Conversion 42

References 47

Exercises 48

 

Chapter 2: Digital Images and Video 53

2.1 Human Visual System and Color 54

2.2 Analog Video 63

2.3 Digital Video 67

2.4 3D Video 79

2.5 Digital-Video Applications 85

2.6 Image and Video Quality 96

References 100

 

Chapter 3: Image Filtering 105

3.1 Image Smoothing 106

3.2 Image Re-Sampling and Multi-Resolution Representations 110

3.3 Image-Gradient Estimation, Edge and Feature Detection 127

3.4 Image Enhancement 137

3.5 Image Denoising 147

3.6 Image Restoration 164

References 181

Exercises 186

MATLAB Resources 193

 

Chapter 4: Motion Estimation 195

4.1 Image Formation 196

4.2 Motion Models 202

4.3 2D Apparent-Motion Estimation 214

4.4 Differential Methods 225

4.5 Matching Methods 233

4.6 Nonlinear Optimization Methods 245

4.7 Transform-Domain Methods 249

4.8 3D Motion and Structure Estimation 251

References 263

Exercises 268

MATLAB Resources 272

 

Chapter 5: Video Segmentation and Tracking 273

5.1 Image Segmentation 275

5.2 Change Detection 289

5.3 Motion Segmentation 298

5.4 Motion Tracking 317

5.5 Image and Video Matting 328

5.6 Performance Evaluation 330

References 331

MATLAB Exercises 338

Internet Resources 339

 

Chapter 6: Video Filtering 341

6.1 Theory of Spatio-Temporal Filtering 342

6.2 Video-Format Conversion 349

6.3 Multi-Frame Noise Filtering 367

6.4 Multi-Frame Restoration 374

6.5 Multi-Frame Super-Resolution 377

References 394

Exercises 399

 

Chapter 7: Image Compression 401

7.1 Basics of Image Compression 402

7.2 Lossless Image Compression 417

7.3 Discrete-Cosine Transform Coding and JPEG 431

7.4 Wavelet-Transform Coding and JPEG2000 443

References 454

Exercises 456

Internet Resources 459

 

Chapter 8: Video Compression 461

8.1 Video-Compression Approaches 462

8.2 Early Video-Compression Standards 467

8.3 MPEG-4 AVC/ITU-T H.264 Standard 483

8.4 High-Efficiency Video-Coding (HEVC) Standard 491

8.5 Scalable-Video Compression 497

8.6 Stereo and Multi-View Video Compression 502

References 512

Exercises 514

Internet Resources 515

 

Appendix A: Vector-Matrix Operations in Image and Video Processing 517

A.1 Two-Dimensional Convolution 517

A.2 Two-Dimensional Discrete-Fourier Transform 520

A.3 Three-Dimensional Rotation – Rotation Matrix 521

References 525

Exercises 525

 

Appendix B: Ill-Posed Problems in Image and Video Processing 527

B.1 Image Representations 527

B.2 Overview of Image Models 528

B.3 Basics of Sparse-Image Modeling 530

B.4 Well-Posed Formulations of Ill-Posed Problems 531

References 532

 

Appendix C: Markov and Gibbs Random Fields 533

C.1 Equivalence of Markov Random Fields and Gibbs Random Fields 533

C.2 Gibbs Distribution as an a priori PDF Model 537

C.3 Computation of Local Conditional Probabilities from a Gibbs Distribution 538

References 539

 

Appendix D: Optimization Methods 541

D.1 Gradient-Based Optimization 542

D.2 Simulated Annealing 544

D.3 Greedy Methods 547

References 549

 

Appendix E: Model Fitting 551

E.1 Least-Squares Fitting 551

E.2 Least-Squares Solution of Homogeneous Linear Equations 552

E.3 Total Least-Squares Fitting 554

E.4 Random-Sample Consensus (RANSAC) 556

References 556

 

Index 557