Neural Networks with Model Compression (eBook)

Deep learning has achieved impressive results in image classification, computer vision and natural language processing. To achieve better performance, deeper and wider networks have been designed, which increase the demand for computational resources. The number of floating-point operations (FLOPs) has increased dramatically with larger networks, and this has become an obstacle for convolutional neural networks (CNNs) being developed for mobile and embedded devices. In this context, our book will focus on CNN compression and acceleration, which are important for the research community. We will describe numerous methods, including parameter quantization, network pruning, low-rank decomposition and knowledge distillation. More recently, to reduce the burden of handcrafted architecture design, neural architecture search (NAS) has been used to automatically build neural networks by searching over a vast architecture space. Our book will also introduce NAS due to its superiority and state-of-the-art performance in various applications, such as image classification and object detection. We also describe extensive applications of compressed deep models on image classification, speech recognition, object detection and tracking. These topics can help researchers better understand the usefulness and the potential of network compression on practical applications. Moreover, interested readers should have basic knowledge about machine learning and deep learning to better understand the methods described in this book.

Baochang Zhang is a full Professor with Institute of Artificial Intelligence, Beihang University, Beijing, China. He was selected by the Program for New Century Excellent Talents in University of Ministry of Education of China, also selected as Academic Advisor of Deep Learning Lab of Baidu Inc., and a distinguished researcher of Beihang Hangzhou Institute in Zhejiang Province. His research interests include explainable deep learning, computer vision and patter recognition. His HGPP and LDP methods were state-of-the-art feature descriptors, with 1234 and 768 Google Scholar citations, respectively. Both are 'Test-of-Time' works. Our 1-bit methods achieved the best performance on ImageNet. His group also won the ECCV 2020 tiny object detection, COCO object detection, and ICPR 2020 Pollen recognition challenges.

 

Tiancheng Wang are pursuing their Ph.D. degrees under the supervision of Baochang Zhang. His research topics include model compression and trustworthy deep learning, and he has published several high-quality papers on deep model compression. He was selected as visiting student of Zhongguancun laboratory, Beijing, China. 

 

Sheng Xu are pursuing their Ph.D. degrees under the supervision of Baochang Zhang. His research topics mainly focus on low-bit model compression, and he is one of the most active researchers in the field of binary neural networks. He has published more than 10 top-tier papers in computer vision with two of them are selected as CVPR oral papers.

 

Dr. David Doermann is a Professor of Empire Innovation at the University at Buffalo (UB) and the Director of the University at Buffalo Artificial Intelligence Institute. Prior to coming to UB, he was a program manager at the Defense Advanced Research Projects Agency (DARPA), where he developed, selected and oversaw approximately $150 million in research and transition funding in the areas of computer vision, human language technologies and voice analytics. He coordinated performers on all of the projects, orchestrating consensus, evaluating cross team management and overseeing fluid program objectives.

---

Deep learning has achieved impressive results in image classification, computer vision and natural language processing. To achieve better performance, deeper and wider networks have been designed, which increase the demand for computational resources. The number of floating-point operations (FLOPs) has increased dramatically with larger networks, and this has become an obstacle for convolutional neural networks (CNNs) being developed for mobile and embedded devices. In this context, our book will focus on CNN compression and acceleration, which are important for the research community. We will describe numerous methods, including parameter quantization, network pruning, low-rank decomposition and knowledge distillation. More recently, to reduce the burden of handcrafted architecture design, neural architecture search (NAS) has been used to automatically build neural networks by searching over a vast architecture space. Our book will also introduce NAS due to its superiority and state-of-the-art performance in various applications, such as image classification and object detection. We also describe extensive applications of compressed deep models on image classification, speech recognition, object detection and tracking. These topics can help researchers better understand the usefulness and the potential of network compression on practical applications. Moreover, interested readers should have basic knowledge about machine learning and deep learning to better understand the methods described in this book.