Binary convolutional neural network: Review

被引：0
|
作者：
Ding W. [1 ]
Liu C. [2 ]
Li Y. [2 ]
Zhang B. [3 ]
机构：
[1] Unmanned System Research Institute, Beihang University, Beijing
[2] School of Electronic and Information Engineering, Beihang University, Beijing
[3] School of Automation Science and Electrical Engineering, Beihang University, Beijing
来源：
Hangkong Xuebao/Acta Aeronautica et Astronautica Sinica | 2021年 / 42卷 / 06期
基金：
中国国家自然科学基金;
关键词：
Binarization; Binary convolutional neural networks; Deep learning; Full-precision convolutional neural networks; Lightweight; Model compression; Quantization;
D O I：
10.7527/S1000-6893.2020.24504
中图分类号：
学科分类号：
摘要：
In recent years, Binary Convolutional Neural Networks (BNNs) have attracted much attention owing to their low storage and high computational efficiency. However, the mismatch between forward and backward quantization results in a huge performance gap between the BNN and the full-precision convolutional neural network, affecting the deployment of the BNN on resource-constrained platforms. Researchers have proposed a series of algorithms and training methods to reduce the performance gap during the binarization process, thereby promoting the application of BNNs to embedded portable devices. This paper makes a comprehensive review of BNNs, mainly from the perspectives of improving network representative capabilities and fully exploring the network training potential. Specifically, improving network representative capabilities includes the design of the binary quantization method and structure design, while fully exploring the network training potential involves loss function design and the training strategy. Finally, we discuss the performance of BNNs in different tasks and hardware platforms, and summarize the challenges in future research. © 2021, Beihang University Aerospace Knowledge Press. All right reserved.
引用
收藏
相关论文
共 55 条
  • [1] HE K M, ZHANG X Y, REN S Q, Et al., Deep residual iearning for image recognition, Proceeding of IEEE conference on computer vision and pattern recognition, pp. 770-778, (2016)
  • [2] REN S Q, HE K M, GIRSHICK R, Et al., Faster RCNN: Towards real-time object detection with region proposal networks, Advances in Neural Information Processing Systems, pp. 91-99, (2015)
  • [3] CHEN L C, PAPANDREOU G, KOKKINOS I, Et al., Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs, IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 4, pp. 834-848, (2017)
  • [4] JI R R, LIN S H, CHAO F, Et al., Deep neural network compression and acceleration: A review, Journal of Computer Research and Development, 55, 9, pp. 1871-1888, (2018)
  • [5] LEI J, GAO X, SONG J, Et al., Survey of deep neural network model compression, Journal of Software, 29, 2, pp. 251-266, (2018)
  • [6] LI J Y, ZHAO Y K, XUE Z E, Et al., A survey of model compression for deep neural networks, Chinese Journal of Engineering, 41, 10, pp. 1229-1239, (2019)
  • [7] CAO W L, RUI J W, LI M., Summary of neural network model compression methods, Application Research of Computers, 36, 3, pp. 649-656, (2019)
  • [8] GENG L L, NIU B N., Survey of deep neural networks model compression, Journal of Frontiers of Computer Science & Technology, 14, 9, pp. 1-16, (2020)
  • [9] ZHANG C, TIAN J, WANG Y S, Et al., Overview of neural network model compression methods, Proceedings of the 22nd Annual Conference of New Network Technologies and Applications in 2018 of the Network Application Branch of China Computer Users Association, (2018)
  • [10] CAI R C, ZHONG C R, YU Y, Et al., CNN quantization and compression strategy for edge computing applications, Journal of Computer Applications, 38, 9, pp. 2449-2454, (2018)
123456