Automated Tensor Decomposition to Accelerate Convolutional Neural Networks

被引：0
|
作者：
Song B.-B. [1 ]
Zhang H. [2 ,3 ]
Wu Z.-F. [2 ,3 ]
Liu J.-H. [2 ,3 ]
Liang Y. [2 ,3 ]
Zhou W. [2 ,3 ]
机构：
[1] School of Information Science and Engineering, Yunnan University, Kunming
[2] National Pilot School of Software, Yunnan University, Kunming
[3] Engineering Research Center of Cyberspace, Yunnan University, Kunming
来源：
Ruan Jian Xue Bao/Journal of Software | 2021年 / 32卷 / 11期
基金：
中国国家自然科学基金;
关键词：
Automatic machine learning; Convolutional neural network; Neural network acceleration; Neural network compression; Tensor decomposition;
D O I：
10.13328/j.cnki.jos.006057
中图分类号：
学科分类号：
摘要：
Recently, convolutional neural network (CNN) have demonstrated strong performance and are widely used in many fields. Due to the large number of CNN parameters and high storage and computing power requirements, it is difficult to deploy on resource-constrained devices. Therefore, compression and acceleration of CNN models have become an urgent problem to be solved. With the research and development of automatic machine learning (AutoML), AutoML has profoundly impacted the development of neural networks. Inspired by this, this study proposes two automated accelerated CNN algorithms based on parameter estimation and genetic algorithms, which can calculate the optimal accelerated CNN model within a given accuracy loss range, effectively solving the error caused by artificially selected rank in tensor decomposition. It can effectively improve the compression and acceleration effects of the convolutional neural network. By rigorous testing on the MNIST and CIFAR-10 data sets, the accuracy rate on the MNIST dataset is slightly reduced by 0.35% compared to the original network, and the running time of the model is greatly reduced by 4.1 times, the accuracy rate on the CIFAR-10 dataset dropped slightly by 5.13%, and the running time of the model was greatly decreased by 0.8 times. © Copyright 2021, Institute of Software, the Chinese Academy of Sciences. All rights reserved.
引用
收藏
页码：3468 / 3481
页数：13
相关论文
共 44 条
  • [1] LeCun Y, Bottou L, Bengio Y, Haffner P., Gradient-based learning applied to document recognition, IEEE Institute of Electrical and Electronics Engineers, 86, 11, pp. 2278-2324, (1998)
  • [2] Bai C, Huang L, Chen JN, Pan X, Chen SY., Optimization of deep convolutional neural network for large scale image classification, Ruan Jian Xue Bao/Journal of Software, 29, 4, pp. 1029-1038, (2018)
  • [3] Zhou FY, Jin LP, Dong J., A review of convolutional neural networks, Chinese Journal of Computers, 40, 6, pp. 1229-1251, (2017)
  • [4] Howard AG, Zhu M, Chen B, Kalenichenko D, Wang WJ, Weyand T, Andreetto M, Adam H., Mobilenets: Efficient convolutional neural networks for mobile vision applications, (2017)
  • [5] Ji RR, Lin SH, Chao F., A survey of deep neural network compression and acceleration, Journal of Computer Research and Development, 55, 9, pp. 1871-1888, (2018)
  • [6] Denil M, Shakibi B, Dinh L, Ranzato M, DeFreitas N., Predicting parameters in deep learning, Proc. of the 2013 MIT Press Conf. on Neural Information Processing Systems (NIPS), pp. 2148-2156, (2013)
  • [7] Welling M, Weber M., Positive tensor factorization, Pattern Recognition Letters, 22, 12, pp. 1255-1261, (2001)
  • [8] Kim YD, Choi S., Nonnegative Tucker decomposition, Proc. of the 2007 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 2007), (2007)
  • [9] Hazan T, Polak S, Shashua A., Sparse image coding using a 3D non-negative tensor factorization, Proc. of the 2005 IEEE Conf. on 10th IEEE Int'l Conf. on Computer Vision (ICCV 2005), pp. 50-57, (2005)
  • [10] Nion D, Sidiropoulos ND., Tensor algebra and multidimensional harmonic retrieval in signal processing for MIMO radar, IEEE Trans. on Signal Processing, 58, 11, pp. 5693-5705, (2010)
12345