Dynamic sparsity and model feature learning enhanced training for convolutional neural network-pruning

Recently, model-pruning approaches have become popular in reducing the high burden of deep neural networks in real-world applications. However, several existing pruning methods simply use a well-trained model to initialize parameters without considering its feature representation. Thus, we propose a label-free and dynamic pruning method based on model feature learning enhanced training. Furthermore, we use the category-level information and features of intermediate layers (well-trained model) to guide the task learning of the compression models, which enhances their ability to learn the features of the well-trained model. Additionally, we use different submodels (compression models) output information to learn from one another, promoting the feature learning ability between different submodels. Moreover, we use a structured sparsity-inducing regularization in a dynamic sparsity manner. The expected pruning parameters are identified using Taylor series-based channel sensitivity criteria. The proposed method solves the optimization problem using an iterative shrinkage-thresholding algorithm with dynamic sparsity. After the training is complete, the proposed method only eliminates redundant parameters without fine-tuning. Extensive experimental results show that the proposed method achieves good compression performance on multiple datasets and networks. © 2022, Science China Press. All right reserved.