Learning a Smart Convolutional Neural Network with High-level Semantic Information

With the wide application of big data and the development of computer computing capability, deep Convolutional Neural Network (CNN) has been widely applied in the field of computer vision. The current architecture of deep neural network is becoming deeper and more complex for achieving a better performance. However, their natural disadvantages such as larger consumption of computation or memory, and longer run-time make CNN models difficult to be applied to the mobile and embedded devices. In this paper, we learn a Smart Convolutional Neural Network (S-CNN) under the guide of neurons' high-level semantic information distilled from a cumbersome neural network. S-CNN can be seen as an improved CNN model, which is with less consumption of computation and memory in the predicted process. We verify the superiority of S-CNN in terms of image classification task on three benchmarking datasets, including CIFAR-10, CIFAR-100 and SVHN. Experimental results clearly demonstrate that the proposed S-CNN can get an exciting performance compared with traditional CNN models.