Adaptive Binarization Method for Binary Neural Network

In order to make convolutional neural network (CNN) run more effectively on embedded devices, many studies about binary neural networks have appeared. The traditional binary neural networks adopt the STE-based binarization method. This method has the problem of gradient mismatch during the training process. It will cause an enormous loss of accuracy to the network. For this reason, this paper proposes an adaptive binarization method. The method selects hardtanh function as the basic binarization function. This function changes alternately as the training epoch increases. It will eventually approach the sign function. And the method adjusts the threshold of binarization adaptively by introducing a learnable parameter. Based on the two CNN models of VGG-Small and ResNet-20, this paper conducts training with the adaptive binarization method on the CIFAR-10 dataset. The results show that the proposed method is better than traditional methods (BC, BWN). And Its accuracy is very close to some advanced methods, while these methods have lower running speed. Therefore, the adaptive binarization method has higher accuracy, and it can achieve faster running speed in the application of binary neural network.