Hybrid Binary-Stochastic Computing-based ANN Design with Binary-in-Series-out ReLU

Because of the remarkable performance on the problem of classification and recognition, Artificial Neural Networks (ANNs) have received much attention in recent years. However, due to the massive parameters and complicated interconnections between each neuron, ANN hardware architectures suffer from considerable area overhead and power consumption, which becomes the design challenge nowadays. To mitigate the design problem, the Stochastic Computing (SC)-based computing method has been proposed to use stochastic bit-stream for each kind of arithmetic operation in recent years. Besides, many SC-based ANN designs were proposed and showed their efficiency to save the computing power and area of the hardware. However, the conventional SC-based ANNs (SC-ANNs) suffer from low computing accuracy due to the applied stochastic bit-stream. To further improve the computing accuracy of the SC-ANN, we propose to replace the SC-based multiply-accumulator (MAC) with the parallel counter (PC) in this work. In addition, we further propose a binary-input-series-output (BISO) ReLU function to transfer the binary to the corresponding stochastic bit-stream efficiently. Compared with the conventional SCANN approach, the proposed SC-ANN design with PC-based MAC and BISO ReLU can improve computing accuracy by 86.6%. In addition, the proposed approach can reduce 95.3% area cost and 90% power consumption over than non-SC-ANN design, which achieves higher hardware efficiency.