A Convolutional Heterogeneous Spiking Neural Network for Real-time Music Classification

While artificial Neural Networks(ANNs) have gradually become the dominating frameworks for music classification, single-structured ANNs have certain functional and computational limitations to meet real-time and low-latency requirement in this era of the Internet of Things. With Convolutional Neural Network(CNN) module to address spatial correlations and Recurrent Neural Networks(RNN) module to capture temporal dependencies, hybrid CNN-RNN architectures have attracted academic focus. However, the inference promptness and energy efficiency of these hybrid models are still confined by their computationally costly RNN modules. With sparse and binary activated data flows, the brain-inspired Spiking Neural Networks(SNNs) enable RNN-like sequential processing in a computationally efficient and economic way. Taking respective advantages of dynamically different spiking neurons, we proposed the Convolutional Heterogeneous Spiking Neural Network(CHSNN). With deliberate choices of input spectrograms and training strategy, the proposed CHSNN have outperformed most existing CNN-RNN classification models, demonstrated satisfactory real-time processing ability even with only 20.8% inference time steps, and shown the substitutability of SNN against RNN in terms of both classification performance and inference latency.