Multimodal detection of epilepsy with deep neural networks

Epilepsy constitutes a chronic noncommunicable disease of the brain affecting approximately 50 million people around the world. Most of the existing research initiatives propose methods for detecting and predicting epilepsy, which rely on the extraction of handcrafted features and the train of traditional machine learning classifiers. In this paper, we present two new methods to distinguish healthy, interictal, and ictal cases without the time-consuming procedure of feature extraction. Firstly, we apply the short-time fourier transform (STFT) to the single-channel electroencephalogram (EEG) signals and construct an image consisting of three channels. This image is passed through pretrained models, including AlexNet, DenseNet201, EfficientNet, ResNet18, etc. Secondly, we introduce a multimodal deep neural network. Specifically, we pass each single-channel EEG signal through two branches of convolutional neural networks (CNNs), which can extract low and high frequency features. Also, we apply the short-time fourier transform (STFT) to the EEG signals and create an image consisting of three channels. The image is passed through a pretrained EfficientNet-B7 model. Finally, we employ a gated multimodal unit to control the importance of each modality. We evaluate the performance of the proposed model on five different cases on the EEG database of the University of Bonn and show that our introduced model achieves comparable performance to state-of-the-art approaches.