FSTA-Net: Motor Imagery EEG Decoding Based on Frequency-Spatial-Time Features

Decoding electroencephalography (EEG) signals based on motor imagery (MI) is vital in rehabilitation and motor-assisted instrumentation. As an essential step in decoding, feature extraction usually extracts single-domain features, such as spatial and frequency features, or dual-domain features, such as time-frequency features. However, the multidomain features that describe the intent more comprehensively are not simultaneously acquired. Therefore, we propose a frequency-space-time (FST) multidomain feature extraction method. In the feature extraction process, short-time Fourier transform (STFT) is first used to obtain the time-frequency features of the signal and spliced by channel. The covariance operation is performed for each time segment. For multidomain features, we designed a deep-learning network named FSTA-Net. FSTA-Net mainly consists of a frequency-spatial attention module, a time-domain attention module, and a classification network. The computational logic of the weighted summation performed by the two-attention module aggregates the relevant features of all positions of the acting object. To overcome the overfitting problem during training, a data enhancement method based on time-domain translation is designed. We validated the performance of FST features on the brain-computer interfaces (BCIs) Competition IV IIa and IIb datasets. FST features have the highest recognition accuracy and low standard deviation compared to common spatial patterns (CSP), STFT, and continuous wavelet transforms (CWT). The proposed decoding framework obtains 77.1% and 85.1% recognition rates on the two datasets, comparable to the current state-of-the-art methods. It provides a reference for a more in-depth analysis of multidomain features in MI-EEG.