Semantic category-based decoding of human brain activity using a Gabor-based model by estimating intracranial field potential range in temporal cortex

Predicting and decoding the recorded neural activity for visual stimuli is the topic of many studies. This prediction can be made by comparing the model's responses to different stimuli with the available recorded brain signal. The neural activities can be decoded then by finding the stimulus which has generated the nearest model's response to the recorded signal. In this study, a model is proposed which can estimate the response of human brain to images from different conceptual categories by inserting the visual stimuli as the model input after filtering by Gabor wavelets. This helped us to find each image's low level visual features. Afterward, the extracted image features were applied to the input of a curve fitting neural network. As the output, the range of intracranial field potential was estimated. This was performed separately for each pixel of the image. To evaluate the model's accuracy, two factors were used, namely the Pearson correlation and Normalized root mean square error. The results show that the proposed model can accurately estimate the brain's response to conceptual categories To decode the brain' activity based on the observed semantic category in each test observations by using of the model, we calculated the distance between the recorded signal and the model responses to all stimuli from different categories and assigned the category of the nearest model response to brain's response in that trial. To this end, a K-nearest neighbors classifier based on Euclidean distance was used. This leaded to a classification accuracy which was significantly higher than chance level. So, the proposed model can be used to decode the activity of the brain in response to the visual stimuli.