How are visual words represented? Insights from EEG-based visual word decoding, feature derivation and image reconstruction

被引:10
|
作者
Ling, Shouyu [1 ]
Lee, Andy C. H. [1 ,2 ]
Armstrong, Blair C. [1 ,3 ]
Nestor, Adrian [1 ]
机构
[1] Univ Toronto, Dept Psychol Scarborough, Toronto, ON, Canada
[2] Baycrest Ctr Geriatr Care, Rotman Res Inst, Toronto, ON, Canada
[3] Basque Ctr Cognit Brain & Language, BCBL, San Sebastian, Spain
基金
加拿大自然科学与工程研究理事会;
关键词
EEG; multivariate analysis; reading; word processing; TIME-COURSE; BRAIN ACTIVATION; FORM AREA; RECOGNITION; ERP; PATTERNS; IMPACT;
D O I
10.1002/hbm.24757
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
Investigations into the neural basis of reading have shed light on the cortical locus and the functional role of visual-orthographic processing. Yet, the fine-grained structure of neural representations subserving reading remains to be clarified. Here, we capitalize on the spatiotemporal structure of electroencephalography (EEG) data to examine if and how EEG patterns can serve to decode and reconstruct the internal representation of visually presented words in healthy adults. Our results show that word classification and image reconstruction were accurate well above chance, that their temporal profile exhibited an early onset, soon after 100 ms, and peaked around 170 ms. Further, reconstruction results were well explained by a combination of visual-orthographic word properties. Last, systematic individual differences were detected in orthographic representations across participants. Collectively, our results establish the feasibility of EEG-based word decoding and image reconstruction. More generally, they help to elucidate the specific features, dynamics, and neurocomputational principles underlying word recognition.
引用
收藏
页码:5056 / 5068
页数:13
相关论文
共 30 条
  • [1] A multivariate investigation of visual word, face, and ensemble processing: Perspectives from EEG-based decoding and feature selection
    Nemrodov, Dan
    Ling, Shouyu
    Nudnou, Ilya
    Roberts, Tyler
    Cant, Jonathan S.
    Lee, Andy C. H.
    Nestor, Adrian
    PSYCHOPHYSIOLOGY, 2020, 57 (03)
  • [2] Image Reconstruction from Bag-of-Visual-Words
    Kato, Hiroharu
    Harada, Tatsuya
    2014 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2014, : 955 - 962
  • [3] Auditory attention decoding from EEG-based Mandarin speech envelope reconstruction
    Xu, Zihao
    Bai, Yanru
    Zhao, Ran
    Zheng, Qi
    Ni, Guangjian
    Ming, Dong
    HEARING RESEARCH, 2022, 422
  • [4] How Is the Serial Order of a Visual Sequence Represented? Insights From Transposition Latencies
    Hurlstone, Mark J.
    Hitch, Graham J.
    JOURNAL OF EXPERIMENTAL PSYCHOLOGY-LEARNING MEMORY AND COGNITION, 2018, 44 (02) : 167 - 192
  • [5] The Neural Dynamics of Facial Identity Processing: Insights from EEG-Based Pattern Analysis and Image Reconstruction
    Nemrodov, Dan
    Niemeier, Matthias
    Patel, Ashutosh
    Nestor, Adrian
    ENEURO, 2018, 5 (01)
  • [6] Developing an efficient functional connectivity-based geometric deep network for automatic EEG-based visual decoding
    Khaleghi, Nastaran
    Rezaii, Tohid Yousefi
    Beheshti, Soosan
    Meshgini, Saeed
    BIOMEDICAL SIGNAL PROCESSING AND CONTROL, 2023, 80
  • [7] VISUAL IMAGE RECONSTRUCTION FROM HUMAN BRAIN ACTIVITY: A MODULAR DECODING APPROACH
    Kamitani, Yukiyasu
    JOURNAL OF PHYSIOLOGICAL SCIENCES, 2009, 59 : 113 - 113
  • [8] Visual image reconstruction from human brain activity: A modular decoding approach
    Miyawaki, Yoichi
    Uchida, Hajime
    Yamashita, Okito
    Sato, Masa-aki
    Morito, Yusuke
    Tanabe, Hiroki C.
    Sadato, Norihiro
    Kamitani, Yukiyasu
    INTERNATIONAL WORKSHOP ON STATISTICAL-MECHANICAL INFORMATICS 2009 (IW-SMI 2009), 2009, 197
  • [9] Visual image reconstruction from human brain activity: A modular decoding approach
    Kamitani, Y.
    PERCEPTION, 2009, 38 : 61 - 61
  • [10] Decoding Visual Motions from EEG Using Attention-Based RNN
    Yang, Dongxu
    Liu, Yadong
    Zhou, Zongtan
    Yu, Yang
    Liang, Xinbin
    APPLIED SCIENCES-BASEL, 2020, 10 (16):