EngageME: Exploring Neuropsychological Tests for Assessing Attention in Online Learning

During the pandemic, online learning has gained immense popularity. However, assessing student engagement in online settings remains challenging due to reliance on potentially biased and logistically complex self-reporting methods. This study investigates the use of neuropsychological tests, originally designed for attention assessment, to measure cognitive engagement in online learning. To conduct our analysis, we initially correlated clinical models of attention with pedagogical approaches in online learning. Subsequently, we pinpointed the three most crucial attention types in online learning-selective, sustained, and alternating attention. We used three neuropsychological assessments to evaluate attention in a cohort of 73 students contributing to the EngageME dataset. We manually annotated students' facial videos during neuropsychological assessments, revealing substantial agreement (Krippendorff's Alpha: 0.864) and a strong correlation (Spearman's Rank Correlation: 0.673) with neuropsychological test scores. This confirms the convergent validity of our approach in measuring attention during online learning. We further propose Nuanced Attention Labeling using neuropsychological test scores-based models in online learning attention assessment, enhancing sensitivity to nuanced cognitive engagement. To assess the reliability of our approach to online learning, we performed a user study in online settings. This work implies the potential for a more accurate and nuanced assessment of students' cognitive engagement in online learning, contributing to the refinement of personalized and effective educational interventions.