How Well Do Online, Self-Administered Measures Correspond to In-Lab Assessments? A Preliminary Examination of Three Measures in Healthy Older Adults

The COVID-19 pandemic unexpectedly plunged investigators around the world into "digital neuropsychology" and online testing, and clinicians into telemedicine. Objective: The objective of the present study was to examine the correspondence between previously collected in-lab scores and online scores administered using a fully-automated platform, and identify issues that should be considered for automated testing. Method: We examined correspondence between in-lab and online scores for three types of measures often used as cognitive screeners in studies of older adults: A performance test, a subjective symptom-rating scale, and vocabulary test. Participants were a community sample of healthy older adults (n = 55). Results: Both preregistered and exploratory analyses found only modest correspondence for the performance test (Cronbach's alpha .604), but relatively good correspondence for the subjective rating (.821) and vocabulary test (.952). Additional analyses identified issues with specific items on the performance test, as well as potential indicators of problematic cases such as the use of external aids on the vocabulary test. Conclusion: Our results identify several pitfalls when adapting tests for online testing, but the relatively strong correspondence for some measures suggests a promising outlook for the validity of online testing once these are resolved. We make suggestions for how the identified pitfalls might be addressed in future studies attempting to adapt performance measures for online testing.