How Did Students Engage with a Remote Educational Assessment? A Case Study

Many educational summative and formative assessments have been transferred to a remote online setting because of the pandemic. Educational professionals and stakeholders have shown interest in learning how this change in the test mode influenced test takers; that is, whether test-taking experiences in a remote test setting were different from those in a standard test center setting. We propose an innovative approach that makes use of log data collected from the digital platforms of a high-stakes international test and applies data-mining techniques to address the mode effect. In the study, the test-taking behaviors were measured by where students spent their time during the test, extracted from log files collected from both remote test and test center modes. Results showed that four distinct clusters existed among students' test-taking behaviors, regardless of the test modes and students' test scores. Within each cluster, statistically significant behavioral differences existed between the two mode groups; however, the magnitude of the mode effect was limited by where students spent their time during the test.