Exploring indicators of engagement in online learning as applied to adolescent health prevention: a pilot study of REAL media

Engagement is central to the effectiveness of online health messages and the related educational programs that aim to deliver these messages to the intended audience (Li et al. in Educ Tech Res Dev. 10.1007/s11423-019-09709-9, 2019). Drawing from health communication and social learning theories, the Theory of Active Involvement (TAI) (Greene in Health Commun 28:644-656. 10.1080/10410236.2012.762824, 2013) posits that an online prevention program's impact depends on how engaged participants are. In practice, measuring engagement in this context has relied primarily on self-report measures (e.g., Hamutoglu et al. in Educ Tech Res Dev. 10.1007/s11423-019-09705-z, 2019). However, the emergence and growth of online learning platforms to deliver health-specific information offers other options for assessing engagement. This includes program analytics that capture interaction with content and facilitate examination of patterns via multiple indicators such as responses to interactive questions and time spent in the program (Herodotou et al. in Educ Technol Res Devel 67:1273. 10.1007/s11423-019-09685-0, 2019; Li et al. 2019; van Leeuwen in Educ Tech Res Dev 67:1043. 10.1007/s11423-018-09639-y, 2019). However, little is known about the relationships between these different indicators of engagement as it applies to health curricula. This study uses self-report, observational, and program analytic data collected on a small (N = 38) sample using REAL media, an online substance use prevention program, to examine relationships among various indicators of engagement. Findings suggest a cluster of indicators across the three modalities that provide a useful way of measuring engagement. A cluster centered around complexity suggests a separate factor to be considered when designing engaging interventions.