A Hierarchical Probabilistic Framework for Recognizing Learners' Interaction Experience Trends and Emotions

We seek to model the users' experience within an interactive learning environment. More precisely, we are interested in assessing the relationship between learners' emotional reactions and three trends in the interaction experience, namely, flow: the optimal interaction (a perfect immersion within the task), stuck: the nonoptimal interaction (a difficulty to maintain focused attention), and off-task: the noninteraction (a dropout from the task). We propose a hierarchical probabilistic framework using a dynamic Bayesian network to model this relationship and to simultaneously recognize the probability of experiencing each trend as well as the emotional responses occurring subsequently. The framework combines three modality diagnostic variables that sense the learner's experience including physiology, behavior, and performance, predictive variables that represent the current context and the learner's profile, and a dynamic structure that tracks the evolution of the learner's experience. An experimental study, with a specifically designed protocol for eliciting the targeted experiences, was conducted to validate our approach. Results revealed that multiple concurrent emotions can be associated with the experiences of flow, stuck, and off-task and that the same trend can be expressed differently from one individual to another. The evaluation of the framework showed promising results in predicting learners' experience trends and emotional responses.