Automatic Engagement Recognition for Distance Learning Systems: A Literature Study of Engagement Datasets and Methods

With the paradigm shift of learning processes from traditional classroom to distance learning systems, the recent success of artificial intelligent research and applications, including machine learning and deep neural networks, have increasingly been leveraged to learn how to include engagement state analysis into the distance learning process. Recent automatic engagement estimations employ several modalities, such as, video, audio, and biological signals or neuro-sensing information as source input to be analyzed. In this paper, we provide a literature review of engagement estimation, including dataset, algorithms as well as the discussion and evaluation. First of all, we present the engagement datasets, including publicly available and proprietary sources built for some specific concerns. We then describe the methodology of engagement measurements that are widely used in literature and state-of-the-art algorithms to automate the estimation. The advantages and limitations of the algorithms are briefly discussed and summarized in benchmark of the used modalities and datasets. Additionally, we extend this literature review to the insight for the practical use of automatic engagement estimation in a real education process that is crucial to distance learning improvement. Finally, we review the remaining challenges for robust engagement estimation as part of attempts in improving distance learning quality and performance by taking personalized engagement into account.