Design of an Online Education Evaluation System Based on Multimodal Data of Learners

Online education breaks the time and space constraints of learning, but it also presents some new challenges for the teachers: less interaction between instructors and learners, and loss of real-time feedback of teaching effects. Our study aims to fill these gaps by designing a tool for instructors that shows how learners' status change along the lecture video timeline. The study uses multimodal data consist of facial expressions and timeline-anchored comments and labels the data with two learning status dimensions (difficulty and interestingness). To acquire training dataset, 20 teaching video clips are selected, and 15 volunteers are invited to watch the videos to collect their facial expressions and subjective learning status ratings. Then we build a fusion model with results from a CNN (Convolutional Neural Network) model and a LSTM (Long Short-Term Memory) model, and design an effective interface to present feedbacks from the model. After evaluation of the model, we put forward some possible improvements and future prospects for this design.