Non-contact emotion analysis based on videos

Facial temperature changes contain information about emotional states. Judging human emotional states by facial temperature is an active area of research. In this paper, we use film clips to induce emotions. During the whole process, we obtained RGB video, thermal video, and Galvanic Skin Response (GSR) data of the subjects. We employ GSR data as a baseline reference for emotional state judgment. The RGB video was used to calibrate and track key facial regions in the thermal video to obtain more accurate temperature variability data for later sentiment analysis. The validity of sentiment state detection was tested by using Galvanic skin response data as a reference metric. Firstly, we obtain some desired emotional features based on temperature data at different locations on face, and then fuse these features into the linear discriminant analysis (LDA) classifier to judge the emotional state. We studied and verified 128 emotional video datas (fear, happiness, depression) from 45 participants. We achieved 77% accuracy rate in emotional state judgment.