Automatic Detection of Emotion Valence on Faces Using Consumer Depth Cameras

Detection of positive and negative emotions can provide an insight into the person's level of satisfaction, social responsiveness and clues like the need for help. Therefore, automatic perception of affect valence is a key for novel human-computer interaction applications. However, robust recognition with conventional 2D cameras is still not possible in realistic conditions, in the presence of high illumination and pose variations. While the recent progress in 3D data expression recognition has alleviated some of these challenges, however, the high complexity and cost of these 3D systems renders them impractical. In this paper, we present the first practical 3D expression recognition using cheap consumer depth cameras. Despite the low fidelity facial depth data, we show that with appropriate preprocessing and feature extraction recognition is possible. Our method for emotion detection uses novel surface approximation and curvature estimation based descriptors on point cloud data, is robust to noise and computationally efficient. Experiments show that using only low fidelity 3D data of consumer cameras, we get 77.4% accuracy in emotion valence detection. Fusing mean curvature features with luminance data, boosts the accuracy to 89.4%.