Micro-expression Analysis by Fusing Deep Convolutional Neural Network and Optical Flow

Micro-expression is a kind of brief facial movements which could not be controlled by nervous system. Micro-expression indicates that a person is hiding his truly emotion consciously. Micro-expression analysis has various potential applications in public security and clinical medicine. Researches are focused on the automatic micro-expression recognition, because it is hard to recognize the micro-expression by the naked eye. This research proposes a novel algorithm for automatic micro-expression analysis which combines a deep multi-task convolutional network for detecting the facial landmarks and a fused deep convolutional network for estimating the optical flow features of the micro-expression. Firstly, the deep multi-task convolutional network is employed to detect facial landmarks with the manifold related tasks for dividing the facial region. Furthermore, a fused convolutional network is applied for extracting the optical flow features from the facial regions which contain the muscle changes when the micro-expression presents. Finally, a revised optical flow feature is applied for refining the information of the features and a Support Vector Machine classifier is adopted for recognizing and detecting the micro-expression. The result of experiments on two spontaneous micro-expression database proves that our method achieved competitive performance in micro-expression recognition and detection.