A novel micro-expression recognition algorithm using dual-stream combining optical flow and dynamic image convolutional neural networks

Human facial expressions play an important role in interpersonal communication. However, most algorithms use a single feature in micro-expression recognition, which leads to low performance. We propose a novel micro-expression recognition algorithm using dual-stream combining optical flow and dynamic image convolutional neural networks. First, the algorithm can construct multiple features to classify micro-expressions. Second, the OF-Block module is proposed to extract optical flow features and preserve face motion information considerably. Finally, the proposed algorithm presents a hierarchical fusion strategy that fuses face motion features and spatiotemporal features several times to protect the diversity of features. Our proposed method is evaluated in three publicly available micro-expression databases (CASME, CASME II and SAMM). Experimental results demonstrate that the proposed model can achieve state-of-the-art results compared to other models. Our accuracies are 61.20%, 73.09% and 58.07% in the CASME and CASME II and SAMM datasets, respectively.