Robust Facial Expression Recognition With Generative Adversarial Networks

被引：0
|
作者：
Yao N.-M. [1 ,2 ]
Guo Q.-P. [1 ,2 ]
Qiao F.-C. [1 ,2 ]
Chen H. [1 ,2 ]
Wang H.-A. [1 ,2 ,3 ]
机构：
[1] Beijing Key Laboratory of Human-Computer Interaction, Institute of Software, Chinese Academy of Sciences, Beijing
[2] University of Chinese Academy of Sciences, Beijing
[3] State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing
来源：
Zidonghua Xuebao/Acta Automatica Sinica | 2018年 / 44卷 / 05期
基金：
国家重点研发计划; 中国国家自然科学基金;
关键词：
Convolutional neural network (CNN); Face completion; Facial expression recognition; Generative adversarial net (GAN); Person-independent;
D O I：
10.16383/j.aas.2018.c170477
中图分类号：
学科分类号：
摘要：
In natural communication, people would express their expressions with head rotation and body movement, which may result in partial occlusion of face and a consequent information loss regarding facial expression. Also, most of the existing approaches to facial expression recognition are not robust enough to unseen users because they rely on general facial features or algorithms without considering differences between facial expression and facial identity. In this paper, we propose a person-independent recognition method for partially-occluded facial expressions. Based on Wasser-stein generative adversarial net (WGAN), a generative network of facial image is trained to perform context-consistent image completion for partially-occluded facial expression images. With an adversarial learning strategy, furthermore, a facial expression recognition network and a facial identity recognition network are established to improve the accuracy and robustness of facial expression recognition via inhibition of intra-class variation. Extensive experimental results demon-strate that 90% average recognition accuracy of facial expression has been reached on a mixed dataset composed of CK+, Multi-PIE, and JAFFE. Moreover, our method achieves 96% accuracy of user-independent recognition on CK+. A 4.5% performance gain is achieved with the novel identity-inhibited expression feature. Our method is also capable of improving recognition accuracy for non-frontal facial expressions within a range of 45-degree head rotation. Copyright © 2018 Acta Automatica Sinica. All rights reserved.
引用
收藏
页码：865 / 877
页数：12
相关论文
共 52 条
  • [1] Zeng Z.H., Pantic M., Roisman G.I., Huang T.S., A survey of affect recognition methods: audio, visual, and sponta-neous expressions, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 1, pp. 39-58, (2009)
  • [2] Valstar M., Pantic M., Patras I., Motion history for facial action detection in video, Proceedings of the 2004 IEEE International Conference on Systems, Man, and Cybernetics, pp. 635-640, (2004)
  • [3] Sandbach G., Zafeiriou S., Pantic M., Yin L.J., Static and dynamic 3D facial expression recognition: a comprehensive survey, Image and Vision Computing, 30, 10, pp. 683-697, (2012)
  • [4] Gunes H., Schuller B., Categorical and dimensional affect analysis in continuous input: current trends and future directions, Image and Vision Computing, 31, 2, pp. 120-136, (2013)
  • [5] Yang M., Zhang L., Yang J., Zhang D., Robust sparse coding for face recognition, Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, pp. 625-632, (2011)
  • [6] Li Y.J., Liu S.F., Yang J.M., Yang M.H., Generative face completion, Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 5892-5900, (2017)
  • [7] Krizhevsky A., Sutskever I., Hinton G.E., ImageNet classification with deep convolutional neural networks, Proceedings of the 25th International Conference on Neural Information Processing Systems, pp. 1097-1105, (2012)
  • [8] Yu Z.D., Zhang C., Image based static facial expression recognition with multiple deep network learning, Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 435-442, (2015)
  • [9] Kim B.K., Dong S.Y., Roh J., Kim G., Lee S.Y., Fusing aligned and non-aligned face information for automatic affect recognition in the wild: a deep learning approach, Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1499-1508, (2016)
  • [10] Mollahosseini A., Chan D., Mahoor M.H., Going deeper in facial expression recognition using deep neural networks, Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision, pp. 1-10, (2016)
123456