Facial Animation Method Based on Deep Learning and Expression AU Parameters

被引：0
|
作者：
Yan Y. [1 ]
Lyu K. [1 ]
Xue J. [1 ]
Wang C. [1 ]
Gan W. [1 ]
机构：
[1] School of Engineering Science, University of Chinese Academy of Sciences, Beijing
来源：
Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/Journal of Computer-Aided Design and Computer Graphics | 2019年 / 31卷 / 11期
关键词：
Blendshape model; Deep learning; Facial action units; Facial animation;
D O I：
10.3724/SP.J.1089.2019.17682
中图分类号：
学科分类号：
摘要：
To generate virtual characters with realistic expression more conveniently using computers, a method based on deep learning and expression AU parameters is proposed for generating facial animation. This method defines 24 facial action unit parameters, i.e. expression AU parameters, to describe facial expression; then, it constructs and trains corresponding parameter regression network model by using convolutional neural network and the FEAFA dataset. During generating facial animation from video images, video sequences are firstly obtained from ordinary monocular cameras, and faces are detected from video frames based on supervised descent method. Then, the expression AU parameters, regarded as expression blendshape coefficients, are regressed accurately from the detected facial images, which are combined with avatar's neutral expression blendshape and the 24 corresponding blendshapes to generate the animation of the digital avatar based on a blendshape model under real world conditions. This method does not need 3D reconstruction process in traditional methods, and by taking the relationship between different action units into consideration, the generated animation is more natural and realistic. Furthermore, the expression coefficients are more accurate based on face images rather than facial landmarks. © 2019, Beijing China Science Journal Publishing Co. Ltd. All right reserved.
引用
收藏
页码：1973 / 1980
页数：7
相关论文
共 30 条
  • [11] Xiong X.H., De-La-Torre F., Supervised descent method and its applications to face alignment, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 532-539, (2013)
  • [12] Luo P., Wang X.G., Tang X.O., Hierarchical face parsing via deep learning, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2480-2487, (2012)
  • [13] Wu Y., Wang Z.G., Ji Q., Facial feature tracking under varying facial expressions and face poses based on restricted Boltzmann machines, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3452-3459, (2013)
  • [14] Zhang Z.P., Luo P., Chen C.L., Et al., Facial landmark detection by deep multi-task learning, Proceedings of the European Conference on Computer Vision, pp. 94-108, (2014)
  • [15] Wu Y., Hassner T., Kim K., Et al., Facial landmark detection with tweaked convolutional neural networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 12, pp. 3067-3074, (2018)
  • [16] Zhang K.P., Zhang Z.P., Li Z.F., Et al., Joint face detection and alignment using multitask cascaded convolutional networks, IEEE Signal Processing Letters, 23, 10, pp. 1499-1503, (2016)
  • [17] Lewis J.P., Mooser J., Deng Z.G., Et al., Reducing blendshape interference by selected motion attenuation, Proceedings of the Symposium on Interactive 3D Graphics and Games, pp. 25-29, (2005)
  • [18] Cao C., Weng Y.L., Zhou S., Et al., FaceWarehouse: a 3D facial expression database for visual computing, IEEE Transactions on Visualization and Computer Graphics, 20, 3, pp. 413-425, (2014)
  • [19] Waters K., A muscle model for animation three-dimensional facial expression, ACM SIGGRAPH Computer Graphics, 21, 4, pp. 17-24, (1987)
  • [20] Lee Y., Terzopoulos D., Waters K., Realistic modeling for facial animation, Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, pp. 55-62, (1995)
123