Head Pose and Expression Transfer using Facial Status Score

We propose a method to transfer both head pose and facial expression of a source person in a video to the face of a target person in an output video. Our method models the entire 2D frame instead of the 3D face, and it generates output results using a status score, which includes the relative facial status about the head pose and expression in a frame. From the target video, the learning process obtains frame features needed for moving to each frame from the neutral frame for all frames, and generates the basis of these features via principal component analysis (PCA). Then, it learns to generate these features from a given status score sequentially. In the transfer process, it obtains a status score from a source frame of the video and generates the features from the given status score. Then, it generates the output frame using the reconstructed features. An output video is generated by repeating these steps for each source frame. Our method generates output results on the trajectory of the target video by using the advantage of PCA. Therefore, in the output results generated by our methods, both head pose and expression are transferred correctly while the non-face regions of the frames are supported. Finally, we experimentally compare the effectiveness of our method and conventional methods.