Speech driven MPEG-4 based face animation via neural network

In this paper, some clustering and machine teaming methods are combined together to learn the correspondence between speech acoustic and MPEG-4 based face animation parameters. The features of audio and image sequences can be extracted from the large recorded audio-visual database. The face animation parameter (FAP) sequences can be computed and then clustered to FAP patterns. An artificial neural network (ANN) was trained to map the linear predictive coefficients (LPC) and some prosodic features of an individual's natural speech to FAP patterns. The performance of our system shows that the proposed teaming algorithm is suitable, which can greatly improve the realism of real time face animation during speech.