The approach of Chinese speech triseme recognition for human mouth animation

Different from text driven and phoneme based human mouth synthesis approaches, this paper presents the novel natural speech driven mouth animation approach. To capture the context information of continuously speaking mouth shapes, the triseme based modeling technique is employed for acquiring the trisemic HMMs. To obtain the robust model parameters with the limited training data, the states tying procedure is introduced. Considering the compatibility and ambiguity issues, the visemic questions which assigned in the leaf nodes of decision tree are generated that based on the training data. With the modeled HMM parameters, the viterbi beam searching algorithm is applied to time align the trisemic sequences. Mapping the recognized trisemes to the corresponding MPEG-4 FAPs-represented mouth shapes, the speaking mouth can be finally animated through a smoothing process. In terms of the proposed evaluation criterion, the experimental results illustrate that the recognition accuracy is applicable and also the aligning speed is acceptable in human vision.