Speaker Dependent Real-Time Vowel Recognition Algorithm for Lip Sync in Digital Contents

Previous results of researches related speech recognition, especially vowel recognitions, can be applied for synchronizing the mouth movement with a dialogue in digital contents such as animations and e-learning contents. Since the mouth movement of objects has to be synchronized with a dialogue exactly, lip-sync is one of tedious works for animators and a time consuming work. The mismatch or artificialness between mouth shape of characters and speaking reduces the immersion in the contents. This paper proposes a new technique to automatically perform lip synching for a computer generated character to match a real speech or a dialogue using real time vowel recognition with the formant analysis. The proposed algorithm should be one of speaker dependent speech recognition since our concern is only for a certain voice actor related a certain character. The result shows that the vowels are recognized from a voice actor' speech with real time and the average of the recognition ratio is 97.3%.