Lip Synchronization System For Animation Based On Speech Recognition

With the growth of the requirements and the support from government, the animation industry is entering an era of rapid development. Lip synchronization is an indispensable part in the procedure of making animation. The traditional method in solving lip synchronization is really inefficient. And the effect of the lip synchronization relies on the experience of the people. So we need some technology to help them to improve the efficient and the synchronization rate.This paper introduced a way to solve the lip synchronization problem by using the speech recognition technology based on HMM. By studying the speech recognition technology and analyzing the lip synchronization problem, we find the entrance to solve the lip synchronization problem: using the speech recognition technology to get the time information of every character in the speech.Based on this idea, we designed two methods to solve the lip synchronization problem. The first method using the recognition algorithm directly, and get the time info during recognition. The second method connect the HMMs together according to the speech, and apply the viterbi algorithm on the connected HMM. Compared with the method that uses the recognition algorithm directly, this method is much more effective and has better lip synchronization performance.Then we implemented a system based on the second method. And we introduced how to decide the model structure and how to choose the proper recognition unit according to our system. We also introduced how to modify the general algorithm to make it work correctly on our system and how to parameterize the speech signal.