Design And Implementation Of Video Character Lip Modification System Based On Generative Adversarial Network

When watching a video,the lip of the video character is not synchronized with the audio,which will greatly affect the user’s watching experience.At the same time,due to commercial requirements such as video advertisement insertion and film dubbing,lip sync has also attracted more and more attention.Therefore,the research of lip sync has become a hot topic in the field of computer vision.The existing lip sync model is’mainly based on static images to output a lip sync video matching the target voice,but for dynamic and speaking characters,the lip synchronization is often ineffective.At the same time,the problem of face image and video frame mismatch exists in the video output by this method.To solve these problems,this paper proposes a lip sync model based on generative adversarial network,which is suitable for any identity video and keeps face fit.The input video is preprocessed to provide the face and cheek information for the generator.The generator uses gate recurrent unit to extract the features of the audio data,and learns the mapping relationship between the lip shape and the audio by combining the self-attention mechanism.At the same time with the lip sync discriminator and an additional face fitting discriminator for game learning,output lip sync and face keeping fit video.The validity of the model is verified by comparativeexperiments.On the basis of this model,this paper designs and implements a lip modification system for video characters,and uses the lip sync model to assist video creation and editing.Through the multi-module communication between the front-end,the server,and the model,the lip synchronous processing of the input video and the target audio is completed.Finally,the system function and performance test verify the usability of the video character lip modification system.