Research On Synthesis Technology Of Facial Animation Driven By RealSense

Facial expression is one of the most important ways for human beings to communicate emotional state in social life,which,together with gestures,language,sound and body postures,constitute the basic communication system of human beings in the social environment.With the improvement of computer performance,the research of virtual face expression display technology,taking facial expressions as objects,has become one of the most active directions in the field of computer vision.Its research results are widely used in virtualteaching,facial expression analysis,mixed reality,artificial intelligence,film special effects production and other fields.This paper presents a real-time virtual face expression display system based on RealSense and Unity 3D.The system does not require face marking.The users just need to perform in a natural environment and use RealSense to capture the user’s 3D facial video as a driver.This paper introduces the expression synthesis scheme and face modeling techniques in the field of virtual face expression display,then analyzes their advantages and disadvantages.It proposes the use of image-based modeling technology solutions and uses the virtual face synthesis scheme based on the face feature point drive;This article uses depth information preprocessing images that collected by RealSense,which reduce the background interference to face detection tracking.The Adaboost algorithm based on LBP operator traces and detects the face position,improves the time efficiency of face detection,and uses the mean shift algorithm,in combination with facial skin color,alternately detecting and tracking the face,improving the time efficiency of the capture link,and enhancing the ability of tracking the multi angle face;Then,on the basis of the capture of the human face and the active model of the prototype,it combines the global texture and the geometry,to extract the two-dimensional features of the human face.For the model control unit based on the expression coding system,the extracted two-dimensional feature points are combined with the depth map information to calculate the change trend and pass it to the model;Combining with the previous algorithm,RealSense-based color depth camera realtime detection and tracking face-driven virtual face expression display system is implemented in the Unity 3D game engine.The prototype system has been tested on ordinary computer platforms.The results show that the technical scheme improves the time efficiency of the system and enhances the robustness of face animation under complex backgrounds.the face model can imitate the facial expression of the performer’s basic facial expressions,and Facial animation can reach video shooting speed of 30 frames per second.Compared with other existing virtual face expression display technologies,this system is cheaper and easy to operate,which can be applied to the fields of virtual reality,game entertainment,and so on.