Real-time Simulation For3D Subtle Facial Expression Driven By Actions Captured With Kinect

Generating compelling dynamic animated facial expressions is an extremelyimportant and challenging aspect of computer graphics. In recent years, virtual charactershave been well designed and animated in computer games, advertising, and filmproduction, which makes subtle facial expressions of character animation is becomingmore and more important. This paper proposes a new technology which generatedreal-time facial expression animation with subtle facial expressions, so as to drive the3Dfacial mesh model generated virtual3D character animation with subtle movements.Firstly, in order to capture the user’s real-time different expressions, we useMicrosoft’s Kinect3D to track the user’s facial expressions in real-time, then decomposethe motion data into two components: the rigid movement of the head and the change ofthe facial expression. Compared to other human motion capture systems which rely onspecific hardware device, Kinect reduces system hardware and maintenance costs, and forthe natural environments of complex background it has good adaptability.Secondly, by making use of the local-detail preserving property of the Laplaciancoordinate, we clone the captured facial expression and posture onto a neutral3D facialmodel using Laplacian deformation. Eventually produce a virtual3D character animationwith the eyes and mouth movement, and consistent with the expression of the user as well.Thirdly, in order to produce subtle facial expressions such as wrinkles in real-timefacial animation, we generate the coarse skin textures with pore details and those normalmaps, and use blending wrinkle maps to render animated details on a human face based ona GPU shader program. According to the action units captured by the Kinect, we calculatethe dynamic texture mapping and the weight of wrinkles, and apply the wrinkles functionto simulate the motion of wrinkles. As a result, our experiments generate motion-drivenreal-time facial expression animation.Finally, according to professional graphics programming interface OpenGL andadvanced shading language GLSL, we design and implement a real-time subtle facialexpressions simulation system. Experimental results show that our method can generate facial details in real time. The method suits the fields such as digital entertainment andvideo conferencing．...