Accurate Expression Modeling Method Based On High-precision 3D Face Model

Nowadays,with the explosion of the concept of metaverse,the generation of digital people is also attracting more and more attention.The most important part of the digital human generation process is the expression of facial expressions.For example,in AR/VR,virtual anchor and game production,the use of high realistic facial expressions can give users a near-perfect experience,so it is very important to study how to generate high realistic 3D digital facial expressions.In this paper,we propose a method for accurate modeling of expressions based on high-precision 3D face models,using a highprecision structured light system to capture highly realistic 3D models of faces,parametrically representing the obtained high-precision face models,obtaining face driving parameters based on 2D face videos,and using the obtained face parameters to drive parametric face models,so as to obtain high-precision,highly realistic digital human expressions.The specific work is as follows.(1)Face 3D data acquisition system: Research on 3D reconstruction algorithm based on structured light and realize a system based on high precision structured light for face data acquisition.In this paper,a structured light system is built to collect 3D face point cloud data,and the 3D point clouds of face expressions are collected from multiple angles,and then stitched together using the alignment algorithm,and then mapped to the 3D model texture after the meshing operation,and a DSLR camera is added to the structured light system for the acquisition of 3D model face texture,and then the acquired high-resolution 2D pictures are used as texture mapping to the 3D model.(2)Design of orthogonal expression base and database acquisition: We proposed a series of 37 different expression actions based on the face characteristics with reference to the face muscle model,and then recruited volunteers and used the face 3D data acquisition system built in(1)to carry out 3D face database acquisition on these volunteers,so as to obtain a high-precision and highly realistic 3D face database.Because our database has the advantages of high precision,rich expressions and sufficient samples,we can generate virtual expressions with high realism on the basis of this database.(3)2D video-based expression-driven system: The 2D expressions for training are collected in a 3D model,and then an adjustable,3D expression base is designed to find the expression coefficients of the training dataset by using automatic optimization.The two-dimensional images in the training set are trained with the obtained expression coefficients to obtain a highly robust and efficient recognition model for video expression driving,which can drive highly realistic 3D virtual expressions using 2D video.In this thesis,we conduct detailed research and experiments on the above work,construct a high-precision face expression database,and conduct a large number of experiments using the actual collected data.The experimental results show the effectiveness and accuracy of the expression modeling system designed in this paper.The deep learning method is used to drive the parametric face model to achieve expression synthesis,and the synthesized face animation is realistic and the face action is natural,which provides a reliable solution for the generation and driving of virtual human expressions.