3D Face Reconstruction Optimization Algorithm Based On Gaussian Process Latent Variable Model

Under the background that some concepts like metaverse and applications of digital human such as Meta Human and Chat GPT have received wide attention,the technology of 3D face reconstruction based on 2D images,which is an important part of the image mapping of virtual characters,has more and more research value and practical significance.By doing 3D reconstruction for target portraits,3D facial feature information of the target persons can be obtained,which can provide convenient and effective basic template and prior information for editing the expression and posture of the virtual characters or picture driving.However,due to the lack of 3D information in2 D images,it is very difficult to reconstruct model from monocular images,especially from wild images.3D Morphable Model is a powerful tool to deal with this problem.It can be used as priori knowledge in traditional methods or deep learning.General 3D Morphable Model uses linear representation vector to describe the facial shape,but in face recognition fields or others,face is usually described as a nonlinear object,which is more in line with intuitive cognition of people.Therefore,this thesis reconstructed the whole process of the establishment of representation model and the reconstruction of3 D editable grid model from input 2D images,and introduces Gaussian Process Latent Variable Model to study the method of constructing nonlinear deformation space of face.The main research work of this thesis is as follows:1.Propose the method of extracting latent variables of face shape based on GPLVM.In this thesis,a multi-kernel GPLVM is used to extract the representation of latent variables from the sample dataset,and the extracted latent variables are decomposed and mapped,and the generation of latent variables is controlled by a set of coefficients.2.Propose the modeling method of face regionalization and endow latent variables with semantic structure information.In this thesis,after manually dividing the face region,each part is modeled separately,and the structure information is shared by a coregionalization function to realize the semantic decomposition of latent variables.By setting anchor points inside the region,a reasonable reconstruction model can be obtained by solving the least square solution of an overdetermined equation for the smoothing problem between the region boundaries.The experimental results show that in the monocular image reconstruction task,the mean errors obtained by the nonlinear latent variable method and the regional modeling method are 2.0636 mm and 2.0265 mm,respectively,which are reduced by0.1149mm(5.27%)and 0.1520mm(6.98%)compared with the linear representation vector method.In the multi-visual image reconstruction task,the mean error obtained by the method of latent variable is 1.46 mm,which is at least 0.57mm(28.1%)less than that obtained by other excellent algorithms based on linear vectors and optimized by neural network,which proves that latent variable can effectively provide the reconstruction accuracy of face.