Research On 3D Face Expression Synthesis Based On Multi-Branch Spatially Varying Convolutional Networks

Facial expression synthesis is a difficult research problem in computer graphics and computer vision.3D facial expression synthesis is a popular research area that involves controlling the deformation of a 3D facial model to produce realistic expressions or animation sequences.Recently,deep learning-based techniques,such as convolutional neural networks,have led to rapid developments in 3D facial expression synthesis,making it an area of interest for researchers and widely applicable in various fields,including virtual reality,movie animation production,digital games,and security technology.The human face has a complex physiological structure with individual characteristics and rich expressions that convey powerful emotional information.Even slight changes in facial expression can convey vastly different emotions.Therefore,the 3D facial expression synthesis technology mainly faces the following technical difficulties and challenges: one is how to ensure the controllability and diversity of 3D facial expression synthesis,and obtain rich and diverse 3D facial expressions that meet users’ needs;the second is how to synthesize real and natural 3D facial expressions while maintaining the given facial identity feature information;the third is how to completely extract the high-dimensional and lowdimensional feature information of the 3D facial mesh,so as to ensure that the generated 3D facial expressions contain rich,realistic and reasonable texture information;the fourth is how to improve the ease of use of 3D face synthesis technology to lower the threshold for users.Aiming at the above-mentioned problems,we study the high-realistic 3D facial expression synthesis technology from two aspects: interactive editing of 3D facial expressions and automatic synthesis of 3D facial expressions,based on spatially varying convolutional networks.The research work and contributions of this paper are as follows:1.We propose a multi-level 3D facial expression editing method based on a multibranch spatial varying convolutional network.This method can generate new expressions with rich details that meet user expectations by applying slight displacements to a small number of control points on a facial model.Firstly,the high-frequency region division module of the network model is used to identify the high-frequency regions of the facial mesh.Then,the entire facial mesh and control point constraints are input into the rough editing module to generate basic expressions.Meanwhile,the high-frequency regions of the facial mesh and control point constraints are input into the fine editing module to generate expression details.Finally,the basic expression and expression details are merged to obtain the final new expression.The hierarchical processing of expressions can generate fine expression details and make the entire network run faster.The introduction of spatial varying convolution in the branch module greatly improves the accuracy of expression editing.We propose a new spatio-temporal correlation criterion and improve the K-Means algorithm for automatic division of high-frequency regions.We also propose a new loss function that combines vertex position constraints and vertex normal constraints,improving the accuracy of the entire network model.Experimental results show that the method can generate highly realistic 3D facial expressions with rich details according to user editing requirements.2.We propose a multi-scale 3D facial expression synthesis model based on a multibranch spatial variation convolutional network.This model can automatically generate diverse and natural 3D facial expressions while maintaining the identity information of the facial mesh.The model includes three spatial variation convolutional neural network branches: the large-scale spatial variation convolutional network(L-sv Conv),the mediumscale spatial variation convolutional network(M-sv Conv),and the small-scale spatial variation convolutional network(S-sv Conv).L-sv Conv is used to capture large-scale feature information,S-sv Conv extracts tiny and fine feature information,and M-sv Conv captures feature information between the two.Then,a selective fusion method is proposed to weight and merge the feature information extracted by the three branches,and self-attention mechanism is introduced to further enhance the spatial non-local representation ability of the feature information.Finally,the feature information is decoded to generate a 3D facial mesh with new expressions.Experimental results show that the model can automatically generate diverse and natural 3D facial expressions with rich details.