Unsupervised Dense 3D Human Body Correspondences From Point Clouds

3D human correspondence estimation is one of the key problems which can help solve the semantic driven technology of 3D human modeling.In recent years,shape correspondence problem has become a hot research direction in Computer Vision.With the development of deep learning,the performance of human dense correspondence estimation based on point clouds has been greatly improved.The use of point clouds and its sequence will further improve the adaptability and stability of the estimation of dense correspondence of 3D human shapes,and provide better help for understanding human behavior.First of all,at present,most of the methods to estimate the correspondence of 3D human shapes use the mesh models as the inputs.In fact,the mesh models are difficult to obtain directly,and most of them are generated by simulation,and there are few real datasets,so which brings a lot of trouble to the relevant research.Second,most of the relevant works are to use the complete two models as inputs and estimate the correspondence among them.However,in fact,the images or videos captured by the camera are all in a perspective view,so the estimation of correspondence under occlusion or single perspective is a promising research direction.Finally,unsupervised deep learning has become an important topic in the field of 3D human correspondence estimation because it is difficult to obtain the groundtruth correspondence between two human shapes and the existing human datasets are lack of labeled datasets.This paper explore the present situations of the method for estimating the density of 3D human point clouds.Firstly,a traditional method based on function map is proposed.Secondly,based on the traditional method,a supervised deep learning method is proposed by constructing supervised error function and replacing manual calculated descriptor with deep learning feature extraction network.Finally,an unsupervised deep learning method is proposed by introducing deformation module and motion analysis.The main work is as follows:(1)In this paper,a method based on function map is proposed to estimate the dense correspondence of 3D human point clouds.This method uses single view point clouds instead of complete mesh models as inputs,and solves the challenging problem of human dense correspondence of single view non-rigid point clouds.Different from the most of the current related works,which are based on the mesh model with triangular information to estimate the correspondence,the innovation of this method is to directly process on the point clouds of single perspective.Because the single perspective point clouds lose the human body information and do not contain triangular information,these current related works cannot effectively calculate the correspondence.The method is divided into the following steps: Firstly,the template point clouds are used to complete the missing human body information of the single view point clouds,so as to obtain the complete human body structure,so that the Laplace-Beltrami operator(LBO)can be calculated effectively.Then,the features of point clouds are calculated by LBO,so the geometric information in Euclidean space is converted to the spectral information in Frequency space.Finally,a linear function map matrix is calculated to characterize the dense correspondence.Experimental results show that the proposed method improves the performance of 3D human correspondence estimation on CAPE,TOSCA,SCAPE and FAUST datasets compared with the existing methods.(2)In this paper,a method based on deep learning is proposed to estimate the dense correspondence of 3D human point clouds.Based on the previous work,deep neural network is introduced to extract geometric information of point clouds,and the supervised error is constructed by the groundtruth correspondence as constraints.The method is divided into the following steps: Firstly,the template point clouds are used to complete the missing human body information to obtain the complete human body structure,so that the Laplace-Beltrami operator(LBO)can be calculated effectively.Then,the features of point clouds are obtained by features of point clouds extraction network,and the geometric information in Euclidean space is converted to the spetral information in frequency space.Finally,a linear function map matrix is calculated to characterize the dense correspondence.The error of the correspondence is calculated by the groundtruth correspondence and the estimated correspondence,then supervised learning is carried out.This method can calculate a better linear function map to characterize the dense correspondence.Test results on CAPE,SCAPE and FAUST datasets show that the proposed method improves the performance of human correspondence estimation.(3)An unsupervised dense correspondence estimation method for 3D human point clouds based on single view point clouda sequence is proposed.On the basis of the previous two works,this method introduces point clouds deformation error based on point clouds deformation and motion consistency loss based on human Timing Analysis,and takes single perspective point clouds sequence as inputs,proposes an unsupervised dense correspondence estimation method for 3D human point clouds.The single frame module: Firstly,the template point clouds are used to complete the missing human body information to obtain the complete human body structure,so that the Laplace-Beltrami operator(LBO)can be calculated effectively.Secondly,features of point clouds are obtained by point clouds features extraction network,and the geometric information in Euclidean space is converted to the spectral information in Frequency space.Then,a linear function map is calculated to characterize the dense correspondece.Deformation module: the template point clouds is transformed into the space of the completed point clouds,the deformed and reordered template pint clouds are obtained through the deformation module,both which and the completed point clouds are computed to the deformation loss function.Timing Analysis module: human motion consistency based on Timing Analysis.By converting the completed point clouds of three consecutive frames into the space of template point clouds,their motion consistency error is calculated.These two loss functions work together as geometric constraints for unsupervised learning.The test results of CAPE,SURREAL,HM36 and DFAUST datasets show that this method improves the performance of human correspondence estimation.