| Human 3D reconstruction refers to estimating human shape and pose in 3D space by using clues related to human postures,such as human motion video,human 2D key points,and other information.Traditional methods use professional motion capture equipment and depth cameras to collect human pose and shape.With the development of deep learning technology and the accurate human pose data collected by professional equipment,it is possible to train a deep neural network taking monocular video as input to estimate 3D human shape and pose.However,due to the lack of action diversity and data volume of 3D datasets,the training effect of the existing methods is poor,and the estimation results also have some problems,such as action jitter and so on.In order to solve the above problems,the algorithm introduces the unsupervised learning method of adversarial learning to assist the supervised learning with training.The generator and discriminator networks that affect the performance of adversarial learning are improved,increasing the accuracy of the algorithm.For the generator part,the attention based temporal encoder(ATE)is introduced to dynamically fuse the picture features at each time,which makes full use of the temporal information of the input video.For the discriminator part,we propose a discriminator based on graph convolution neural network(GCNdiscriminator),which takes into account the human skeleton topology information in the process of evaluating the input motion sequence,so that the discriminator can supervise the generator more effectively.The thesis also proposes two post-processing methods:rotation disentangled smoothing method and motion a priori optimization,which are used to eliminate jitter and correct error in estimation results respectively.Based on the above algorithm design,the algorithm is implemented in software.The software can visualize the estimation results in two ways:rendering the human body mesh coordinates for two-dimensional video display and generating FBX files to drive the human body model in the 3D animation engine.After the software implementation,the algorithm proposed in this thesis is evaluated on two public in-the-wild datasets 3DPW and MPI-INF-3DHP.The results show that the accuracy and acceleration error of this method are better than the baseline method.Finally,taking a variety of sports motion videos in the internet as the input of the algorithm,the estimation results are obtained and visualized.The estimation results are analyzed to prove that the algorithm has good generalization ability. |
