Research On Adversarial Training Model With Reprojection Constraint For 3D Human Pose Estimation

With the rapid development of human-computer interaction technology,machines need to correctly recognize and understand human behavior through computer vision.3D human pose estimation is a hot research topic in the field of computer vision.The recent rapid development of deep learning has made more and more scholars use neural networks to conduct 3D human pose estimation,and achieved good results.However,most of these methods are supervised methods.They use ground-truth 3D pose data as supervision information to achieve high performance,but annotating these supervised 3D pose data requires a lot of time and resources.Therefore,weakly supervised and unsupervised methods for 3D human pose estimation have become a research hotspot.This paper studies 3D human pose estimation methods based on a single image using generative adversarial network(GAN).When building an adversarial training model,in order to improve the accuracy of 3D human pose estimation,not only the constraints that the generated 3D pose should maintain reasonable when viewed from multiple angles can be considered,but also the constraints that the generated 3D pose should be consistent with the input 2D pose can be considered.In this paper,two 3D human pose estimation methods based on reprojection constraints and adversarial training constraints are proposed.The main results obtained are as follows:(1)Based on the fact that the generated 3D pose should maintain reasonable when viewed from multiple angles,this paper proposes an unsupervised 3D human pose estimation method based on single-view-multi-angle consistency constraints.The generator first performs 3D pose estimation and corresponding weak perspective camera estimation from a 2D pose extracted from a single image,then the generated 3D pose is randomly rotated at multiple angles and re-projected through the estimated weak perspective camera,and then the generator regenerates the corresponding 3D poses and weak perspective cameras base on the 2D reprojections.Since the 2D reprojections from the same angle should be consistent,the model mixes the generated 3D poses and cameras to generate multiple 2D reprojections during training,and imposes a single-view-multiangle consistency loss function to improve the accuracy of 3D pose estimation.Experimental results on public datasets show that the proposed method outperforms stateof-the-art methods by 15% in evaluation metrics,and the effectiveness of each constraint is verified by ablation experiments.(2)Based on the fact that the generated 3D pose should be consistent with the input2 D pose,this paper proposes a weakly supervised method based on 2D-3D consistency constraints.The framework consists of a generator,a discriminator and a reprojection network.The generator is used to perform 3D pose estimation from a 2D pose extracted from a single image,and then the generated 3D pose is rotated by a random angle and reprojected to 2D using the reprojection network.The discriminator performs joint discrimination on the 3D pose,2D reprojection and KCS matrix calculated from the 3D pose to impose 2D-3D consistency constraints.In order to reflect the influence of bone distance on the importance of joint angle information,the KCS matrix is weighted by a formula.Finally,the model trains the three networks synchronously to effectively reduce the influence of errors in the input 2D poses,which improves the accuracy of 3D human pose estimation.Experimental results on public datasets show that the proposed method outperforms state-of-the-art methods by 24.7% in evaluation metrics,and the effectiveness of each constraint is verified by ablation experiments.