Research On Real-time Synchronization Technology Of Human Digital Twin

In the current golden age of the meta-universe,virtual digital people have gradually become an important part of the development of the metauniverse through the perfect combination of multiple scenes,which improves the immersion and interaction of the virtual world.However,the current technology of human digital twin based on RGB images cannot reach the real-time and low accuracy.Therefore,it is necessary to propose a new design scheme to realize real-time synchronization of 3D human digital twin.The SMPL-X body model,FLAME face model and MANO hand model are analyzed in order to achieve higher precision 3D reconstruction of human body.The above model was trained using a large public dataset Human3.6M dataset.According to the needs of model training and real-time processing of human body images,a real-time synchronization system of human body digital twins is designed.The system includes six subsystems:depth camera capturing system,image frame optimization system,body pose extractor system,face fitting system,hand pose estimation system and 3D display system.The grid model of the human body is iteratively optimized,subdivision-based processing of human motion capture data,elimination of representation limitations on the complex geometry of clothed human bodies,and finally,the achievement of real-time mapping speed in human body posture twinning have improved the real-time synchronization robustness of human digital twinning.In the stage of image frame optimization,after replacing the complete convolution network in the benchmark algorithm with Transformer architecture,the trained mannequin compares and evaluates the algorithm under 15 different action scenes in Human3.6M datasets.The results show that the more accurate proportion of key points and human posture can be estimated in real time by using this algorithm.The results of four test error values are reduced,that is,higher accuracy is achieved.MPJPE average decreased from 52.0mm to 50.4mm.The P-MPJPE average decreased from 40.6mm to 38.8mm.The mean N-MPJPE decreased from 50.0mm to 48.2mm.The MPJVE average decreased from 3.35mm to 2.72mm.The results show that this algorithm can be used to estimate more accurate key point ratio and human posture in real time.