Representation And Reconstruction Of High-Fidelity Virtual Digital Human

Human virtual digitization is a hot research topic in computer vision and computer graphics,and it is widely used in many applications like AR/VR,holographic communication,film/game production,etc.Digitally representing humans and related objects such as the human body,human face,and clothing is the research core of the digital human.The representing content of the digital human contains the shape&appearance and semantic information,where the former aims to accurately describe the digital human’s 3D geometric shape and appearance material,and the latter is dedicated to effectively representing the digital human’s semantic attributes.They are respectively responsible for the digital human’s external representation and semantic description,and can be associated through the semantic parametric model.In addition,as different tasks have different needs,some are for efficient reconstructing,some are for effective training,and some are for semantic-level understanding.Thus it is necessary to design different digital human representations and reconstruction methods for different tasks.This thesis focuses on the representation and reconstruction of high-fidelity virtual digital human,and has achieved technical contributions and innovative results in the following aspects:A high-fidelity textured atlas representation and reconstruction for dynamic human reconstruction.With the development of depth imaging technology,realtime dynamic human reconstruction using consumer-grade depth sensors has been extensively studied.For this problem,we propose a method of fast generating spatiotemporal-consistent texture atlas for dynamic human reconstruction.Specifically,we convert the texture atlas sequence’s consistency into the mesh sequence’s segmentation problem.Then we propose a practical objective function to measure the segmentation quality by comprehensively considering the segmentation consistency and texture imaging requirements.Besides,we design an efficient method for fast calculating the triangle-wise correspondence of adjacent meshes,which is further utilized to help formulate and optimize the above objective function.Thanks to the consistency of generated results,the bitrate of generated atlas video can be drastically brought down,which means that our method can effectively reduce data memory occupation and the possible transmission bandwidth requirement for technologies like holographic communication.A high-fidelity human digitization based on neural implicit geometric representation via binocular images.With the development of deep learning,the community began to explore 3D geometric representations suitable for learning and reconstruction methods with easily accessible input.For this problem,we propose a representation that integrates the geometric constraints of stereo vision with the neural implicit representation.This representation is further utilized to recover the human body from a pair of low-cost rectified images.Specifically,we introduce the effective voxel-aligned features to enable depth-aware reconstruction.Moreover,the novel relative z-offset is employed to help restore fine-level surface details,which also improves the robustness,completeness,and accuracy of clothed human reconstruction by utilizing the depth priors.Compared with the previous works,our algorithm significantly improves the reconstruction accuracy and achieves state-of-the-art reconstruction quality.A high-fidelity semantic parametric head model based on neural radiance field representation.The semantic parametric model can act as a bridge between the digital human’s geometry&appearance and semantic information.For the given semantic parameters,the parametric model can be used to generate corresponding 3D representations.Here we focus on the semantic parametric head model and find that the existing parametric models based on traditional representation often suffer from limited expressive ability and have difficulty collecting geometric training data.To address this,we take the neural radiance field as the 3D proxy for representing the human head,and innovatively integrate the neural radiance field into the parametric representation of the human head.Thanks to the effective network architecture and well-design training strategy,HeadNeRF can render high-fidelity head images in real-time,support directly controlling the pose of rendered images,and independently editing the identity,expression,and appearance of generated images.We propose a series of improved representations and reconstruction methods for virtual digital humans.The research content of this thesis involves the digital human appearance representation for dynamic reconstruction,the digital human representation suitable for deep learning,and the digital human representation for parametric representation.Besides,we conduct extensive experiments to verify the effectiveness and superiority of these representations and reconstruction algorithms.We believe that the comprehensive improvement proposed by us will effectively promote the related research on the virtual digital human,which will take a significant step toward the future digital human.