Research On Key Technologies And System Implementation Of Virtual Avatar Driven By Real-time Data

"Digital twin",that is,"virtual avatar",or "avatar".It’s a virtual character of a physical world human in a virtual space.Existing real-time driving methods for "avatar" either cannot achieve common driving of avatar faces and limbs,or cannot meet the driving requirements for avatar models of various specifications,or there are problems such as real-time performance,model integrity and model position determination when driving in a virtual scene.Based on these key issues and current situation,this paper conducts research on the subject of real-time driving of avatar in virtual scenes with a panoramic perspective.The related work and innovations are as follows:In terms of avatar driving:Firstly,the facial driving of avatar is completed through expression estimation based on 34 facial feature points.Secondly,in the aspect of pose driving,while calculating the body pose of the avatar based on the real-time posture data of the driver collected by the 33 posture feature points and the agreed parent-child relationship between the key bones’joint of avatar,a model specification difference solving method is proposed:the coordinate system of the bone to be driven is converted from its model domain to the bone domain of its agreed child bone,to solving the problem that the pose driving method above may not be reusable for avatar with different specifications.Based on the above research results,a real-time avatar universal driving algorithm based on 67 human landmarks is proposed.Aiming at two aspects of the avatar driving in the virtual scene:Firstly,a root position-based avatar dynamic fixation method is proposed:the global root position node is introduced before collecting driver’s pose data and driving the avatar.Each frame of pose data is reorganized and calculated based on the root position node,and at the same time when the model is driven,by calculating the average difference between the hip bone position and the root position node in N frames of pose data,Dynamically calibrate the position of avatar in the virtual scene,so that the avatar can always move in a spherical space with the expected placement position as the center of the sphere.Solved the problem that avatar needs to be in the expected position in the virtual scene and accurately reflect the movement trend of driver’s jumping and squatting postures when driving.Secondly,a driving optimization method based on animation fusion is proposed:by cutting off the connection between some bones and the ancestor bone Hip,the disconnected bones will be driven by pose data,and the remaining bones will be driven by animation.Solved the preemption conflict of pose data and animation on model bones.By integrating animation,the driving performance is greatly optimized.Finally,based on the research results of this paper,the "virtual conference room" system is implemented,which supports multiple users to drive their avatar images online in real time.At the same time,solving the problem of insufficient computing power and graphics rendering capabilities of users’ lightweight devices through cloud services.