Research On Human Pose Estimation Based On Visual And Inertial Sensing Information

With the rapid development of deep learning,human pose estimation has been widely used in animation,human-computer interaction,motion analysis,and other fields.At present,human pose estimation is mainly realized by vision-based methods and inertial sensing information-based methods.The vision-based approach uses computer vision technology to process images or videos to obtain the position of human body joints;the inertial sensing information-based approach installs inertial measurement units on the human skeleton to collect acceleration and rotation data of the limbs to achieve the estimation of body posture.However,both methods have certain limitations.The vision-based approach is susceptible to factors such as occlusion and the models are often complex;the inertial sensing information-based approach suffers from time accumulation error and environmental noise interference during data acquisition and requires binding a large number of sensing devices to capture limb movement information,otherwise,it is difficult to provide sufficient data constraints to the model to obtain accurate pose.Therefore,this thesis proposes two human pose estimation models to alleviate the data limitations and multi-device invasiveness problems in inertial sensing informationbased methods,and the model complexity and image occlusion problems in visionbased methods.The main research of this thesis is summarized as follows:(1)To alleviate the data limitations and multi-device invasiveness problems in inertial sensing information acquisition,this thesis proposes a multi-stage jointconstrained pose estimation model PFPose based on sparse inertial sensing information.The model uses six sets of inertial measurement unit data as input,and by rationalizing the joint distribution in the human joint tree and introducing free joints and core joints as intermediate-stage joint constraints,it alleviates the problem of lack of data constraints for the regression pose of sparse inertial sensing information is alleviated.In addition,a dynamically tuned low-pass filter is proposed to generalize and filter the global pose output in the online pose inference process to reduce jitter.(2)To alleviate the model complexity and image occlusion problems of visionbased methods,a lightweight pose estimation model SGORN-ORPSM based on the fusion of vision and inertial sensing information is proposed in this thesis.this model fuses inertial sensing information to assist the pose regression of vision-based methods and alleviates the problems such as image occlusion.In addition,a 2D pose estimation model SE-GRN based on Ghost Reg Net lightweight network and SE attention module is proposed to alleviate the model parameter complexity.Then,the ORN-Refine model is proposed for weight-based cross-view fusion of the initial heat map to further improve the 3D pose prediction accuracy.(3)To verify the prediction effects of the two models,comparative experiments,ablation experiments,and visualization experiments were conducted on multiple datasets in this thesis.Compared with the baseline model,the PFPose model reduced the error of the SIP Error metric by 1.18%-4.75% on multiple datasets;the SGORNORPSM model reduced the error of the MPJPE metric by 1.2% and the number of parameters was reduced by 1/3.The experimental results showed that both models had better prediction results.