The Research On 3D Human Pose Estimation Base On Pressure Information On Human Body

In the field of 3D human pose estimation,many current studies have achieved high accuracy of estimation results by using multi-view visual data.However,there are significant limitations in using multi-view visual information,mainly in the range of usage scenarios.There are few scenarios in real life where utilize multi-view cameras to capture the 3D human pose.Therefore,to address the limitations of 3D human pose estimation in the range of application scenarios,this thesis proposes to utilize the pressure data between human body and clothing to estimate 3D human pose.If the pressure data between human body and clothing can be utilized for 3D human pose estimation,3D human pose estimation can be realized more conveniently and daily.To the best of our knowledge,only a few studies have introduced pressure data in 3D human pose estimation.In those studies that have introduced pressure data,most of them have utilized pressure data between the human foot and the ground or between the human body and a bed or chair.These pressure data occur in a plane.This thesis proposes using pressure information between the human body and clothing to estimate 3D human pose.Based on this idea,This thesis mainly relies on pressure data at key points on the human body which is not in a plane to estimate 3D the human pose.In addition,multimodal 3D human pose estimation methods are designed in this thesis to improve the accuracy of monocular 3D human pose estimation by introducing pressure data.The main research of this thesis can be summarized as follows:(1)A data collection system was designed,which includes a pressure data collection system and a visual groundtruth collection system.The pressure data collection system consists of mainly 20 pressure sensors,a microcomputer that converts the pressure sensor data into pressure data,and protective gears that hold the pressure sensors.This thesis simulates the pressure on the human body when wearing clothing by tying protective gears.The data obtained by this method can be directly utilized in some applications of 3D human pose estimation for sports that require protective gears.The visual groundtruth collection system mainly consists of two cameras.The video captured by the cameras is first utilized to obtain the 2D human pose for each frame via Open Pose.Subsequently,the two corresponding 2D human poses of each frame are lifted to 3D by triangulation.(2)Based on the collected pressure data,a 3D human pose estimation method mainly based on pressure data and human body shape parameters as input is designed called Pressure Pose.A bidirectional LSTM based network is designed to estimate the 3D human pose.The estimation error of 43.4 mm on average for each joint is achieved on data with trained human body shape of people,and a estimation error of 150.9 mm on average for each joint on data with untrained human body shape of people.(3)Based on the collected pressure data,the multimodal fusion methods called Post Fuse and Naive Fuse fusing monocular 2D human pose and pressure data was designed in order to improve the generalization of the study while maintaining a high practicality of the study.The accuracy of the 3D human pose estimation is improved by introducing pressure data.The final implemented multimodal methods estimate the accuracy of 3D human pose better than the method that relies only on monocular human 2D pose as input and the method that relies mainly on pressure data as input.In summary,based on the idea of using pressure data between human body and clothing,this thesis designs a data collection system,a method that relies mainly on pressure data to estimate 3D human pose,and the multimodal methods that fusing monocular 2D human pose and pressure data to estimate 3D human pose,respectively.The proposed methods achieve competitive results in certain scenarios.Therefore,the research in this thesis has a high practical application value and proposes a new solution for 3D human pose estimation.