| Three-dimensional(3D)hand pose estimation is the main component in the field of pose estimation,and it is also a key research direction in the field of human-computer interaction.It has broad application prospects in the fields of robotics,virtual reality,and remote control.The 3D hand pose estimation method based on deep learning has the advantages of low cost,easy portability,non-contact and so on,and its application prospect is broad.However,the acquisition of high-precision 3D hand poses is hindered by the characteristics of the human hand,such as high degrees of freedom,multi-shape poses,frequent occlusions,high similarity between fingers,and high latitude of 3D hand poses.This paper takes image-based3 D hand pose estimation as the research object,and conducts research on estimating high-precision 3D hand pose for data input in different formats.The main work and innovations of this paper are as follows:1)In order to improve the accuracy and robustness of the 3D hand pose estimation network with point cloud as input,this paper designs a finger point reinforcement strategy and a multi-level fusion attention module.Finger point enhancement strategy,which analyzes the characteristics of the point cloud of the hand,proposes a new feature as the prior knowledge of the network,improves the feature abstraction ability of the point cloud feature extraction network for the point located at the finger position in the point cloud,and strengthens the effective point Learning effects on network models.The multi-level fusion attention module,which introduces the channel attention mechanism in 2D image processing,improves the contribution of more effective local features in the point cloud to the global features,and realizes the screening of feature channels by the network.Ablation experiments and comparison experiments have carried out on the datasets ICVL and MSRA.The ablation experiments confirmed the effectiveness of the finger point reinforcement strategy and the multi-level fusion attention module.The comparison experiments verified the superiority of the method proposed in this paper.It visually demonstrates the improvement of the proposed algorithm in 3D hand pose estimation.2)In order to improve the performance of a 3D hand pose estimation network using heatmaps as intermediate states,this paper designs a progressive hourglass network.The progressive hourglass network takes the hourglass network as the basic unit,and designs a heat map fusion module and a multi-feature fusion module.The heat map fusion module introduces a spatial attention mechanism in the cascade process,and uses the heat map as spatial auxiliary information.The input to the next-level feature extraction unit is corrected to reduce the negative impact of heat map prediction errors on subsequent feature extraction;the multi-feature fusion module is used in the cascaded hourglass network to fuse multiple features of the hourglass network,such as input features.,intermediate features and heat map features,through the fusion of different levels of features,enhance the connection of the extracted semantic information in different processes,enhance the feature expression ability,and then improve the performance of 3D hand pose estimation.The ablation experiments and comparison experiments have carried out on the NYU dataset.The ablation experiments confirm the effectiveness of the heatmap fusion module and the multi-feature fusion module,and the comparison experiments verify the superiority of the method proposed in this paper.Finally,the improvement of the algorithm proposed in this paper for 3D hand pose estimation is visually demonstrated. |
