3D Hand Pose Estimation Based On Hierarchical Ensemble Convolutional Neural Network

As one of the most frequently used human body parts,the hand plays an important role in human-computer interaction.Accurate 3D hand pose estimation has become a key technology in the fields of human-computer interaction,and vision-based 3D hand pose estimation has gradually become a research hotspot.At present,most of the 3D hand pose estimation tasks are based on depth images,but such images require special camera acquisition,which are not easy to obtain.However,the application scope of RGB images is more extensive,the constraints on users are smaller,and the acquisition costs are lower.Therefore,this paper develops the research on 3D hand pose estimation from RGB images.Due to the lack of absolute depth information,as well as severe self-occlusion and high flexibility brought by hand structure,the task of hand pose estimation based on a single RGB image is extremely challenging.Aim to above problems,a 3D hand pose estimation convolutional neural network based on the characteristics of hand structure is designed in this paper.The main works are as follows:(1)Combined with hand structural characteristics,a novel five-layer ensemble convolutional neural network(5LENet)to extract hand features more effectively is proposed,in which hand pose estimation is decomposed into five single-finger pose estimation by using a hierarchical ensemble network.More deeper and representative finger feature information is extracted to estimate the 3D finger pose respectively,and then the features generated in the process of 3D finger pose estimation are fused to estimate a full 3D hand pose.It can not only extract more effective features,but also can enhance the association between fingers through the fusion of finger features.(2)Five 3D finger pose constraints are newly added in 5LENet to improve pose estimation accuracy,which can not only promote 3D finger pose estimation effectively,but also can form soft constraints on 2D finger pose estimation by means of error back propagation,which helps to further indirectly optimize the precision of 3D hand pose estimation.(3)According to the structure of hand,a model of hand with the Palm and middle finger being connected is built.It is because that middle finger locates in the middle of five fingers,and its connection with the Palm is more tightly.Therefore,we connect the Palm to middle finger,which can successfully solve the accuracy degradation problem caused by connecting the Palm with multiple fingers.(4)A large number of compared experiments are conducted on the RHD and STB public datasets,yielding new state-of-the-art in 3D hand pose prediction accuracy,which exceeds the estimation accuracy of other advanced methods,and verifies the effectiveness of the designed network.