Research On Domain Adaptive Methods For 3D Hand Pose Estimation

In recent years,3D hand pose estimation is an increasingly popular research direction in the field of computer vision,and it has applications in many fields such as virtual reality,augmented reality,and human-computer interaction.Due to the lack of local features and complex posture configuration of hands,occlusion scenarios such as object occlusion and selfocclusion,as well as inherent scale and depth ambiguity of RGB images,it is a very challenging task to reliably reconstruct corresponding hand pose from monocular RGB images.Currently,the common practice for this task is to use deep neural networks to rely on 3D labels for fully supervised learning in the training phase,so that the model can accurately predict 3D hand poses.However,existing methods rely heavily on the supervision of 3D tags for hand gestures,and obtaining them from real images is expensive,often requiring repeated calibration in multiple views and labor-intensive manual annotation.In order to reduce the dependence on 3D labeled training data,this paper proposes a domain adaptation method for 3D hand pose estimation.This method combines the MANO parameterized hand model commonly used in the field of hand pose estimation with the domain adaptation method,which can transfer the model trained on the synthetic dataset with low cost to the real dataset,and achieve good results.This method does not need to rely on any labels of the real dataset to train the model,and only needs to obtain geometric information for the 3D hand pose estimation of the real image through the easy-to-obtain 2D joint detection point information.The method is divided into two stages.The first stage uses the synthetic dataset as the source domain dataset to train the model,and then uses the hand pose domain adaptor to transfer the model to the target domain dataset.The domain adaptor is a model ensemble consisting of multiple sets of hand pose estimation models,and the model training is guided by the domain adaptation loss consisting of the difference value of the model prediction results and the feature consistency loss.In the second stage,the adaptive model of the first stage is used as the pre-trained model.Without the use of 3D labels,the model is finetuned by selfsupervised training using a geometric loss function constrained by 2D joint detection points.Since labeled data is not used,we also use a regularization loss that includes biological constraints to rationalize the output of the hand pose estimation model.We demonstrate for the first time the feasibility of training accurate 3D hand pose estimation models with domain adaptation methods without relying on manual annotations.We design and validate experiments on the synthetic Obman dataset and the real dataset Freihand.The experimental results show that our domain adaptation method can achieve very competitive performance without using any hand pose 3D annotations,even compared to partially fully supervised methods.