3D Hand Reconstruction From Single Image Using Inverse Kinematics With Multi-geometric And Biomechanical Constraints

Extraction of 3D information and reconstruction of mesh models are important tasks in both computer vision and computer graphics fields.Recovering 3D human hand shapes from a single image is a hot topic of this type of tasks due to its widespread applications in computer animation,virtual reality,human-computer interaction,and so on.This paper proposes a novel method for 3D hand reconstruction from a single image,which applies MANO(a hand parametric model)to fit multi-geometric attributes and biomechanical constraints and then takes use of inverse kinematics to solve the issue.Concretely,a novel multi-objective convolutional neural network is first designed to extract abundant geometric attributes from the input image including 2D position and relative depth of hand joints,and the normal of planes defined by joints of each finger and the spam.Second,human biomechanical constraints are employed to constrain the solution space of human hand gestures as priors.Finally,with the help of MANO,the hand pose reconstruction is cast into a fitting optimization issue under constraints of biomechanical priors.What's more,we propose a novel method to address the optimization by combining the block-coordinate descent method and the inverse kinematics technique.Experiments show that the proposed method that utilizes both multi-geometric information and biomechanical constraints can achieve more accurate and reasonable hand pose than existing methods.In addition,analysis on intermediate results demonstrates that the novel multi-objective CNN is able to accurately predict the normal vectors of finger and spam planes.Finally,ablation experiments also show that these normal vectors can help improve reconstruction accuracy of hand poses.