Research On Real-Time Gesture Recognition Algorithm Based On Depth Information

With the continuous development of the electronics industry,the forms of electronic devices are constantly evolving,and the way in which people interact with electronic devices has undergone tremendous changes.As a direct and natural way of interaction,gestures are widely used in VR(Virtual Reality)and MR(Mixed Reality)devices.Accurate and robust gesture recognition algorithm has also been a hot research topic.Vision-based gesture recognition technology faces many challenges,such as the complex background,ambient light,occlusion and the beginning of the gesture is difficult to judge.The emergence of devices capable of capturing deep information,represented by Microsoft's Kinect,provides a good basis for solving these challenges.Based on the research of a large number of hand gesture recognition related literatures in recent years,this paper proposes a hand gesture recognition algorithm based on the depth information provided by the Kinect.The proposed algorithm includes two parts,static gesture recognition and dynamic gesture recognition.In the part of static gesture recognition,firstly this paper determines the local region where the hand is located based on the skeleton point information provided by the Kinect,and uses the elliptical skin color model and depth information for this region to segment the hand region without facial interference,and the binarized hand image is obtained;then contour extraction and convex hull detection are performed on the hand image and the exact boundary of the hand area is determined.After scaling the hand region to a specified size,a collection of possible finger points is obtained according to the result of the convex hull detection.Then the wrist point captured by the Kinect and the centroid calculated from Hu moment are used to determine the orientation angle of the palm.Then the palm's orientation angle is used as a reference to filter out the interference points that exceed the normal angle of the finger point,and the shape features of the finger is used to filter out other interference points to obtain the final set of finger points.Finally,according to the detected finger point information and the Hu moment of the hand image,a hand feature vector is constructed to perform SVM(Support Vector Machine)training,and the training result is used to perform static gesture recognition.Experimental results show that the algorithm has high accuracy and good robustnessIn the part of dynamic gesture recognition,firstly the video frame is classified to construct a finite state automaton,and the gesture sequence is captured by the transformation of each state of the finite state automaton.Then the motion of the captured gesture sequence is decomposed into three planes,the DTW(Dynamic Time Wrapping)algorithm is used to calculate the distance between the gesture sequence and the gesture template for each plane.Finally,the weights are calculated according to the intensity of the motion on different planes,so that the three distances are combined into one comprehensive distance.In the process of matching,in order to reduce the situation of recognizing the unknown gesture as the known gesture,a threshold value is set for each gesture template and the threshold is trained to obtain a suitable threshold value.Experimental results show that the proposed algorithm can distinguish the beginning and the end of different gestures and has high recognition accuracy.