Research On 3D Gesture Recognition Method Based On RGB-D Image

Human-computer interaction technology has attracted more and more attention from people.With the development of science and technology,the way of human-computer interaction continues to be improved.New control and input technologies are becoming more and more common,especially gesture recognition technology.There are many ways to obtain gesture data.In visual based methods,people and computers communicate directly.It is simple,natural,and easy to use.However,vision-based gesture recognition is susceptible to light and background changes.Depth camera also contains the color information and depth information of an object.Depth information is mainly used to separate the foreground from the background without being affected by illumination.Color information is used to extract the two-dimensional features.Based on this research,this thesis proposes an effective solution for static gestures and dynamic gestures respectively,and proves the effectiveness of the proposed scheme through experiments.First of all,in the context of hand gesture segmentation in complex environments,this thesis uses a segmentation method combining RGB images and depth images.According to the clustering characteristics of skin color in the YCr Cb space,the distribution of skin color information in the image is counted,and the segmentation threshold is set on the Cr channel to remove the areas other than skin color.Then,the maximum class variance method and threshold segmentation are used to separate the hand from the background.Finally,the accurate segmentation of hands is obtained by combining the segmentation results of RGB images and depth images.For feature extraction of static gestures,the depth comparison feature is used in this thesis.Pixel features are described by calculating the depth differ ence of the context pixels in the neighborhood of each pixel.Then the random forest classifier is trained to classify the pixels.For the pixels belonging to the same classes,the center of the pixel is determined by the mean-shift clustering algorithm,which is used as a joint of a hand component to generate a joint model of the hand.In this paper,an adaptive mean-shift algorithm of the searching window is proposed.The size of the window,which is related to the size of the hand component,is set up to adjust the size of the window in the iteration,and the interference of the unrelated pixels in the iterative process is eliminated,and the calculation is simplified.For the angle deviation between the input gesture and the three-dimensional standard gesture template,the SVD decomposition is used to solve the rotation and translation matrix of the input gesture joint model,and the error of the input gesture and the template is corrected in the three-dimensional space.For dynamic gesture recognition,because the two-dimensional projection of the three-dimensional trajectory may have errors in viewing angle,the feature is extracted directly from the three-dimensional trajectory.The relative trajectories of different joints are established,and different features are used respectively.The palm trajectory is used as the root trajectory.In this thesis,a rotation invariant centroid distance function(CDF)is proposed to represent the root trajectory characteristics.The difference between the remaining trajectories and the root trajectory is called a child-trajectory,and is represented by a spherical coordinate including direction and distance information.The combination of the CDF feature and the spherical coordinate feature forms a fusion feature.The hierarchical limit learning machine is a kind of neural network algorithm with fast learning speed and strong generalization ability.Using the fusion feature training classifier,the dynamic gesture recognition can improve the recognition rate while satisfying real-time requirement.Experiments show that the recognition accuracy and time of this method are better than those of SVM and ELM.