Research On Gesture Tracking Based On Deep Learning

With the development of video processing technology,people pay more and more attention to human behavior and status in videos.As the most flexible part of human body,gestures and tracking of gesture contain rich body language information.Hand tracking is a key technology for gesture interaction,gesture translation,and gesture recognition.It has been widely studied and applied by academia and industry.However,existing gesture tracking algorithms require complex model construction and optimization solutions,and using traditional manual features can hardly meet the demand of varied scenarios.Therefore,this thesis studies how to use deep learning methods to track gestures in natural scenes,relying on the ability of deep learning to automatically learn image features,as well as combine the characteristics of gestures to solve the problem of gesture tracking in natural scenes.The main content of this thesis is as follows:1.This thesis studies a gesture detection algorithm based on weighted deep learning feature maps fusion.This algorithm uses a weight methond to fuse the lower-level,middle-level feture maps and higher-level feature maps.The fused feature map contains rich edges,color,texture and semanteme features.It can also automatically learn the importance of different levels of feature maps,while allowing the network to automatically learn the importance of different levels of feature maps,making the gesture detection more accurate.2.This thesis studies a gesture similarity comparsion algorithm.Through analysising of different hand gestures,we found that the information distribution of the same hand gesture in space has high similarity.Therefore,this thesis uses the method of feature map partitioning to hand gestures.The different areas of the block are described.After that,the features of corresponding areas of the two gestures are compared and learned,and the results of comparison of the different areas of blocks are combined to form a comparison of gesture similarities.3.This thesis studies a gesture tracking method based on association information.This method uses the previous frame tracking result to define the search area for the the next frame.In this thesis,the hand gesture segmentation network is used to segment the gesture of the search area.And locate the gesture of the search area according to the result of the segmentation.In order to train the model and verify the effectiveness of the studied algorithm in this thesis,we construct gesture detection dataset,gesture similarity comparation dataset and gesture segmentation dataset.The experimental results shows that the gesture tracking algorithm based on gesture detection and gesture tracking algorithm based on associated information have excellent tracking performance.