Research And Application On Virtual Sand Painting With Gesture Recognition Based On Leap Motion

With the development of information technology,human-computer interaction is constantly innovating.People's demands of interactions have experienced a gradual process from functional design needs to humanization and naturalization.Gesture interaction is an important research direction in this human-computer interaction transformation.The art of sand painting is a new form of artistic expression.The virtual reality technology gives the traditional sand painting a new carrier.In this topic,gesture recognition based on Leap Motion is the main research content and the virtual sand painting is as the background.Several common algorithms of gesture recognition algorithm and Leap Motion hand tracking technology have been systematically studied.An improved gesture recognition algorithm is proposed in the paper.A system of virtual sand painting gesture recognition is designed and implemented based on Leap Motion and Unity 3D development platform.The main works of this paper are as follows:(1)By analyzing several common gesture recognition algorithms,the dynamic time warping algorithm(DTW)was selected as the basic method.An improved dynamic time warping algorithm was proposed by limiting the search range of the distance matrix and changing the search path of the distance matrix.Its feasibility was verified through experiments.(2)The improved dynamic time warping algorithm was applied to gesture recognition based on Leap Motion.According to Leap Motion's data collection characteristics,the method using multiple features to describe hand gesture trance was proposed to complete the collection of the position feature and vector feature of the hand and improve the accuracy of gesture recognition.The effectiveness of the improved method was verified through experiments.(3)The gesture recognition method based on Leap Motion was applied to the virtual sand painting gesture recognition system.Eight kinds of virtual sand painting gestures and three kinds of flow control gestures were designed in the system,and the accuracy and real-time performance of a gesture was tested through experiments.Experiments show that it can meet the requirements of real-time and accuracy in virtual sand painting.