Research On The Technology Of No-touch Gesture Interaction Based On Vision

With the development of computer technology, human computer interaction is developing toward the direction of humanization and simplicity, and the non-touch gesture interaction has become an important way of human-computer interaction.Compared with the mouse, keyboard, touch screen and other type of touch operation,non-touch gesture interaction is more in line with human nature. In recent years, with the development of computer vision and image processing technology, the way of gesture-computer interaction based on machine vision is becoming the mainstream of new human-computer interaction. However, existing gesture interaction is not ideal.In this paper, we focus on how to overcome the interference of environmental factors to obtain accurate gesture data and effective identification, how to achieve direct dialogue between human and computer for gesture interaction system.The main research work in this paper includes the following aspects:Firstly, human arm bone joint information was obtained through the Kinect sensor. Effective arm joint trajectories in 3D space were extracted and 3D motion trajectory was converted to one-dimensional gesture trajectory. Combined with the gesture trajectory characteristic, the new gesture segmentation method combined speed and angle of gestures was proposed. And then, improved dynamic time warping(DTW) algorithm was used to achieve a fast and efficient recognition for dynamic gesture in the complex environment and different light illumination conditions.Secondly, the spatial coordinates of the joint of hand was transformed into the position of a cursor on a computer screen. As for positioning the cursor, the moving direction and the moving distance of the cursor was determined by the move of the hand. The gesture recognition result was converted to the computer order to achieve the mapping of the gesture and the mouse.Finally, a gesture operation Gobang game software was developed based on MFC, which could be manipulated by designed gestures and achieve a non-touch gesture interaction application.Non-touch gesture interaction technique is an important part of the research field of intelligent human computer interaction, which is also a key component of intelligent human-computer interaction. Combined intelligent technology and the traditional interactive mode, developing a new interactive technology is in line withthe current trend of science and technology development. This project is supported by the National Natural Science Foundation of China "No.61573029" and "No.4162012" of Beijing Natural Science Foundation, which has important practical significance to promote the wide application of gesture interaction technology.