Research On Gesture Recognition Technology Based On Millimeter Wave Sensing

Gesture recognition based on millimeter wave mainly uses sensors to capture the behavior information of the human body when performing gestures,and uses algorithm models to analyze and recognize these behaviors,providing a natural and convenient way for humancomputer interaction.In this paper,two main scientific issues of millimeter wave gesture recognition are studied:the mechanism of real-time gesture recognition in low signal-tonoise ratio scenarios,and the mechanism of multi-user gesture recognition.A series of recognition models and methods are proposed,a large-scale millimeter wave gesture dataset is established publicly,and two interactive prototype systems are implemented.The main contributions of this paper are as follows:(1)Real-time gesture recognition in low signal-to-noise ratio scenarios.At present,millimeter wave gesture recognition is limited to short-range scenes within 1 meter,and cannot support long-distance scenes(for example:3-5 meters in smart homes).The reason is that the millimeter-wave signal attenuates rapidly with the propagation distance,and the influence of the surrounding noise also increases,making it difficult to detect,separate and accurately recognize long-distance gestures from the noise.To solve this problem,this paper proposes a long-distance gesture recognition method based on point cloud spatial portraits,constructs a spatial portrait feature that can characterize the details of long-distance gestures to deal with the problem of millimeter wave signal attenuation,and designs a neural network method that can analyze the portrait details,so as to efficiently use point cloud spatial portrait features to achieve accurate recognition of long-distance gestures.Second,the existing works of gesture recognition are based on perfectly segmented gesture samples,which are not suitable for real-time interaction scenarios.To overcome the challenge,this paper proposes a real-time gesture recognition method based on human-like understanding features,which can realize accurate and real-time recognition of multiple continuous gestures at a distance.(2)Gesture recognition for multi-user interaction.Due to the behavioral differences between the untrained user’s gesture and the standard action,the recognition method produces frequent misrecognition,which leads to the problem of poor generalization.In response to this problem,this paper proposes a gesture recognition method with high generalization based on feature change transfer.Transfer learning method is used to adjust the parameters of the classification network to adapt to the differences in the feature changes of untrained user gestures,thereby improving recognition generalization.On the other hand,in addition to accurately recognizing the gestures of different users,identifying the user’s identity through the same gesture can reduce the user’s subsequent operations,thereby improving the user’s personalized experience.To this end,this paper proposes a user gesture identification method based on the Siamese neural network,which extracts gesture details to describe and recognize the user’s subtle behavior and habits,and uses a lifelong learning method to adapt to the user’s behavior and habits that change over time.(3)Millimeter wave gesture dataset construction and prototype system implementation.This paper constructs and publicly releases a large-scale millimeter-wave gesture dataset,collecting more than 158,000 samples from 246 volunteers in 17 indoor and outdoor scenes.Based on the above samples and methods,this paper designs and implements two end-to-end prototype systems:a gesture perception system mPhoneSys for smartphone interaction and a gesture recognition system mHomeSys for smart home interaction.System experiments and user studies show that the method proposed in this paper can recognize gestures in real time and accurately in human-computer interaction and smart home scenarios,and can determine the identity of the performer through the gestures performed by the user.In summary,this paper explores the millimeter-wave sensing gesture recognition technology,and proposes a series of models,algorithms and strategies for the two scientific issues of real-time gesture recognition mechanism in low signal-to-noise ratio scenarios and multi-user gesture recognition mechanism.On the basis of large-scale data collection and measurement,the effectiveness of the proposed method is verified through theoretical analysis and evaluated on a prototype system,which is helpful for the development of millimeter wave gesture recognition technology and the natural human-computer interaction above it.