Research On Micro-Gesture Recognition Method Based On Acoustic Side Channel

With the popularization of mobile devices,the problem of small operation interface and hard operation should to be solved.As a universal method of human interaction,gestures can express people's intentions directly and clearly in life.It has become a hot spot in the human-computer interaction field.The gesture recognition technology based on the acoustic side channel does' t require additional equipment and characteristic of fitness have received much attention.The existing acoustic-based gesture recognition technology needs to detect Doppler frequency shift to recognize different gestures,but they have requirements for the amplitude and speed of gestures.In real life,micro-gestures are more in line with human habits.However,due to its small amplitude and weak reflection energy,the Doppler shift caused by micro-gesture can not detected by the existing methods.Therefore,how to detect such a weak frequency shift and recognize the gesture on the mobile device is a challenging subject.In this paper,we proposed a new method to solve the problem that the energy caused by micro-gesture is too weak to detect.A new physical mapping model is built on the mobile device to track the gesture trajectory.The main research contents and contributions of this paper include:First,a device-free tracking system of finger gesture is proposed to accurately tracks and recognizes finger movements.We converted mobile devices into active sonar systems by establishing two acoustic side channel links between the built-in speakers and the microphone.The acoustic side channel can not perceived by humans,and will not affect the normal function of the mobile device,and since the wavelength is short,the amplification frequency shift can be more significant.Therefore,the acoustic side channel is used to sense the gesture motion in this paper.Secondly,due to the weak reflection energy and the slow moving speed of the finger,the existing method can not detect the Doppler shift accurately.To this end,we propose a micro-gesture Doppler shift detection method,which consists of two steps.The first step is to detect the frequency shift event based on the symmetric threshold method,and thesecond step is to detect frequency shift value based on the Fourier fitting method.Finally,we proposed a physical mapping model on the mobile device to track the gesture trajectory.The particle filtering method is simplified to simulate the initial position of the gesture.After obtaining the gesture track,the Canny algorithm and the Hu-moment feature algorithm are used to extract the features of the gesture track and match the final gesture result.A large number of experiments prove the high precision and robustness of the micro-gesture recognition system in dynamic environment.