A Joint Deep Learning Approach For WiFi-based Gesture Recognition Ultrasound And Location Classification

With the wide deployment of wireless communication systems and smart devices,gesture recognition and indoor positioning classification technology based on Wi Fi devices are increasingly employed.The principle is to extract gesture and position features from the Channel State Information(CSI)of Wi Fi signals to recognize human activities and positions.Unlike image data,CSI data lacks spatial structure information and cannot capture spatial features in the data.In addition,time series data requires special data cleaning and preprocessing techniques to reduce the influence of noise and outlier values.Compared with the variation in gesture features at fixed positions,changes in location often affect the amplitude variation of gesture features,which affects feature extraction and recognition.To effectively address these issues,this dissertation adopts a deep learning method to conduct research on gesture recognition and position classification based on Wi Fi.The main research works include:This dissertation addresses the lack of spatial feature information in CSI data and proposes a Gramian Angular Fields(GAF)method in Chapter 3 that transforms CSI time series data containing human location and gesture change features into a two-dimensional image to increase the number of features.Then,a Multilayer Perceptron SE Res Net(MLPs-Res Net)is proposed,which replaces the first 3x3 convolution layer in the residual block with two 1x1MLP(Multi-Layer Perceptron)convolution layers to reduce the number of model parameters and speed up the training process.Finally,a Squeeze-and-Excitation(SE)module is added to the 7x7 convolutional layer of the MLP-Res Net network to capture the importance of each channel,highlight important features,and suppress non-important environmental noise.The experiments show that the classification performance of the converted images is better than 1D Res Net and DTW+KNN,and the training time of MLP-Res Net is 120 s shorter per epoch than Res Net,demonstrating that MLP can speed up Res Net training.Compared to time series data,images have more features but requires longer training duration.To address this issue,in Chapter 4 and 5,this dissertation uses one-dimensional networks to extract the shared features of gesture and position.During the collection of CSI data,the signals are subject to environmental interference and multipath effects that create noise.To address this problem,this dissertation proposes a Residual Shrinkage Multi-tasking Network(RSM-Net)in Chapter 4 to dynamically identify and eliminate transformed environmental noise by adaptively setting a reasonable threshold using a shrinkage module.Then,the CSI data with eliminated noise is used to extract the shared features of gesture and position and output the classification results of both tasks simultaneously.In addition to the shared features,a Multichannel Fully Convolutional SE LSTM(MFCs-LSTM)is proposed in Chapter 5 to extract and focus on gesture features that are unrelated to position,reducing the influence of positionindependent features.Finally,the gesture features extracted by MFCs-LSTM are fused with the shared features of RSM-Net to improve gesture recognition accuracy and address the problem of insufficient model generalization ability.This dissertation uses CSI images converted by the GAF algorithm as the dataset and applies the proposed MLPs-Res Net to gesture recognition and indoor positioning classification tasks,achieving classification accuracies of 94.94% and 98.4%,respectively,which are better than several classical networks.Later,experiments show that the AFERSM-Net model has a gesture recognition accuracy of 97.84% and a positioning classification accuracy of 98.92%,which is better than the 1D Res Net,DRSN,and LSTM-FCNS models,indicating that the proposed method in this dissertation is effective.