| With the in-depth development of Internet technology,humancomputer interaction technology has become the focus of all walks of life.The movements of the human body contain a wealth of information,which can be regarded as a very vivid,intuitive and personalized human language.Human Activity Recognition(HAR)technology using this human language,as an important field of human-computer interaction technology,has very broad practical application potential and development prospects.HAR systems based on wireless signals have attracted much attention due to their advantages of being easy to deploy,not affected by lighting conditions,no need to carry equipment,and no human privacy concerns.With the further research on WiFi protocol,the research of identification system technology based on Channel State Information(CSI)has become the mainstream research field of current WiFi signal identification system technology.This research mainly studies the human activity recognition system technology based on WiFi signal from the following two aspects:1.In order to extract the key features in a more distinguished way,in this research,we propose a WiFi-based device-free HAR system leveraging wavelet integrated convolutional neural network.Instead of utilizing pooling operations,our proposed network has introduced discrete wavelet transform into the convolutional architectures,which can combine the good time-frequency local characteristics of the wavelet transform with the self-learning ability of the neural network.Consequently,not only high-level features from low-frequency components can be obtained automatically,but also the the size of feature map can be reduced.The experiment results demonstrate that our method achieves average 94.87%accuracy for distinguishing ten actions in real-world home environment.2.With the purpose of fusing the information of multiple modalities and obtain more comprehensive features,this research introduces the selfattention mechanism to design a multi-modal fusion network based on the fusion of the CSI data stream of the WiFi signal and the image flow of the video signal.The experimental results demonstrate that the multi-modal fusion activity recognition network based on the self-attention mechanism can capture the local detail features in multi-modal signals,effectively fuse the features extracted by a single modality,and improve the activity recognition rate in real complex environments. |
