Research On Multimodal Perception Of Human Behavior Recognition Based On Wearable Sensor

With the technological progress of smartphones and sensor technology,with the quality and low-cost characteristics to promote the widespread popularity of wearable devices,bringing great convenience to all aspects of human life,in health care,human-computer interaction,sports assistance and other aspects of a strong advantage,making the wearable sensor-oriented human behaviour recognition research is of great significance,by the researcher’s extensive attention,become a hot spot for research.The research on human behaviour recognition is becoming more and more widespread.Video-based non-contact behaviour recognition has been developed earlier,but it is affected by ambient light and occlusions and easily offends the privacy of user data,while contact behaviour recognition is increasingly superior using wearable sensors with strong computing power,low cost and good portability.In early work,traditional machine learning methods were used to classify data,simply designing or mining data features without considering the relationship between channels and the characteristics of the time series,and the accuracy of the behavioural recognition obtained was not high.Along with the development of deep learning technology,behaviour recognition has made great progress on the basis of deep learning,efficient models have been proposed and performance has been improved,but existing research work mainly improves recognition accuracy by designing different deep learning models for feature extraction,while ignoring the problem of uneven information distribution and data redundancy in the multimodal sensory data collected by wearable sensors.To address the above problems,this paper proposes a keyframe-based dynamic sampling network and a spatially adaptive sampling network from the temporal and spatial perspectives,respectively.Firstly,the BLSTM network is used to build a pre-processing module for behaviour recognition,which is used to initialise the original sensory data and obtain the initial behaviour prediction results;secondly,based on the initialisation information of each data,the BLSTM is used to build a selection network,through which the key frame dynamic sampling network predicts the probability of data frames being selected to eliminate temporal redundancy,and through which the spatial adaptive sampling network predicts the probability of data channels being selected to eliminate spatial redundancy.Finally,the behavioural prediction is performed again using the selected key data,and the initial behavioural prediction results and the selected key data prediction results are used to form a utility function for the training of the selected network.Experiments are conducted on three publicly available datasets UCI HAR,Opportunity and UCI MHEALTH,and the experimental results show that the proposed model can improve the accuracy and F1 values of behavioural recognition.