The Research Of Multi-resolution Fusion Convolutional Network And Open Set Activity Recognition Method

Human Activity Recognition(HAR)technology has been the focus of extensive research in the field of computer vision.With the growing development and maturity of wearable devices,the sensor-based HAR technology has received more and more attention.Meanwhile,the performance improvement of various deep learning algorithms provides a lot of theoretical basis and practical experience for the wide applications of the sensor-based HAR technology in various fields.In recent years,the sensor-based HAR technology has been successfully applied to medical rehabilitation,motion analysis,smart home and special group monitoring.Currently,the sensor-based HAR technology has been developed greatly,but there are still some deficiencies.Firstly,for the feature extraction and model building,the existing algorithms usually reduce the dimension to feature data to extract the deep features,but it is difficult to distinguish the confusing activities.Secondly,in terms of classification,the current model can only classify the short time and simple activities,but it is difficult to classify the complex activities which consist of multiple simple activities.In addition,most of the current methods are only applicable to the closed set,and there are some defects in the recognition of unknown activity classes in the open set.Finally,in terms of data set,most of the public data sets of sensor-based HAR usually only contain several daily activities,and the data amount of non-daily activities is far from enough,while non-daily activities have the important application value for activity recognition in some special scenarios.In view of the above problems,this thesis mainly studies from the following aspects:(1)Aiming at the problem of imprecise and inadequate feature extraction of sensor data in HAR model,a simple activity recognition network based on multi-resolution fusion convolution(MRFC)is proposed.Features with multiple resolutions are deeply fused through MRFC module,and the correlation between multi-sensor data is extracted through the self-attention mechanism,so as to obtain finer,multi-scale feature information with sufficient discrimination and improve the recognition accuracy of confusing activities.(2)On the basis of simple activities recognition network based on MRFC,a complex activity recognition network based on multi-window time series(MWTS)is further proposed,which can extract temporal features of sensor data by the layer of recurrent multi-window attention and assign adaptive weights to features in different windows.The proposed network can classify not only the short time simple human activities in real time,but also the long time complex human activities effectively.(3)To solve the problem that the model is difficult to classify unknown activity classes,a method for open set activity recognition based on multi-resolution fusion convolution variational auto-encoder(MRFC-VAE)is proposed.MRFC-VAE is used to reconstruct feature data and corresponding thresholds are set according to reconstruction losses,which can effectively classify both known and unknown activity classes in the open set.It greatly reduces the recognition error rate of HAR model in practical applications and further enhances the practical application performance.At the same time,aiming at the problem of insufficient categories of human activities in the existing public data set,the data for several kinds of daily and abnormal human activities data are collected through the smart bracelet.A richer data set named daily-abnormal activity data set of special groups is constructed,which can be applied to the daily monitoring of special groups and other scenarios.Through the above research,the problem of low recognition accuracy caused by imprecise extraction of sensor features is effectively solved,and the effective recognition of complex human activities and unknown activities is realized.The experiments and analyses are carried out on the wireless sensor data mining(WISDM),the physical activity monitoring(PAMAP2),the OPPORTUNITY public data sets,and the constructed daily-abnormal activity data set of special groups,respectively,to prove the effectiveness and superiority of the proposed models and methods.Both the HAR classification models and the open set classification method proposed in this thesis can be applied to a lot of HAR scenarios such as smart home,daily monitoring of special groups and so on,which has great research value and application prospect.