Research On Daily Human Activity Recognition Methods Based On Spatial-temporal Motion Information

Human Activity Recognition(HAR)is a typical application of pervasive computing.It provides important basic information for many upper-level applications in medical,military,entertainment and other scopes.With the development of Micro-electromechanical Systems(MEMS)technology,intelligent sensor module has received a significant boost in sensing ability and identification precision,predominantly accelerometer sensing module in the field of HAR gets more widely used.Therefore,a lot of latest research works utilize sensors to extract human body movement information of time and space,such as acceleration and angular velocity behavior data to realize the recognition behavior.Similar to other pattern recognition systems,HAR system based on spatio-temporal motion information also follows the general identification process,including the stage of device deployment and data preparation,the stage of model training and the stage of activity recognition.There are problems existing in these kay stages,which deeply affect the performace of the HAR performance,sucha as :(1)In the stage of device deployment and data preparation,the sensing equipment may be offset due to the user’s misoperation or violent movements,etc.,resulting in changes in the position and orientation of the device,which will lead to the distortion of the collected data and affect the final recognition accuracy.The existing works focuse on eliminating the influence of equipment migration through spatial coordinate transformation,but this method requires more priori information and does not conform to the random migration in the real scene.(2)When a deep learning model is used as a identifier,it often needs a large amount of data in the training stage to ensure the recognition accuracy.However,data collection requires more manpower and time.The size of the public data set is usually small,which may cause the model unefficiently trained.Therefore,how to automatically generate training data to adequately train deep learning models is a major challenge in the field of behavior recognition.The commonly used data expansion methods are either costly and difficult to implement,or similar to the original data but of little value to model training.Therefore,it is essential to generate data with different details in an automatically way.(3)In the stage of behavior recognition,there are often transitional activities existing between basic activities.If such transitional behaviors are not properly handled,the recognition accuracy of basic human behaviors will be affected.Therefore,the identification of transitional activity is a problem that must be considered in the stage of behavior identification.Related works on recognizing transition activity are relatively complex.It is more convinient to judge the state of behavior from the trend of behavior data.(4)The spatiotemporal-based HAR system is mostly applied under the ubiquitous environment,and the mobile and portable devices are often used,which have relatively limited battery capacity.Therefore,how to minimize equipment energy consumption and extend the service cycle of the whole system while ensuring certain recognition accuracy is an important challenge for low-power ubiquitous computing scenarios.The energy saving strategies of behavior recognition are diversified and involve multiple levels of the system.Most relevant studies focus on the role of one energy saving strategy in the system,and lack of research on the comprehensive effect of multiple energy saving strategies.All these problems above will affect the recognition performance of HAR system.Therefore,it is necessary to tackle these problems to improve the recognition accuracy of HAR system for various of activities.This paper takes the general identification process of HAR system as the main line to study the method on recognizing daily human activity based on the spatio-temporal motion information.It focuses on the key stage of recognition process to improve the recognition accuracy.The main contents of this paper are as follows:1.To solve the data distortion problem caused by changes on device placement and orientation,this paper proposes a hybrid structure,which combines the Convolutional Neural Network(CNN)and Support Vector Machine(SVM).The core idea of this method is to alleviate the negative effect on recognition accuracy caused by distorted data through utilizing CNN.The whole model is robust to data distortion by using CNN to extract features and train SVM.In the experiment part,we first proved the negative effect of device offset by simulation,and also tested the performance of SVM itself.Finally,we use the proposed model to conduct recognition experiments on data sets with distortion effects.Results show that the proposed model can produce over 15% accuracy improvement on the distorted public data set,and about 10%-20% accuracy improvement on the distorted self-collected data set.2.To tackle the problem of automated data generation in training deep learning models,this paper designs a Human Activity Recognition data Augmentation Generative Adversarial Network(HARAug-GAN)for spatial information data generation based on a classical structure named Deep convolution generative adversarial network(DCGAN).DCGAN can generate pseudo-true data with a certain degree of similarity to the original data by zero-sum game method.The proposed model takes advantage of this characteristic of GAN to generate a large amount of pseudo-true data.By using the generated data to conduct model training and recognition experiments,the overall recognition accuracy can reach 87.88% and recognition accuracy on some types of activities has benn significantly promoted when the generated data and real data are mixed in a certain proportion.Meanwhile,arrcording to the experiment results,the generated data can improve the richness of the dataset,and somehow supplement and replace the real data,which can greatly reduce the cost of data collection.In addition,according to the experimental results,the generated data can improve the richness of the data set and supplement and replace the real data.The data automaticaly generated by HARAug-GAN can not only guarantee the recognition accuracy of the model,but also greatly reduce the cost of data acquisition.3.To deal with the transition activity,the paper proposes a standard deviation analysis(STD-TA)algorithm,which combines the SVM as core classifier.The main ider is to identify the basic activities using SVM and determine whether it is a transition activity by combining the output probability results with the trend of the standard deviation of the accelerometer data in the current window.In the experimental part,we first determined the key parameters of the model,and used the real data containing transition activities to test the model.Results show that the proposed algorithm can effectively identify the transitional behavior in the data.Compared with the situation without the transitional behavior recognition mechanism,the recognition accuracy of the whole model for the basic behavior is improved by 20%through the recognition of the transitional behavior by the proposed algorithm,reaching 82.85%.4.To optimize the recognition system for energy saving,the paper proposes a energy-efficient recognition model based on multiple energy saving strategies,which is named the energy-Efficient CNN(EF-CNN).There are mainly two strategies for energy-saving in the system.One is the pre-classification strategy,whose core idea is to pre-classify activities that are easily identified based on central features.The method can reduce the recognition latency and the call frequency of the main classification algorithm,so as to save energy.The other is adaptive sampling frequency control strategy,whose core idea is to control the sampling frequency of the sensor adaptively through the analysis of past activity history.The method can reduce the sensor’s sampling frequency at proper time to reduce the energy consumption to extend ite service cycle.Experiment results suggest that the whole model can significantly reduce the energy consumption of the sensing device.Meanwhile,the real-time performance of the system is thus improved.According to the evaluation criteria proposed in this paper,the proposed model is superior to other control groups.To summrize,this issue starts with the key stages of HAR identification process and studies the four typical problems existing in it.From the experiment results,the proposed models can effectively improve the recognition accuracy of the system and solve the proposed problems.