Research On Deep Learning Based Human Activity Recognition

Benefit from technological developments in integration technology and sensor technology,smart portable devices are equipped with powerful built-in sensors and computing power.Smart phones are favored by consumers due to their comprehensive applications and diverse price options.Therefore,human activity recognition(HAR)systems for smart phones have broad application prospects.Compared to the existing HAR system based on traditional machine learning which requires heavy manual feature engineering,deep learning is promising in the field of HAR by virtue of its ability to extract significant features from data automatically.In view of the above situation,this thesis deeply investigated existing researches on deep learningbased HAR,and the main research contributions are given as follows:(1)This thesis found two challenges in the application of HAR system based on the deep recurrent neural network(DRNN)through comprehensive investigation and experiment.Firstly,the DRNNbased HAR system performs activity recognition with the records from the Uni Mi B SHAR dataset collected by the built-in accelerometer from smart phone.This system has to pay much computation overhead(e.g.,training time and recognition time)in order to perform accurate recognition;Secondly,when all subjects are divided into two groups(i.e.,the source domain and target domain),the DRNNbased HAR system only trained by records from source-domain subjects is hard to achieve high recognition accuracy when this trained system is directly applied to recognize activities performed by target-domain subjects without learning from their records before.(2)We proposed the HAR scheme based on power spectral density(PSD)and DRNN,aiming at performing accurate activity recognition with low computation overhead.Specifically,PSD features are firstly extracted from linear accelerations explicitly,and then DRNN further extracts virtual features from PSD features implicitly and finally completes recognition.Explicit PSD-based feature extraction saves much computation because the number of parameters in DRNN that need to be calculated are reduced via using fewer time steps in the input of DRNN.Simultaneously,explicit PSD-based feature extraction ensures recognition accuracy via increasing frequency-domain features at each time step.Experiments on the Uni Mi B SHAR dataset show that proposed systems can achieve high recognition accuracies,which are close to that of the DRNN-based system,and our proposals save much computation overhead.(3)We proposed the HAR scheme based on transfer learning and DRNN to realize accurate activity recognition by exploiting virtual features in the adaption stage with a few records from new users.Specifically,the feature extraction component based on DRNN is trained by records from sourcedomain subjects in the pre-training stage,and then transferred to establishing the HAR scheme in the adaptation stage.With the pre-trained feature extraction component,the source-domain virtual features corresponding to classes of activities(i.e.,activity patterns)are obtained and transferred to calculating domain loss in the adaptation stage.In the adaptation stage,similarity between source-domain virtual features(i.e.,activity patterns)and target-domain virtual features(i.e.,the output of each dense layer when the input of system is target-domain data)is measured layer by layer in the adaptation component and then accumulated as the domain loss.Finally,domain loss and classification loss are combined as the loss function used in the training process of the adaptation stage.Experiments on the Motion Sense dataset shows that the proposed systems can use a small number of records from new users to perform activity recognition accurately by adapting the pre-trained DRNN component via transfer learning.