Research On Human Action Recognition Using Inertial Sensors

Joint combat simulation based training uses the modeling theories and methods to design simulation models to simulate specific military objects and operational environments to support military training.This training method will play a vital role in future military training.The training process of the trainees using the combat simulation system is basically done by human-computer interaction.The immersion brought by the good interactive experience is crucial to the improvement of the simulation training effect.The traditional way of interaction makes the trainees pay too much attention to the operation of the mouse and keyboard,which cause the interaction being not real and the training efficiency being low.The use of motion capture and activity recognition technology,regarding the user's limb movement as an important input channel,combined with virtual reality technology to create a virtual training ground can bring better training immersion,and thus enhance the training effect.Wearable sensor based human action recognition system can provide a larger activity space and a higher degree of freedom than external sensor(such as a depth camera)based recognition system,and the recognition effect will not be affected by external factors such as illumination and occlusion.It is more suitable for virtual reality technology.Aiming at solving the human action recognition problem based on wearable inertial sensors,we use deep learning methods to construct the classifier.The deep Long Short Term Memory(LSTM)network is used as the main frame and improved from three aspects such as neural network information transmission,feature extraction and final classification decision.It is demonstrated by experiments with three datasets that the final Attention-based Temporal Weighted Convolutional LSTM(ATW C-LSTM)network has achieved or even exceeded the state of the art in some degrees.The main innovations of the paper are as follows:(1)Introducing the residual learning into deep LSTM network to solve the gradient vanishing and gradient exploding problem.The main problem to be solved of human continuous action recognition based on wearable inertial sensors is to model time series.This paper selects the LSTM network which is better at processing time series as the main frame,and adjusts the structure based on the residual learning method,which mitigates the gradient vanishing and gradient exploding of the deep LSTM network in the depth degree to some extent..(2)Adding convolutional layers in the LSTM network to implement feature extraction automatically and improve the generalization ability of the network.Classical human action recognition usually uses hand-made features or heuristic features.The disadvantage of this is that it highly relys on domain knowledge and experience.The features are manually designed for specific tasks and do not have strong generalization ability.With the support of large datasets,this paper adds convolutional layers after the input layer of the network to learn features in a data-driven form,aiming at replacing artificial features.Without requiring or greatly reducing the dependence on domain knowledge and experience,the network has good generalization capabilities.(3)Introducing Attention mechanism to automatically determine the temporal context associated with the modeling actions.Previous deep learning approaches have focused on representing and modeling a fixed size temporal context for all sensor readings based on a fixed-length sliding window.However,this approach will not naturally lead to ideal modeling and hence classification performance.In order to solve this problem,we introduce the attention mechanism into human action recognition.By adding the attention Layer,the network can automatically determine the temporal context that is relevant for modeling actions.