Skeleton Action Recognition Based On Deep Learning

Human action recognition is a subject that identifies and classifies action types from human action data acquired from sensors,cameras,and other devices through computer vision and machine learning techniques.As one of the branches,skeleton action recognition aims at understanding and classifying human actions from skeleton joint point data,it can be used in many applications such as intelligent monitoring,human-computer interaction and motion analysis.The skeleton data,as the 3D pose information of human body,can well reflect the spatial position and direction information of human action,and the skeleton data has compact information with less redundancy,which requires less computational resources and is robust in the face of disturbances such as illumination,human scale change,motion speed change and viewpoint change.With these advantages,skeleton-based action recognition has gained more attention and development prospects.In this paper,two methods,long short-term memory network and deep residual network ResNet50,are used to classify skeleton action recognition through different pre-processing and feature extraction methods for skeleton data.A skeleton action recognition model based on human geometric features combined with LSTM networks is constructed.Firstly,for the problem of different lengths of action sequences,the skeleton time series data were segmented using time series segmentation technique to extract each human action sequence,and the segmented action sequences were normalized,then the skeleton sequence data were feature extracted,and two static features,joint relative angle and joint relative distance,and one dynamic feature,angular acceleration of joint relative angle,were extracted respectively.Then the features were fused and fed into the LSTM network to recognize and classify the skeleton actions,and several main parameters of the neural network were optimized during the experiment,and the Dropout mechanism was introduced to cope with the overfitting problem,and finally a high recognition accuracy was achieved.A skeleton action recognition model based on the optimized ResNet50 network was constructed.The network structure was first optimized for the problem of feature information loss caused by the ResNet50 network,followed by the use of label smoothing technique to regularize the cross-entropy loss function to cope with the overfitting problem during the training process and improve the model generalization ability.The differences of various learning rate decay algorithms in terms of convergence speed are compared during the experiments,and the optimal solution is selected to accelerate the network convergence speed.For the processing of the dataset,a tree-structure-based joint point sequence rearrangement technique is used to make adjacent joints adjacent in the graph,enhance the kinematic dependencies between joint points,and perform data enhancement on the skeleton data by rotation matrix.Finally,the skeleton data are transformed into image data and fed into ResNet50 network for training,and validated on the NTU-RGB+D dataset,and the accuracy rates of 88.39% and 92.79% were respectively achieved on the two protocols of "Cross-Subject" and "Cross-View".