Skeleton Edge Direction And Projection Guided Human Action Recognition

In recent years,human action recognition based on 3D skeleton data has received wide attention in the fields of human-computer interaction and intelligent monitoring.For example,in video surveillance,recognition of abnormal human behavior is used to prevent potential threats;in human-computer interaction,computers are trained on human behavior data to obtain recognition models for understanding human behavior and achieving a more natural interaction between humans and machines.In skeleton-based human action recognition,the use of convolutional neural networks for spatio-temporal modeling of skeleton data has been shown to be effective.To address the problem that most existing methods focus on modeling skeleton "point-level" information and seldom consider combining the directionality of human motion to describe the movement changes,this paper proposes a skeleton edge direction-guided human action recognition method,which defines skeleton "edge-level" information with direction for human action feature construction."In this paper,we propose a method of human action recognition based on skeleton edge information in projection subspace,which uses 2D projection subspace on skeleton edges to characterize actions in different subspace perspectives.In order to make full use of skeleton edge-level information and combine the directionality of edges to more naturally characterize human actions.In this paper,we propose a new direction-guided dual-stream convolutional neural network.In the first stream,our model focuses on edge-level information with directionality(both edge and edge motion information)defined in the skeleton data to explore the spatio-temporal characteristics of the action.In the second stream,since the motion is directional,we define bidirectional skeleton edges and extract different motion information(including translation and rotation information)in different directions to better exploit the motion features of the action.In addition,we propose a method to describe human motion by combining translation and rotation,and explore the way they are fused.In order to explore the integrated representation of the action in different viewpoints and portray the local correlation of human motion.In this paper,an action recognition method based on skeleton edge information in the projection subspace is proposed.First,the method defines the skeleton edge information combined with the body’s own connection for capturing the spatial characteristics of the action;then,the direction and size information of the skeleton edge motion is introduced on the basis of the skeleton edge information for capturing the temporal characteristics of the action;furthermore,the 2D projection subspace is used for action characterization under different subspace perspectives;finally,a suitable feature fusion strategy is explored and the above features are integrated and extracted by an improved Finally,a suitable feature fusion strategy is explored,and the above features are synthetically extracted by the improved CNN framework.We conducted extensive experiments on two challenging datasets,NTU-RGB+D 60 and NTU-RGB+D120,to verify the superiority of our proposed method relative to the state-of-theart methods.The experimental results show that the proposed direction-guided edge-level information and motion information complement each other for better action recognition;the proposed subspace projection under the edge with its size and direction information can be effectively fused for better integrated characterization of the action.