Research On Action Recognition Method Based On Skeleton Data And Forward-reverse Adaptive Graph Convolutional Networks

As a key technology for many real-world applications and needs,human action recognition has attracted close attention from the computer vision and artificial intelligence communities in recent years.In the latest research based on skeleton data,the graph convolutional networks(GCN)based methods have achieved excellent performance on action recognition tasks.Compared with methods based on RNN and CNN that cannot fully represent the graph structure of skeleton data,GCN can model the human body more naturally and achieve better results.Hence GCN are extensively studied in skeleton-based action recognition because they are suitable for data with non-Euclidean structure.How to extract as many features as possible from skeleton data is the key to GCNbased action recognition.Skeleton data can be divided into joint and bone data in the spatial dimension while existing GCN-based state-of-the-art(SOTA)methods usually adopt the strategy of fusing joint and bone data streams to achieve better results.However,these methods ignore the reversibility of skeleton data in the temporal dimension,that is,skeleton data can be divided into forward and reverse data.Compared with the forward sequences input,which can achieve better results in certain actions through the end-to-end networks,the reverse skeleton data has excellent discrimination and richer information for some specific actions.Based on the two-stream adaptive graph convolutional network 2s-AGCN,this paper proposes forward and inverse adaptive graph convolutional network FR-AGCN.Compared with 2s-AGCN,the main contributions of this paper include three aspects:(1)Aiming at the problem that the information of the skeleton has not been fully utilized,this paper proposes to introduce the reverse GCN in the end-to-end networks,so that the model can learn the features in the skeleton sequence more comprehensively.The FR-AGCN proposed in this paper can extract the features of four different inputs respectively in the four-stream framework.Concretely,FR-AGCN adopts a multi-stream fusion strategy and optimizes by combining the advantages of the four types of data,i.e.,forward joints,reverse joints,forward bones,reverse bones.The best results are achieved in the recognition of most action categories.(2)Aiming at the problem that the resampling strategy used by the previous SOTA model for data augmentation may not be optimal,this paper proposes an optimization strategy for inter-frame interpolation of skeleton data.This data preprocessing method can alleviate the overfitting problem during the training process.(3)This paper effectively verifies the superiority of FR-AGCN.Exhaustive experiments are carried out on three datasets,i.e.,NTU RGB+D,NTU RGB+D 120 and UAV-Human,and the results show that FR-AGCN greatly improves the recognition performance of 2s-AGCN.In particular,the proposed model FR-AGCN outperforms other SOTA models on UAV-Human and has significant advantages.The proposed FR-AGCN achieves competitive performance while its realization only requires meticulous processing in the data preprocessing stage.