Group Activity Recognition Based On Graph Neural Networks

In recent years,action recognition has been extensively studied in the visual domain,such as intelligent security systems,human-computer interaction systems,and motion video action analysis.Action recognition mainly includes single-person action recognition and group action recognition.This paper presents two types of action recognition using graph neural networks.It is objective to develop a model for action recognition that is efficient,accurate,and highly generalizable.In the field of individual action recognition,since human skeleton data is robust in behavior analysis,this paper constructs an action model based on human skeleton data,which overcomes the lack of inherent correlation between the adjacency matrix and the classifier and insufficient spatio-temporal feature extraction in the classifier.Due to insufficient inherent information correlation in the adjacency matrix,this paper proposes a decoupled adaptive graph convolutional network(DAGCN).Using a non-local adaptive algorithm,an adjacency matrix dynamically acquires rich correlation information between skeleton nodes.At the same time,a decoupling aggregation mechanism is used to obtain multi-channel adaptive adjacency matrices,which achieves decoupling of the spatial aggregation of the graph convolutional network.The decoupled adaptive graph convolutional network can obtain richer node connections during the behavioral changes,which is crucial for extracting non-Euclidean data features from the graph convolutional network.Regarding the problem of insufficient spatiotemporal feature extraction in the classifier,inspired by the decoupled head,this article decomposes the spatiotemporal feature input to the last global average pooling layer into spatial,temporal,and spatiotemporal dimensions,and then performs classification,thereby obtaining classification results with different emphases,which improves the action classification ability.In addition,this article proposes a new residual multi-scale temporal convolution structure,which effectively extracts temporal information.Experimental results show that the proposed method achieved recognition accuracies of 96.4% and 85.6% in the NTU-RGB+D and NTU-RGB+D120 datasets respectively.In group activity recognition,the primary focus lies in improving the graph neural network’s ability to infer group member interaction relationships.Prior methods for modeling interaction relationships have been constrained by limited local receptive fields,making it challenging to capture long-range dependencies.Models based on Transformers or Graph Attention Networks(GATs),which offer larger receptive fields and long-range dependencies,require extensive computations of key-value elements.Consequently,these models impose high demands on time and memory resources,with attention weights dispersed in the early stages,resulting in weaker focusing effects.Inspired by deformable DETR and DAT,this paper introduces a model based on deformable attention graph convolution.By employing a dual-stream fusion approach,this model extracts features from video data and utilizes attention offsets to learn the feature similarity between group members,enabling the measurement of their interactions.The proposed model assigns similarity-based scores to selectively extract important member information for activity recognition.It also utilizes spatial structural information to simulate and infer dependency relationships among group members,facilitating the attention module’s identification of key members.The algorithm framework achieves average recognition accuracies of 93.3% and 94.5% on the public volleyball dataset(CAD)and collective activity dataset(VD),respectively,surpassing mainstream methods like ARG and Hi GCIN,thereby validating the effectiveness of the proposed approach.