Research On Multi-scale Human Motion Prediction Algorithm Based On Graph Convolutional Networks

With the generation of large amount of human-oriented image video data in modern society,human motion prediction as an important component of human behavior analysis becomes an increasingly important research topic.As a typical spatio-temporal prediction task,the modeling of spatio-temporal relationships is its key technology,and the core challenge lies in the modeling of human motion synergy relationships.This is where the advantage of graph convolutional networks lies.By constructing human body joints as graph nodes,the synergetic relationships of intra-or inter-human body joints can be further explored while preserving the human skeleton structure.However,the existing graph convolutional network-based human motion prediction algorithms are not sufficient to cope with the complex and changing motion scenarios.Therefore,a multi-scale human motion prediction algorithm based on graphical convolutional networks is urgently needed.This topic is a research on multi-scale human motion prediction algorithm based on graph convolutional network,including two research elements,single-person motion prediction and two-person interactive motion prediction,based on graph convolutional network,to capture intra-or inter-human motion synergistic relationship with multi-scale modeling strategy.The topic is selected from the project of Advanced Information Network Beijing Laboratory:Dynamic Video Retrieval.The main research contents and innovative results of the topic are as follows.1.To address the problem of difficulty in modeling intra-human motion co-relation in single-person motion prediction,this study proposes a multi-scale single-person motion prediction algorithm based on graph convolutional networks.First,the temporal data is coded and decoded using the time-frequency data conversion method.Then,a multi-scale human structure map is defined,and the human body is disassembled into three scales:joint point,small limb and large limb,and they are combined with the graph convolutional network to portray the motion synergetic relationship of small,medium and large scales respectively.Then,in order to obtain more comprehensive motion features,cross-scale interaction based on attention mechanism is performed for features between adjacent scales.Finally,following the human prediction mechanism,the features of small,medium and large scales are decoded by a progressive decoding network in a sequential order to obtain the prediction results.On two publicly available datasets,Human3.6M and CMU-Mocap,the average nodal position error of the proposed algorithm in this study decreases by 18.62%and 16.49%,respectively,relative to the benchmark algorithm.2.To address the problem of difficulty in modeling inter-human motion synergy in two-person interactive motion prediction,this study proposes a multi-scale two-person interactive motion prediction algorithm based on graph convolutional networks.Based on the single-person motion prediction algorithm proposed in Study 1,temporal multiscale feature enhancement is introduced to enrich motion features;spatial multiscale attention coding is introduced to enable motion features to interact in different human bodies as well as in different scales of the same human body to realize inter-human motion synergetic relationship modeling under a multiscale framework.The proposed algorithm in this study decreases the average joint point position error by 5.28 and 1.17%on the SBU and NTU-RGB+D 120 public datasets,respectively,compared with the benchmark algorithm without considering interaction(the algorithm proposed in Study I).