Research On Optimizing The GCN Backward Training Using Execution Path Preparing On GPUs

With the advent of big data era,the emerging Graph Convolutional Network(GCN)is increasingly used in different fields.However,at the same time,the GCN traing overhead grows due to the sparsity of GCN input graphs and their growing size,which affects the efficiency of GCN applications.How to accelerate the training of GCN to further improve the efficiency of the application has become one of the research hotspots in the field of graph neural networks.State-of-the-art GCN training systems employ graph processing techniques to accelerate the massage exchanging(i.e.aggregations)among the graph vertices to reduce the GCN training overhead.However,these systems treat the aggregation stages in both forward and backward propagations as all-active graph algorithms(i.e.,all vertices participate in the aggregations)for processing,which leads to a large amount of redundant computations.The GCN training method based on the execution path preparing strategy treats the aggregation stages in the backward aggregation phase of GCN training as a partially-active graph algorithm,and selects only some vertices of the input graph to participate in the computation.According to the characteristics of the backward propagation phase,the proposed method collects and coalesces the graph data that needs to participate in the backward propagation of each graph convolution layer,to achieve faster backward aggregations on Graphics Processing Units(GPUs),thus improving the overall performance of GCN training on GPUs.The experimental results show that the GCN training system based on the execution path preparing strategy improves the performance of the backward aggregation in GCN training by 1.48x～5.65 x compared to the state-of-the-art GCN training system(i.e.GNNAdvisor)on real-world graphs.In addition,the execution path preparing can be conducted either during preprocessing before GCN training or on-the-fly with the training.When used during preprocessing,the proposed method improves the overall GCN training by 1.05x～1.37 x.And when used on-the-fly,the proposed method improves the overall GCN training by 1.03x～1.35 x.