Research On Spatio-Temporal Sequence Prediction Method Based On Transformer And Graph Convolution Network

Spatial-temporal sequence prediction is to infer the subsequent development trend of data by mining the potential temporal and spatial correlation of historical data,which can assist target monitoring and establish early warning.Spatio-temporal sequence prediction can assist target monitoring,establish early warning,and provide an important reference for future work deployment.It’s widely used in the fields of infectious disease prediction,traffic prediction,financial prediction,network intrusion detection,and so on.The prediction approaches of spatio-temporal sequences based on deep learning mostly use RNN or CNN to extract temporal features.RNN-based approaches need to be executed recursively in order,which is inefficient and prone to gradient disappearance or gradient explosion.Although the CNN-based approaches can calculate in parallel,they need to stack multiple convolution layers to expand the receptive field when processing long sequences.Transformer can process long sequences in parallel through the attention mechanism and can extract the time features of sequences efficiently.Graph convolution networks can model the dependency relationship between nodes and extract the spatial features in the sequence.The fusion of graph convolution network and transformer can not only extract the dynamic spatiotemporal features to improve the accuracy but also can increase the efficiency of the model.The thesis studies spatio-temporal sequence prediction approaches based on transformer and graph convolution.It proposes two spatio-temporal sequence prediction models and applies the models to infectious disease prediction and traffic prediction scenarios respectively.The main contributions are as follows:(1)According to the characteristics of spatio-temporal sequence prediction,the thesis proposes a spatio-temporal sequence prediction model based on Transformer(STS-TF).STSTF introduces the position information into the input features by using the time embedding method.It uses the attention network to extract the temporal and spatial features of the sequence and obtains the temporal and spatial features of the sequence through the spatio-temporal feature fusion network.STS-TF is applied to the incidence trend prediction of hand-foot-mouth disease in Shandong Province and Guangdong Province,and the prediction of METR-LA traffic flow.The comparative experimental results of the benchmark method show that STS-TF can greatly improve the accuracy of spatio-temporal sequence prediction and has better performance than the benchmark method.Results of the positional encoding ablation experiment prove that the time embedding positional coding method in STS-TF effectively introduces time location information into the model.Results of the attention mechanism ablation experiment prove that the attention network in the STS-TF model can effectively extract the temporal and spatial characteristics of the sequence to improve the prediction accuracy.(2)The thesis integrates graph convolution network with transformer framework and proposes a spatio-temporal sequence prediction model based on Transformer and graph convolution(STS-TFGCN).STS-TFGCN strengthens the dynamic spatio-temporal correlation within the sequence through the spatio-temporal attention network and then uses the adaptive graph convolution network to further extract the spatio-temporal characteristics of the sequence.STS-TFGCN is applied to the incidence trend prediction of hand-foot-mouth disease in Shandong Province and Guangdong Province,and the prediction of METR-LA traffic flow.The comparative experimental results of the benchmark method show that STS-TFGCN can effectively extract the dynamic spatio-temporal characteristics of data and improve prediction accuracy.Compared with RNN-based models and CNN-based models,STS-TFGCN can not only obtain better prediction accuracy but also has higher efficiency by parallel processing.Results of the attention mechanism ablation experiment prove that the spatio-temporal attention network in STS-TFGCN can strengthen the internal spatio-temporal correlation of sequences.Results of the graph convolution ablation experiment prove that the adaptive graph convolution network in STS-TFGCN can effectively extract the dynamic spatial characteristics of the sequence and improve the prediction accuracy.