Multi-Variants Geo-sensory Temporal-spatial Series Prediction Based On MT-CAN

With the rapid development of urbanization,sensor technology has been applied in many fields such as water conservancy,transportation,and meteorology projects.Thus a large amount of sensor sequence data has been accumulated on the basis of the city's intricate sensor nets,which need to be processed and modeled to provide guidance for various production and life activities for mankind.Based on the geographic nature of sensor network,the sensor series has doublelayer characteristics of time and spatial.Accordingly,how to deal with the dependence of time and space and dynamic changes in a complex system has become a huge challenge.Existing studies on sensor spatial-temporal series mainly focus on traditional mathematical models and neural network models.In addition,ensemble approaches like multi-task,multi-view and multi-module have also been applied to achieve deep learning effect.However,these models still have restriction on sequence cycle and stability.Long-term dependence is another issue which constraints model performances.In this thesis,based on a multi-module CNN-Attention encoding and decoding architecture,three attention mechanisms are designed according to the spatial-temporal characteristics of the sensor sequences:(1)Global spatial attention mechanism and local spatial attention mechanism,which are used to learn the dependence between different sensor sequences and the correlation between various features in multivariable sequences.(2)Short and long term attention mechanism.By constructing the dot product similarity between long-term coarse-grained data and short-term fine-grained data,long-term information is introduced into the model and the problem of long-term dependence is overcome.In addition,this thesis integrates meteorological,economic and other external factors in the long-term coarse-grained data through embedding methods,which improves the performance of the model to some extent.(3)The temporal attention mechanism in encoding and decoding records time correlation in the weight vector,and realizes dynamic update at each prediction time step.Moreover,in order to further utilize the potential dependencies among multivariate sequences,this thesis introduces a multi-task structure,softly sharing the underlying historical data encoding layer,this architecture allow realizing the prediction(decoding)task of multiple target variables at the same time.Such a parallel structure avoids the huge computational cost brought by several single-variable time series forecasting models.In addition,in order to get rid of the priori assumption that there is a significant correlation between different variable sequences,we use Bayesian likelihood to optimize the multi-task iteration,which effectively solves the problems of "learning bias" and "negative transferring".In the experiment part,it is verified that the multitask structure has a good generalization effect for the situation of data mutation points.The design of multiple attention mechanisms and the Bayesian optimization make our model have a high degree of self-adaptability in the learning process,and can automatically capture different correlation relationship among sensor sequences at different times.What's more,this automatic learning capacity helps our model apply to sensor sequences in other fields easier.In order to verify the effectiveness of the model in this thesis,we compare it with several baseline models such as ARIMA and LSTM in experiments.The results show that the model in this thesis has improved both in mean square error and absolute error of evaluation.At the same time,we also verify the advantages of introducing long-term coarse-grained time modules,CNN+Attention,and multi-task structure.Finally,we demonstrate the interpretability of the model by visualizing each attention module.