Non-stationary Multivariate Time Series Prediction Based On Recurrent Neural Networks

Time series prediction plays an important role in various fields of science and society.However,with the prosperity of the IT industry in the new information era,different types of time series data have emerged endlessly,and the sampling frequency has become higher and higher.The property of nonstationary and multivariate in such a huge amount of data has exacerbated the challenge of prediction,and many traditional models are no longer capable of current prediction tasks.The existing deep learning models based on Recurrent Neural Networks(RNNs),especially Long Short-Term Memory(LSTM)-and Gated Recurrent Unit(GRU)-based neural networks have received impressive performance in prediction.Although the architecture of the LSTM is relatively complex,it cannot always dominate in performance.Latest research has shown that with a simpler gated unit structure,the Minimal Gated Unit(MGU)can not only simplify the network architecture,but also improve the training efficiency in computer vision and some sequence problems.Most importantly,our experiments indicate that this kind of unit can be effectively applied to the NSMTS predictions and achieve comparable results with LSTM-and MGU-based neural networks.However,none of the three gated unit based neural networks can always dominate in performance over all the NSMTS.Therefore,in this thesis we propose 2 novel prediction models based on RNNs to improve the prediction accuracy in NSMTS predictions and the contributions are as follow:1.MGU-based Neural Network is successfully applied to NSMTS predictions for the first time,and we prove that it can obtain comparable prediction performance with LSTM and GRU-based neural networks.2.We propose a novel linear MIX Gated Unit(MIXGU).This gated unit can adjust the importance weights of GRU and MGU dynamically to achieve a better hybrid structure for each MIXGU in the network during training.The experimental results show that this MIXGU neural network has higher prediction performance than other state-of-the-art one gated unit neural network models.3.We further propose a novel Selective Recurrent Neural Networks with Random Connectivity Gated Unit(SRCGUs)that train random connectivity LSTMs,GRUs and MGUs at a time.This model can not only reduce the number of parameters and save time compared to the separate training but also adjust their importance weights dynamically to select a more appropriate neural network for predictions.Experimental results show that SRCGUs have better performance on the benchmarks used and flexibility.