Deep Learning For Time Series

Time series data is everywhere in real life,financial trading,ecommerce,medical records,etc.Accurate and efficient methods for time series analysis is of great practical significance.The actual time-series data often contain complex nonlinear dynamics,while Deep Learning is indeed good at modeling complex patterns;in theory,a deep neural network can approximate any function.For the sequence data,Recurrent Neural Networks(RNNs)have been developed as a highly efficient sequence predictor.In this paper,we focus on the complex dynamics,non-linearity,seasonality and trending in electricity market price forecasting problem;algorithms and strategies are proposed based on Deep Learning and Statistical Machine Learning.The main contributions are as follows:(1)We take a detailed analysis on electricity market price forecasting problem based on EPEX France market,and formalize it as time series forecasting problem.Aiming at the difficulties of the traditional time series processing method in dealing with complex nonlinear time series,statistical machine learning methods are introduced into the energy market price forecasting problem.Due to the limited capacity of statistical learning models when processing complex time series,dynamic update of model is used as an effective strategy to improve the ability of the model to track the dynamics.In order to reducing the unnecessary computation cost of frequent updating,a variant of online kernel ridge regression is proposed to update the model in an efficient way;(2)To improve the model capacity and the ability to capture long-term dependency in time series,Recurrent Neural Networks are introduced to the problem.We propose Connective-state RNN,and based on it we propose state smoothing trick,which is an effective technique to accelerating RNN training for time series data.Furthermore,we propose a neural network architecture based on Gated Recurrent Unit(GRU)and state smoothing,which significally improves the model performance.(3)Seasonality in time series is very common,due to the periodic nature of human activities,which tends to be very important in electricity related problems.Besides,it's very important to capture the trending in practical electricity price forecasting system.In order to model the seasonality and trending explicitly,seasonal loss and trend loss based multitask learning is proposed.It's shown that the seasonal loss and trend loss can improve the generation of neural nets,and be able to improve the performance when forecasting the trend of sequence.