Self-attentive Moving Average For Time Series Prediction

Time series analysis has been widely used in practical problems such as financial market prediction,power load prediction,weather,and environmental state prediction.Due to the difficulty of estimating the exact value of time series data,recent researches mainly focus on judging the future trend of time series.Time series prediction can often be viewed as a classification problem,that is,predicting the future trend direction of a time series,such as rising,falling,and steady.As one of the most commonly used technical indicators in time series analysis,the moving average indicator can summarize the trend of the time series in the past period of time in a simple and quick way.Although moving averages are widely used in different scenarios,traditional moving average indicators are calculated by assigning equal or predefined weights to data at different time points.However,assigning equal weights to each time series data does not reflect the differences in the importance of the data at different time points,and manually determining the weights also requires researchers to have a lot of domain knowledge and engineering skills.In addition,the practice of applying the same data weights to different time series does not take into account the differences between the inherent characteristics of different time series.In general,time series prediction can be done through the interaction between moving averages of different scales.For example,in the stock market,when a short-term moving average crosses up or down below a long-term moving average,it will generate up or down signals,respectively.However,this method of using different-scale moving averages for trend prediction is still based on heuristic rules,and how to better use multi-scale moving averages for time series prediction is still a problem worthy of study.To solve the above problems,this paper proposes an adaptive Self-Attentive Moving Average(SAMA)indicator.Firstly,the input signal of the time series is encoded by the Recurrent Neural Network(RNN),and then a self-attention mechanism is introduced to adaptively determine the weight of the data at each time point to calculate the moving average.In this paper,multiple self-attention heads are used to model multiple moving average indicators of different scales.On this basis,the bilinear fusion method is used to fuse the information between various dimensions of any two indicators.In this way,the cross-scale interaction of indicators is realized,and finally the idea of ensemble learning is used to adaptively learn the weights of the bilinear fusion vectors,and perform effective time series prediction in an end-to-end manner.Through extensive experiments on two real-world datasets,the experimental results verify the effectiveness of SAMA indicators for time series prediction and the rationality of integrating multi-scale SAMA indicators for prediction,and also show that the method proposed in this paper is significantly better than traditional moving average indicators and some current representative methods for modeling time series.