The Multi-step Ahead Prediction Of Time Series Using The Regularized Kernel Learning

Statistical learning involves a large number of the models developed for the analysis and understanding of the data,and blends with the parallel development in computer science,particularly in machine learning.In these tools for model-ing and understanding the complex dataset,the use of positive definite kernels for estimation and learning is becoming more and more popular,and has been widely applied to the problems such as regression,prediction and classification.However,when applying the kernel learning model to practical problems such as prediction and clustering,there exist some problems such as the lack of flexibility,the subjective selection and so on.In addition,the time series forecasting,though establishing appropriate models to dig the inherent laws and trends from the past and present relevant data,provides a scientific basis for decision-making in social and economic activities.Although the scientific research on time series has been carried out very early,but due to the diversity and complexity of the time series,it is still the focus and difficulty of the academic research.In this paper,we study how to use the kernel method based on the regularization framework to study the multi-step prediction of time series.Regularization theory provides a general approach for the study of functions in the Hilbert space.Depending on the actual problems to solve,the adopted loss functions are not the same,resulting in the different models developed for different purpose.Under the framework of regularization,the third chapter tooke advantage of the quadratic loss function,and obtained the regularized single k-ernel learning model(kernel ridge regression and Gaussian process model from a frequentist perspective and Bayesian perspective,respectively),then compared the similarities and differences of the aforementioned two models,finally applied the developed single kernel learning model to time series data for multi-step ahead prediction.In order to overcome the defect that the multi step ahead prediction error of model will gradually enlarge,the third chapter introduced a data-driven signal filtering method,namely,empirical wavelet decomposition,to preprocess the time-series data,thereby reducing the interference of noise prediction process data.Therefore,the third chapter proposed a type of multi-step prediction mod-el based on the noise reduction method,including two hybrid prediction model(EWT-KRR,EWT-GPR)and a combinational prediction model.The combina-tion forecasting model utilized the Gauss process model to combine and optimize the multi-step ahead predictive values of the existing model(ARIMA)and sin-gle kernel learning model(LSSVM,SVM and ELM)using the prepossessed time series.Consequently,the deterministic prediction of the single models were con-verted to the final distribution forecast,thereby providing more accurate and predictive values and the risk of the prediction.In the proposed model training process,this paper proposed to adopt the metric learning to measure the simi-larity between the input data,and the coupling simulated annealing algorithm for the parameters of the model,in order to overcome the defects that the model parameter values is easy to fall into local optimum and stabilityWhen using the single kernel learning model(including the kernel ridge re-gression and Gauss process model)to predict the time series,it is needed to select the kernel function for the model to train and validate the model.However,it is not easy to select a appropriate single kernel function to measure the similar-ity of data.In addition,the requirement of performing multiple tasks are often encountered in the study of the practical problems.One of the solutions is to execute prediction tasks separately and repeat the prediction process many times.however,when there exists a correlation between prediction tasks,it is necessary to study and predict multi-tasks simultaneously.To serve the aforementioned purpose,the fourth chapter put forward a mixed-norm multiple kernel learning prediction model,and formulated two kinds of task model,namely single task multiple kernel learning model and multi-task multiple kernel learning model,and then generated the iterative prediction of time series by single task multi-ple kernel learning and the synchronized prediction by multi-task multiple kernel learning.In the proposed multi kernel learning model,a plurality of candidate k-ernel functions were linearly combined with the weight constraints in mixed norm,in order to automatically select the appropriate features by the model.And when performing multi-step ahead prediction of the time series,the Bregman distance was proposed to measure the metric distance between two time series,which was incorporated into the proposed model.Gauss process forecasting models show great advantages in terms of uncer-tainty and risk in time series prediction.However,the appearance of the outlying observations may significantly reduce the accuracy of the inference.the type and the choice of kernel function also affect the prediction performance of the re-gression model.In addition,in practical applications,it is often encountered the problems of multiple output prediction.For this purpose,the fifth chapter of this paper put forward a novel kernel learning model from the Bayesian perspective.The proposed prediction model is based on a multivariate linear model,using the good properties of the matrix-variate generalized hyperbolic distribution,and formed a function-variable hyperbolic distribution model by Bayesian inference and kernel techniques.This prediction model overcomes the shortcomings of the non-robustness of the Gauss process,and has good robustness to the time series forecasting with abnormal data.On the basis of the proposed model,we natural-ly generated two special models,namely,the single output kernel learning model and the multiple output kernel learning model.When applied to the multi-step ahead prediction of time series,the single output kernel learning model produced the iterative distribution prediction of time series,while the multiple output k-ernel learning model generated the synchronized distribution prediction of time series,thereby obtaining the uncertainty and risk of the multi-step ahead predic-tion of time series.And during the training process of the proposed model,the coupled simulated annealing algorithm was proposed to optimize the maximum likelihood for the optimal parameters of the model.In performing the multiple output prediction of time series,this chapter proposed the Bregman distance to measure the metric distance between two time series,which was incorporated into the proposed model as a priori information.In order to verify the validity of these proposed models and to develop its practical value in the above three scenarios,this paper applied them to the wind speed forecasting of wind fields in China.The experimental results showed that compared with the traditional prediction model,the prediction performance of these proposed models were greatly improved.