Time Series Combination Prediction Model Based On Support Vector Machine

There are a great deal of time series data especially multi-dimensional time series data in natural science and social science. Time series which are affected by environmental factors have inherent dynamic and nonlinear features. It is of great significance to develop high precision time series analysis method especially for nonlinear multi-dimensional time series because prediction is the foundation for understanding and decision-making. There are mainly two development directions for time series analysis:classical time series analysis and phase space reconstruction.The first part of this paper studies the direction of classical time series analysis methodIt is the key to extend order, determine order, filter variables and select regression model in classical time series analysis, the formers are often coupled with selecting regression model together. The traditional classical multidimensional time series analysis methods are modeled linearly, such as controlled autoregressive integrating moving average (CARMA) and controlled autoregressive (CAR), but their prediction abilities are poor. The back-propagation neural networks (BPNN) which is based on the empirical risk minimization has good nonlinear prediction ability, but falls into local minimum easily and has poor interpretation and strong empirical defects. Support vector machine (SVM) which is based on statistical learning theory has solved the local minimum, overfitting, and nonlinear problems, and has the advantage of global optimization and strong generalization ability, so support vector machine is used as the basic modeling tool in this paper.1 Combination prediction model—SLR-LSSVMThis paper proposes a combination prediction model (SLR-LSSVM) which the impact factors are filtered by stepwise linear regression (SLR), and then the model is established based on least squares support vector machine (LSSVM). The simulation experiment is carried out on the second generation corn borer larvae occurrence which has eight meteorological factors, and the prediction results show that SLR-LSSVM’s performance is superior to reference models, which indicates that the proposed model based on the factors filtered and SVM can improve the time series prediction precision. 2 Combination prediction model—CAR-LSSVMSLR-LSSVM only considers the affects of environment factors, without considering time series’inherent dynamic feature (without extending order), and the variables are filtered by SLR. Although CAR considers the effects of environmental factors and dynamic features, its order is determined by multiple linear regression (MLR) and variables are filtered by SLR too. This paper proposes a combination prediction model (CAR-LSSVM) Firstly, the optimal order is determined by minimum mean squared error (MSE) with LSSVM, and then the retained variables are obtained by nonlinear filtering after extending the order, lastly, the prediction model is established on the retained variables by LSSVM and the CAR-LSSVM’s performance is tested on the data of the moth-eaten ration of Leguminivora glycinivorella Mats which has five factors. The prediction results show that CAR-LSSVM’s performance is superior to reference models, such as MLR, SNR, LSSVM, SLR-LSSVM, and CAR, which indicates that it is necessary to consider environmental factors, dynamic features, nonlinear determining order and nonlinear filtering factors together.3 Combination prediction model—GS-LSSVMCAR’s order determined by F test and CAR-LSSVM’s order determined by minimum MSE have common defects:one is that the optimal order is obtained from low to high gradually is time-consuming, another is that the optimal order obtained by extending with the dependent and independent variables together is easy to cause information redundancy, while variables filtering are time-consuming and determining order is terminated before obtaining the optimal easily may reduce model’s performance. This paper proposes a high precision and determinatiion order fastly combination prediction method (GS-LSSVM), which can reflect time series’dynamic features and the affect of environmental factors. Firstly, the time series structure are analyzed by semivariogram of geostatistics (GS) and the optimal order is determined by variable range fastly, secondly, the redundancy information and dimension are reduced by principal component analysis, finally, the model is established on LSSVM. GS-LSSVM is applied to predicting the Dendrolimus punctatus occurrence area and the fifth generation brown planthopper for late-season rice. The prediction results show that GS-LSSVM’s performance is superior to LSSVM, GS-BPNN and has the advantage of determining order fastly and accurately. GS-LSSVM not only reflects the dynamic features and the affect of environmental factors, but also has good generalization ability. Therefore GS-LSSVM has a broad range of applications in the time series prediction field.4 Combination prediction model—ARIMA-DSVMThe training samples will be larger as time passed, the training time of LSSVM will too long to be accepted. More importantly, all history samples are involved in training unreasonable and each sample impact on the prediction results is different. Dynamic insensitive cost function support vector machines (DSVM) can adjust insensitive loss function parameters (ε) dynamically whereby the recentε-insensitive errors are penalized more heavily than the distantε-insensitive errors. This paper proposes a combination prediction model (ARIMA-DSVM) to predict the time series which characteristic is unknown. Firstly, the linear component of time series is predicted by ARIMA, and then the ARIMA prediction errors are corrected by DSVM. The result on Dendrolimus punctatus occurrence area shows that ARIMA-DSVM’s performance is superior to reference models such as ARIMA and DSVM.The second part of this paper studies the direction of phase space reconstructionTime series prediction model based on phase space reconstruction and LSSVM includes two key steps:determining the time delay (τ) and embedding dimension (m) in phase space reconstruction, and selecting the regularization parameter (y) and the kernel function width parameter (σ) of LSSVM. In previous studies, phase space reconstruction and LSSVM parameters are determined independently, so the determined r and m can not always ensure that LSSVM has the optimal prediction precision. Therefore, joint optimization forτ, m,γand a is a very attractive choice, which is driven purely from data and need not any priori knowledge of the time series. However, a multi-factor and multi-level joint optimization by exhaustive search algorithm is very time-consuming.5 Combination prediction model—GA-LSSVMA multi-factor and multi-level joint optimization by exhaustive search algorithm is very time-consuming, while genetic algorithm (GA) is a heuristic algorithm, which has parallel search ability. This paper proposes a combination prediction model (GA-LSSVM) in which theτ、m、γandσare jointly optimized by GA. The simulation experiment are carried out on Mackey-Glass and Mackey-Glass with noise, the prediction results show that GA-LSSVM is a stable and effective time series prediction model.6 Combination prediction model—UD-LSSVMGA is a heuristic algorithm which falls into local optimal easily. Uniform design (UD) arranges the experimental numbers by selecting good lattice points to reduce the experimental numbers greatly, which tends to distribute uniformly with low bias. LSSVM based on the structural risk minimization can solve the local minimum and nonlinear problems, and has excellent generalization ability. This paper proposes a combination prediction model (UD-LSSVM) to solve theτ、m、γandσjoint optimization problem by UD and self-calling LSSVM. The simulation experiments are carried out on Mackey-Glass, Lorenz and yearly sunspot time series, and the results show that the UD-LSSVM reduces the computational complexity and obtains high prediction precision, and the prediction results are superior to the results reported in the literature. The results indicate that UD-LSSVM is a fast and efficient jointτ-m-SVM parameter optimization prediction model for time series based on data driven.