Novel Approaches On Variable Selection And Its Application Of Nonlinear System Based On Kernel Methods

Beginning in the1990s, the world has entered the era of big data, nonlinear systemanalysis facing unprecedented data explosion, where the feasible algorithm in a lowdimensional will enhance its computational complexity exponentially andgeneralization ability becomes poor with the number of observed variables increasing. Itleads to model dimension disaster problem. To this kind of high-dimensional dataprocessing, an intuitive idea is to choose and keep some important variables, calledVariable Selection. But when it faces to the complex nonlinear industrial process, theoriginal observed variables often show their nonlinear, redundancy, anddelay-dependent features. It is difficult to realize this kind of variable selection. Anothereffective way is called feature extraction, by certain nonlinear compression throughextraction of original variables to replace the original variable sets, but the extractedmatrix is obtained after some mathematical mapping, no longer has its original physicalmeaning. In this paper, in terms of dimension reduction advantages, variable selectionmethod of nonlinear system is put forward after the transformation of feature space, theresults are as follows:①Kernel principal component analysis (KPCA) is one of the most commonlyused feature extraction method by far, but dimension in feature space generally higherthan the original dimension of input variables for the nonlinear implicit mappingproduct operations within process data. In this paper, the existing single kernel functionis improved to linear weighted multiple kernel KPCA, called MKPCA. The parametersare optimized with cross validation. Based on the two models of nonlinear static anddynamic system, the proposed method can effectively reduce the dimensions of featurespace compared with PCA, KPCA.②Although proposed MKPCA method can reduce the dimension of featurespace, but it cannot determine an optimal variable subset, which can remove redundantvariables and fully represent the original data structure characteristics. This paperproposes a Kernel Independent Component Analysis (KICA) and False NearestNeighbor method (FNN) variable selection method of nonlinear system. The originalnonlinear data is mapped to the characteristics of linear space using kernel function, atthe same time independent component analysis is used to eliminate themulti-collinearity among variables to build an independent feature space. On this basis, the distance changing to feature vector in KICA subspace with FNN is calculatiedbefore and after removing of each variable. In that case, their importance on thedependent variable is obtained. Taking the conversion rate of Hydrocyanic Acid (HCN)production to predict target, optimal variable subset is selected, and the regressionanalysis results show the reliability of bulit parsimonious model.③Proposed KICA-FNN method only consider the selected process variables tokeep their independence, in some nonlinear discriminant problem, the selected variablesshould have the best explanation ability to dependent variable. A combination of KernelPartial Least Squares (KPLS) and FNN is proposed here for nonlinear variableselection. As numerical validation of two typical nonlinear classification models, theresults show that the method can determine the effective input variable subset,fundamentally, closely related to discriminant model, which can directly reduce thedimension of models, at the same time improve the prediction accuracy and reliability.④Proposed KPLS-FNN method considering the explaining capacity of theselected process variables on dependent variable, in order to further consider theselected variable is set to make inter-class farthest and intra-class most closely indiscriminant model, an combination of Multi-kernel Fisher Discriminant Analysis(MKFDA) and FNN is putted forward. According to the actual nonlinear fault isolationproblems in chemical process Tennessee Eastman for two classes,53-dimension processvariables is dropped to5-dimension variable subset, while, fault identification accuracyincreased to94.55%from72.12%with all53-dimension model.⑤For the high-dimension process variable and small samples of some nonlinearsystem, its pattern analysis will lead to equation ill-posed problem. In this paper, nucleargradient vector variable selection method is presented for this small sample system.First, all the training sample is mapped into support vector gradient. On the basis, theindexes in the projection axis are calculated in order to obtain the size of the importanceof each index. Then, optimal index combination is tested with the highest modelaccuracy, while, variable selection is realized. Index system with16-dimension ofsecurity evaluation in mine ventilation system is studied for two scheme which is5-dimension and the other is2-dimension. The two reduced model is tested bothavailable for the security evaluation system.