Variable Selection In Ultra-High Dimension Nonparametric Additive Model-Forward Additive Regression Method

In this paper,we propose a new method,Forward additive regression method,to solve the variable selection problem in ultra-high dimension non-parametric additive model with the dimension pn following pn = O(exp(n?)).In ultra-high dimensional data,the num-ber of independent variables is much larger than the sample size.The characteristic of the"P>n" has brought new challenges to the traditional multivariate statistical method.When the number of variables p is much larger than the sample size n,we can not use traditional variable selection methods to solve the variable selection problem.Statisticians proposed a method called Independence screening to deal with this issue.Independence screening method use a marginal score between dependent variable and independent variable to mea-sure the significance of the independent variable.If the marginal score is lower than the threshold value,the corresponding variable will be deleted.But independence screening method has following problems:Firstly,when an significant variable is marginal indepen-dent of the dependent variable,the independence screening method tends to ignore the sig-nificant variable.Secondly,when certain insignificant variable is highly correlated with a significant variable with strong signal,it tends to ignore the significant variable with weak signal.Wang(2009)proposed a forward regression method to solve the variable selection problem in ultra high dimension linear model which overcomes the shortcomings of Inde-pendence Screening method.Forward regression method utilizes information of selected variables when conducting variable selection which reduces the probability of ignoring sig-nificant variables.In application,it is impossible for users to get enough information to determine the model structure.The additive model proposed by Stone(1985)is a widely-used model with high elasticity,so we study the additive model in this paper.Forward additive regression method is an extension of Forward regression method which also over-comes the disadvantages of the independence screening method.Based on the results of numerical simulation,we can see that forward additive regression method can keep stable performance in solving variable selection problem under different situations.