Based On Parameter Shrinkage Method Discussing The Fundamental Theory Of Variables Selection

During the last few decades,Variables selection method has been studied in the field of statistics.In traditional statistical linear model,stepwise regression is a very effective method to select variables.However,when it goes to ultrahigh dimensional data situation,the flaws of this method has exposed.From the beginning of Lasso,Tibshirani put forward in 1996,variables selection started to be a major topic of discussion among statisticians and more new models have been provided due to the work of those statisticians.As one of the most important work in this paper,the discussion of development of variables selection model and the fundamental theory of selection will be paid more attention.Tibshirani(1996)put forward Lasso,which could use to solve high-dimensional data problem that could not be figured out by traditional linear model.It could not only get a good estimation of parameters,but also shrink the number of variables to simplify the model.Many good algorithms have been discovered to estimate Lasso,such as,Fu(1998)put forwards shooting algorithm;Osbome,M.R.put forwards Forward-Stagewise Selection and Forward-Stagewise Selection;Efron(2004)put forwards LARS.All of these algorithms are great explanations to the Lasso's theory of selection.Inspired by these algorithms,we are trying to use utility to explain why Lasso method have the property of selection,and the ridge regression without it.For the natural flaws of Lasso,the estimations of Lasso is bias,and the result is unstable.Fan and Li(2001)said that a good penalty function should result in an estimator with three properties,Unbiasedness,Sparsity and Continuity.He called it Oracle property and put forward a new mothed with oracle property named SCAD.Under this new standard,some new methods with oracle property has been put forward.Zou and Hastie(2005)put forward Elastic Net Lasso;Zou(2006)put forward Adaptive Lasso;Yuan and Lin(2006)put forward Group Lasso and so on.Differ from SCAD method with a non-concave penalty function;the methods above all are concave penalized function that their local optimal solution is the global optimal solution.At the same time,Elastic Net Lasso and Group Lasso have the 2-norm penalty so that these two method have group effect that make it could select a pair of variables with strong correlation instead of only one of them.To compare these methods distinctly better,we are trying to apply these methods to some cases and compare the results of them.Nowadays,the developing trend in field has been changed slightly.I plan to talk about some new methods briefly,such as All Pairs Lasso and Pliable Lasso that provided by Bien and Tibshirani.They are of hierarchical interactions effects that makes it having the ability to fix their estimations according to different characteristics of sample to improve the accuracy of their result.In this paper,I will pay more attention to explain the fundamental theory of selection with utility,and try to discuss these methods' advantages and disadvantages.Spontaneously,I will provide some applications about Lasso in a more generalization fields.What's more,I will put forward a generalized Stage-wise method to solve non-convex penalty function problem.In the end of this paper,I try to talks about the recent development of variables selection,and some flaws that this field has.