Tuning Parameter Selection In Multivariate Linear Regression With Penalty Function

There are some methods for variable selection which can help us to do the selection process. However, for high-dimensionality data, those traditional methods are computationally expensive. In addition, ignoring stochastic errors in the variable selection process of previous steps is still a problem. This paper introduces some penalty functions to make penalty on the parameters for high-dimensionality data.Firstly, Some traditional methods are introduced for variable selection. Furthermore, this paper introduces Oracle estimator, proving its feasibility in theory. Considering the number of parameters is finite and infinite, SCAD has Oracle property but LASSO and Ridge do not have. Thirdly, this paper uses cross validation to select tuning parameter of LASSO and Ridge Regression, using GCV and BIC for SCAD. Finally, Linear Regression and robust regression are utilized to fit the model regarding a real data example regarding sexual discrimination of salary, and then doing tuning parameter selection and evaluation of variables with the penalty function that mentioned above and draw some conclusions.