Robust Variable Selection Of Varying Coefficient Models

Variable coefficient model (VCM) is a very important class of non-parametric re-gression model, it takes into account the interactions between the indicator variables and covariates, which lead to very strong adaptability and modeling capabilities compared with the classical linear model. Therefore, VCM has the merits of both avoiding the " dimension curse " problem and good explanatory ability. Meanwhile, many other useful models can be extended from VCM, such as partially linear varying coefficient model (PLVCM), partially linear model, additive model and partially linear additive model, and it has a wide range of applications in the econometrics, biometrics and social sci-ence and it has become a powerful tool to deal with the multivariate nonparametric and semi-parametric regression problems.Existing estimation and variable selection methods of VCM and PLVCM mostly were built on least square or likelihood related methods. However, the existing approaches are expected to be sensitive to outliers and/or heteroscedasticity, and they no longer have good properties and their efficiency may be significantly. In this case, one can seek some robust methods from different aspects. In this paper, we conduct some reaseach about the robust estimation and variable selection methods and the main results of this study are described as following:(1) We provide a new robust variable selection method, refered as QKLASSO method, for VCM in the framework of quantile regression based on kernel smoothing and adaptive LASSO penalty, which extends the KLASSO method of Wang and Xia (2009). The consistency of variable selection is established and the varying coefficient functions achieve the optimal convergence rate in the same the smoothness conditions. Simulated examples and real data analysis are used to evaluate the finite sample performance of the proposed method.(2) Based on kernel smoothing and a double adaptive-LASSO-type penalty, we inves-tigate a unified variable selection approach for VCM in the framework of quantile regres-sion. The proposed method can not only select important variables but also identify the varying coefficient effect and constant effect variables. Under some suitable conditions, we prove the penalized statistic possessing the consistency in both variable selection and the separation of varying and constant coefficients. In addition, the estimated varying coefficients achieve the optimal convergence rate under the same smoothness assumptions. The finite sample performance of the proposed method are examined through simulation studies and real data analysis.(3) Browing the idea of modal regression estimation in Yao et al.(2012), we firstly propose the robust estimation method for PLVCM based on backfitting technique with two step estimation method, where the nonparametric functions are approximated by local polynomial. The optimal statistics are obtained for both parametric compoment and nonparametric component, and they both achieve the optimal convergence rates. We also give the estimation algorithm based on EM algorithm. Moreover, we study the variable selection for parametric component by using the regularized estimate method based on robust estimations. We also prove the Oracle property of proposed variable selection procedure. Extensive simulations and real data application are used to confirm the robustness and efficiency of proposed methods.(4) Based on the B-spline approximation and a double adaptive-LASSO-type penalty, we develop the variable selection procedure for both parametric component and nonpara-metric component in PLVCM. The proposed penalized estimators are consistency and they achieve the the optimal convergence rates under the same smoothness conditions. Simulation studies and real data analysis suggest that the proposed estimator has stable and competitive performance relative to other methods.(5) We discuss the model identification and variable selection problem for VCM with categorical effect modifiers under the framework of quantile regression. Based on the double penalties of adaptive-LASSO and adaptive-Fused-LASSO, we propose a regularized estimation method which can seelect important levels and identify the difference between different levels and the consistency of model selection is established. Extensive simulations and real data analysis further confirm the fine nature of the proposed method.(6) Based on hierarchial LASSO, we develop a new group variable selection method that not only removes unimportant groups effectively, but also keeps the flexibility of selecting variables within a group. We prove the consistency property in group selection of hierarchial LASSO, but it can not guarantee the consistency of variable selection within group. To this end, we further develop the bi-level variable selection based on adaptive hierarchical LASSO, and under the case of divergent dimension of the number of variables, we show that the new method offers the potential for achieving the theoretical Oracle property. Simulations and real data analysis confirm the theoretical findings.The innovations of the achievements in this dissertation are described as following. Firstly, we provide a method to select important variables and a unified variable selection method for the quantile varying coefficient model by using kernel smoothing, which further enrich the variable selection for VCM. Secondly, under the framework of modal regression, we study the robust estimation and variable selection for PLVCM, and the proposed method has good robustness and not sensitive to outliers, which provide a simple,robust and effective estimation and variable selection method. Thirdly, we investigate the bi-level variable selection in the framework of quantile regression, and we also propose the model identification and variable selection method for VCM with categorical effect modifiers.The achievements and methodologies in this study enrich the theory of robust esti-mation and variable selection, which also help to select important covariates to simplify the model and improve prediction accuracy in many fields, such as econometrics and biometrics.