Variable Selection With The Group SCAD

Driven by the development of science and technology, massive amounts of data are produced and stored with much cheaper cost than they used to be, of which high-dimensionality is a common significant feature. The massive amounts of data are often equipped with low dimensional structure at the same time. Hence, it is of great importance to develop sparse modeling and theoretical tech-nics by incorporating the structural prior of data. Regularization is an effective approach to cope with the high-dimensionality which emerges in recent years. In this dissertation, we mainly focus on the the theoretical and computational issues of group variable selection with nonconvex regularizers. Specifically, the contributions can be summarized into two aspects as follows,Firstly, we establish the high-dimensional theoretical framework on the group SCAD. Under regularity conditions, we prove that there exists a local solution of the group SCAD which is the oracle estimator with probability tend-ing to one, provided that the number of groups grows at a certain polynomial rate with the sample size. That is, the group SCAD satisfies the strong oracle property. Experiments show that the group SCAD performs well in variable selection and prediction abilities.Secondly, to deal with the inconsistency of theoretical and computed so-lution of the group SCAD, we propose the group LLA algorithm and analyze its high-dimensional properties. More specifically, we prove that two-step group LLA finds the strong oracle solution with group Lasso as initial estimator. Nu-merical studies indicate that the solution computed by the two-step group LLA has significant improvement over the solution computed by coordinate descent.