Distributed Nonconvex Regularization

With the development of the information technology,it is possible to acquire massive and high dimensional data in many areas such as economics,biological information science and medical science.Because of the large scale of these observations,it is a new challenge for us to store and analyze this type of data.Distributed storage has been a normal storage form for large size data,which classify the data set in different computers.A natural question is,could the traditional machine learning methods suit to analyze the data which store in different computers.The regularization methods are recently used as feasible approaches to select variables and extract features.However,all those algorithms are used to operate based on a single computer.Generalizing the regularization methods is our main goal.The paper is arranged as follows:In chapter one,we review the background,of the regularization framework and briefly describe the significance of the distributed nonconvex regularization methods.In chapter two,we study the distributed model selection with SCAD penal-ty.Based on ADMM algorithm,we propose the distributed SCAD algorithm and prove its convergence.The results of variable selection of the distributed approach axe same with the results of the non-distributed approach.Numerical studies show that this method is both effective and practical which performs well in distributed data analysis.In chapter three,we study the distributed L1/2 regularization.We pro-pose the distributed L1/2 algorithm based on the ADMM algorithm,and prove its convergence.Notice the characteristic of the derivative at zero,we give a definition about the Restricted Prox-Regularity to complete our proof.Three examples show that the algorithm is practical in distributed data analysis and is more sparse than distributed SCAD.