Multiple Threshold Selection For Sparsity Penalty Likelihood

Ensuring the accuracy of high prediction and finding relevant prediction variables are two basic goals of statistics[53].The penalized likelihood method is the most wide-ly used in many ways to pursue these two goals.The penalized likelihood method is also known as regularization.A penalty item(also called a regular term)is added to the likelihood function,and the proportion of the likelihood function and the penalty func-tion in the target function is controlled by the size of the threshold(or the penalty value and the regular term coefficient),and the solution of the problem is weighed between the fitting precision and the sparsity of the model.Just like the learning problem in machine learning,it is "minimize your error while regularizing your parameters",that is,to minimize errors while regularized parameters.The selection of regularization coefficients directly affects the accuracy of estimation and prediction.The penalty likelihood method originates from the ridge regression(Ridge Re-gression)proposed by A.E.Hoerl in 60s and 70s and the R.W.Kennard system,which was originally designed to solve the estimated ill posed problem.And from 1996(least absolute shrinkage and selection operator),the academician of the Acade-my of science of the United States of America(least absolute shrinkage and selection operator),starting with the L1 paradigm as a penalty,the sparsity penalty likelihood is the most widely used variable selection method,and the penalty function with model selection ability is called the sparsity penalty function.At the same time,the selection of regularization coefficients not only affects estimation accuracy,but also affects the accuracy of model selection.The selection of threshold and regular term coefficients is one of the most impor-tant problems of the penalty likelihood and regularization method,and the common method is experience or cross validation(Cross validation).Cross validation requires the introduction of additional data.And because cross validation is used to traverse the threshold in one dimensional space for single threshold selection,cross validation of multiple thresholds needs to be traversed in multidimensional space.Therefore,cross validation can only give a unified regular term coefficient,if the individual vari-ables are to be individualized,then the method of cross validation is powerless,so the unity of punishment is only the choice of people.Compared with individualized punishment,the unified punishment has many shortcomings,for example,under the unified penalty,the same regular term value is difficult to achieve the best results of the estimation accuracy and the model selection accuracy.Uniform punishment will also cause difficulties in dimensional selection of the design matrix.These problems can be overcome under personalized punishment.But to make individualized punish-ment practical,there must be an adaptive selection method based on data of regular coefficients,which is different from cross validation.Starting from the linear model,an adaptive sparse penalty likelihood multiple threshold selection method is presented.In this paper,the relationship between the predicted MSEP and the estimated MSE is first discussed.Under certain conditions,it is pointed out that the optimal regular term coefficients corresponding to MSEP and MSE are consistent,and then the possibility of a given estimated family to reach the lower boundary of MSE is discussed.Then the regular term problem is connected with the estimated MSE and model selection,and the Global Adaptive Generative Alignment(GAGA)algorithm is constructed to imply the regular term implicit in the estimation process,so that the estimation problem is no longer plagued by the selection of regular terms.The GAGA algorithm is not only a method of generating multiple thresholds,but also a complete parameter estimation method.It does not have to set super parameters,and has good theoretical and strong performance.In theory,this paper proves the consistency of the model selection of GAGA al-gorithm and the asymptotic normality of estimation on the support set.The proof of the traditional sparsity penalty likelihood is to examine the nature of the target func-tion to reach the extreme value,and do not consider whether the extreme value can be reached,and under the assumption of n → ∞,a lot of details and problems will be covered.Unlike traditional methods,this paper depicts the nature of the current solution in the execution of the algorithm,and theoretically guarantees the practical performance of the GAGA algorithm.In the experiment,the first is the numerical simulation experiment.We choose the adaptive Lasso in the field of statistics and the Orthogonal Matching Pursuit(OMP)in the signal domain(OMP).These two excellent algorithms,which have both theo-retical guarantee and extensive application,are used as contrast algorithms.Through tens of thousands of numerical experiments,the performance of GAGA algorithm is investigated.The comparison experiment shows that the adaptive multiple threshold GAGA algorithm of the default parameter is better than the adaptive Lasso and OMP algorithm given the optimal super parameter,whether it is the estimated MSE or the model selection ability.Then,in order to investigate the ability of GAGA algorithm to deal with actual engineering problems,we use the GAGA algorithm in the optical system aberration detection based on the extended Zernike diffraction theory,which makes the detection precision greatly improved.