MM Algorithm For Constrained Optimization Problems

In statistics,many problems can be transformed into the optimization problems with constraints,however,these optimization functions tend to form complexly or not differentiable,thus it is very hard to find the optimal value,such as the Lasso,Fused-Lasso,the FLSA.Based on MM(Minorization Majorization)algorithm,this paper studies the problem of lasso,Fused-Lasso,FLSA and so on.Firstly,we have drawn the essence that the process of solving Lasso with the MM algorithm,which is actual to use the ridge regression of variable parameter pressed on lasso and at sparse data.Secondly,we give the MM algorithm of the other relatively simple constrained optimization problems and the new iterative formula derived from the MM algorithm.Thirdly,when we use the MM algorithm to solve the Fused-Lasso problem,we propose a combined record algorithm and a record hop algorithm in order to avoid that the denominator is 0,and the practice prove the effect of consolidation of records.Finally,the paper applies the MM algorithm to solve the two-dimensional Fused-Lasso,and make examples of image smoothing.