Two Doubly Stabilized Bundle Methods For Nonsmooth Convex Optimization

Nonsmooth optimization is an important branch of optimization theories and methods,but also a hot area for many domestic and foreign scholars to research,which is widely used in practical fields such as data mining,neural network learning,machine learning,image restoration,engineering and so on.All the time,scholars are committed to designing fast and efficient op-timization algorithms for solving nonsmooth optimization.Bundle method is not only a mainstream method but also one of the hottest topics in the in-ternational for the numerous methods to solve nonsmooth optimization.The main feature of the bundle method is that the information of a set of(bundle)iterative points generated by the previous iteration is preserved at each itera-tion,and these information will be used to generate the new iteration points.Bundle methods have been developed so far with more substantial theoreti-cal research results,therefore,to promote the relevant theories of the bundle method and developed fast and efficient optimization algorithm to solve the nonsmooth optimization has important theoretical significance and practical value.The bundle method mainly includes the proximal bundle method and level bundle method,this thesis combines the stability of the above two meth-ods,and proposes two new-type doubly stabilized bundle methods for solv-ing nonsmooth convex optimization,aiming at accelerating the convergence speed of the algorithm and improving the theory and numerical effects.Firstly,based on the classic doubly stabilized bundle method,the scheme of multi-step acceleration is introduced,and an accelerated doubly stabilized bundle method for nonsmooth convex optimization is proposed.The main features of this method are:first,different from the traditional method of using only one iterative sequence,the method uses three relevant iterative sequences,and the main function of these sequences is to establish the cut-ting plane model of the objective function,to generate the stability center(the"best" point produced by the algorithm to the current iteration)and control iteration sequence,each sequence is assigned its own role.Second,the algo-rithm combines the stability of the traditional proximal bundle method and the level bundle method,which makes the algorithm more stable and can ob-tain better numerical results.Third,the analysis demonstrates that the algo-rithm has global convergence.In addition,by choosing the parameters of the iteration sequence,the proposed algorithm can be returned to the traditional doubly stabilized bundle method.Secondly,a doubly stabilized bundle method with non-Euclidean norm is proposed.It mainly based on the doubly stabilized bundle method,intro-ducing the prox-function to improve the traditional doubly stabilized bundle method subproblem.The prox-function is introduced to replace the tradi-tional Euclidean distance in the subproblem of the quadratic programming for generating new iteration points,so that the geometry structure of the fea-sible set can be fully utilized in the calculation.The algorithm combines the stability of the traditional proximal bundle method and the level bundle method,so can have a better theory properties.And,the analysis proves that the algorithm has global convergence.When the prox-function function is taken as a special function,the algorithm can return to the original doubly stabilized bundle method.Finally,a preliminary numerical test of the accelerated doubly stabi-lized bundle method proposed in this dissertation is done.Numerical results show that the algorithm is superior to the traditional doubly stabilized bundle method.