Subspace Minimization Conjugate Gradient Methods Based On Regularization Model

Unconstrained optimization problems are widely used in engineering production and other fields.Among the methods used to solve these problems,conjugate gradient method is a very important choice.With the increasing of the problem scale,subspace technology becomes a very effective numerical method for solving large-scale optimization problems.Recently,due to the increasing complexity of unconstrained optimization problems,many scholars have noticed that the subspace conjugate gradient method is formed by the combination of subspace technology and conjugate gradient method.This method is mainly to minimize the approximate model of the objective function on a subspace.Most of the approximate model is quadratic approximation model of the objective function.In the process of solving unconstrained optimization problems,the regular algorithm is proposed as a new efficient algorithm.Because the regular model has more regular terms than the quadratic model,it is more accurate than the quadratic model in the approximation of the target problem.In this paper,based on the new challenge of how to effectively combine the regularization algorithm with the subspace conjugate gradient method and make full use of their advantages,three new subspace conjugate gradient methods are proposed for different subspace combined regularization model.The specific work is as follows:Firstly,for large-scale unconstrained optimization problems,a regular approximate optimal model in two-dimensional subspace is constructed by taking advantage of the fact that regular model contains more function information than quadratic model and the efficiency of subspace technology.The search direction is calculated and a reasonable model selection method is established.That is,when the target problem approximates the quadratic function,the quadratic model is used to approximate the target problem;otherwise,the regular model is considered to approximate the target problem.Subspace conjugate gradient methods based on the regular model are proposed,and it is proved that the search direction in the new algorithms satisfy the sufficient descent condition.Combined with the improved nonmonotone line search,the global convergence and R-linear convergence of the new algorithms are established.Through a large number of numerical experiments,it can be seen that for CUTEr date set,this method is quite effective compared with the classical conjugate gradient methods: CG_DESCENT method,CGOPT method and the valid subspace conjugate gradient method SMCG_BB method,SMCG_Conic method.Secondly,the two-dimensional subspace is extended to the three-dimensional subspace,and a regularized approximate optimal model on the three-dimensional subspace is constructed.Calculate the search direction,according to the reasonable model selection method,the appropriate choice of quadratic model or regular model.A three dimensional subspace minimization conjugate gradient method based on regular model is proposed.The convergence property of the algorithm is established with the improved nonmonotone line search.Numerical experiments show that,for CUTEr date set,this method is better than the method in two-dimensional subspace.