Research On A New Conjugate Gradient Method And Spectral Gradient Methods

This thesis develops a new conjugate gradient method and a spectral gradient method for solving large-scale unconstrained optimization problems, and studies spectral projected gradient method for the minimization of differentiable functions on closed convex sets. Under some mild conditions, the global convergence results of these methods are established. Preliminary numerical results show that the proposed methods are efficient.Chapter 1 reviews the development of conjugate gradient methods and spectral gradient methods. Later, some basic knowledge about projected gradient methods is introduced.Classical Liu-Storey (LS) conjugate gradient method performs very well in practical computation, but it has no global convergence result under traditional line searches. In Chapter 2, a modified LS conjugate gradient method is presented and its global convergence is proved when weak Wolfe-Powell line search conditions are satisfied. The main advantages of this method are that (i) the parameterβ_k^new keeps nonnegative independent of line search used; (ii) The direction generated by the algorithm always satisfies the sufficient descent condition. Preliminary numerical results show that this method outperforms classical LS and PRP methods.The spectral (Barzilai-Borwein) gradient method was proved to perform better than some well-known conjugate gradient methods [1]. In Chapter 3, a new class of stepsizes for spectral gradient method is derived from approaching the new quasi-Newton equation given in [2]. Under some mild conditions, the global convergence result of the proposed spectral gradient algorithm in conjunction with a nonmonotone line search is established. Its features are that the method takes both available gradient and function value information, and achieves better approximation to the second-order curvature of the objective function. Preliminary numerical results show that the proposed algorithm is better than the Barzilai-Borwein approach.Nonmonotone spectral projected gradient (SPG2) method compares favorably with the well-known package LANCELOT for the minimization of differentiable function on close convex sets [3]. In Chapter 4, a new nonmonotone spectral projected gradient algorithm is developed. If the gradient of objective function is uniformly continuous, the global convergence result of the algorithm is established without requiring boundedness below of f and prior existence of the limit point. Numerical experiments show that the proposed algorithm achieves better performance than the SPG2 method.