The Perturbed BFGS Method For The Unconstrained Optimization With The Armijo Line Search

The classical BFGS method is the most efficient quasi-Newton method for solving optimization.The Newton method has quadratic convergence rate and high precision,but it requires computing the Hessian matrix of the objective function at each iteration.Quasi-Newton methods are some approximation of the Newton method,whose iterative matrix is an approximation of the Hessian matrix and satisfies the famous quasi-Newton equation.Quasi-Newton methods can reduce account of calculation without computing the Hessian matrix and possess superlinear convergence locally.When the inner between the difference of adjacent two iterative vectors and the difference of the corresponding gradients is greater than zero,the BFGS iterative matrix is symmetric and positive definite,which guarantees descent property of the quasi-Newton direction.Thus the stepsize can be computed by the Armijo or Wolfe line search.The BFGS method converges globally for convex minimization when the Wolfe line search is used has been proved in paper[11].The results of the paper[13,26]show that the BFGS method may diverge for nonconvex problems even if the Wolfe search is used.In order to ensure global convergence for noncorrvex optimization,it is always required to modify the BFGS foumula or line searches.Different from these globalization techniques,Liu presented a perturbed BFGS method and established its global convergence for nonconvex problems with the Wolfe search in paper[3].However,it has been not known whether this perturbed BFGS method converges globally for nonconvex problems when the Armijo line search is used,which is the main problem we want to deal with.The paper is organized as follows.In Chapter 1,we introduce the preliminary knowledge,the background of the problem and the main work of the paper.In Chapter 2,we study convergence properties of the perturbed BFGS method in[3].We show that this method has global convergence for nonconvex optimization with the Armijo line search.In Chapter 3,under some assumptions,we investigate local convergence properties of the perturbed BFGS method in[3]when the Armijo line search is used and show its superlinear convergence with this search.In Chapter 4,we do some numerical experiments to show efficiency of the proposed method.