Quasi-Newton Methods And Their Convergence Properties

In this paper, we first propose a regularized BFGS method and L-BFGS method for solving monotone nonlinear equations. Under very mild conditions, we show that these methods are globally convergent even if the equation is not smooth and has infinitely many solutions. An attractive property of these methods is that the distance between iterates and the solution set is decreasing monotonically. Compared with the Gauss-Newton bassed BFGS method, the condition number of the iterative matrix is much less.In order to solve large-scale nonlinear equations, based on the Gauss-Newton based BFGS method proposed by Li and Fukushima, we develop a nonmonotone spectral gradient method and prove its global convergence. The method is an extension of the spectral gradient method for solving optimization problems.In Chapter 4, we introduce a nonmonotone line search technique and develop a nonmonotone MBFGS method and a nonmonotone CBFGS method for solving unconstrained optimization problems. Under mild conditions, we establish the global convergence of the proposed methods. Our numerical experiments show that the nonmonotone MBFGS/CBFGS method performs much better than the monotone MBFGS/CBFGS method does.In Chapter 5, we develop a nonmonotone BFGS trust-region method for solving unconstrained optimization problems. A good property of the method is that the objective function of the trust-region subproblem is a strictly convex quadratic function. Consequently, it is relatively easy to solve the subproblem. We establish a global convergence theorem for the method without requirement of the boundedness of the generated matrix sequence.In Chapter 6, we propose a nonlinear conjugate gradient method for solving unconstrained optimization problems. The method is developed based on the secant equation satisfied by the MBFGS formula. An advantage of the proposed conjugate method is that at each iteration, the method generates a sufficient descent direction for the objective function. Under mild conditions, we get the global convergence of the method.We then consider the numerical method for solving the second order cone complementarity problem (SOCCP). Based on a semismooth equation reformulation of the SOCCP, we propose a smoothing Broyden method for solving SOCCP. Under appropriate conditions, we prove the global convergence of the method. In addition, by the use of the hyperplane projection technique, we also propose a projection Newton method for solving SOCCP and establish its global convergence.