Nonlinear Least Squares Gauss-newton The-mbfgs Law

The structured BFGS method is a quasi-Newton method for solving the nonlinear least squares problem. It exploits the structured of the Hessian matrix of the objective function sufficiently. An attractive property of the structured BFGS method is its local superlinear/quadratic convergence property for the nonzero/zero residual problems. The local convergence of the structured BFGS method has been well established. However, so far, no theory has been established for the global convergence of the method. The main difficulty for the global convergence of the structured BFGS method lies in that the matrices generated by the method may not be positive definite. As a consequent, the direction generated by the method may not be a descent direction of the objective function. Although there have been some suggestions on how to ensure the positive definiteness of the iterative matrices, they are restrictive and is not easy to be implemented. In this thesis, by using a modified BFGS update formula, we develop a modified structured BFGS (MBFGS) method. An advantage of the proposed method is that the generated iterative matrices are always positive definite. This property is independent with the convexity of the objective function and the line search used. Therefore, the generated directions are descent directions of the objective function. By means of a backtracking line search, we develop a globalized structured MBFGS method for solving the nonlinear least squares problem. Under mild conditions, we prove that the proposed method is globally convergent. Moreover, we show that after finite iterations, the unit step is always accepted, and the method locally reduces to the modified BFGS method. Consequently, the convergence rate of the proposed method is superlinear. To speed up the method, we combine the structured MBFGS method with Gauss-Newton method to propose a hybrid method. We show that the hybrid method is globally and superlinearly convergent for nonzero residual problems and globally and quadratically for zero residual problems. We also do some numerical experiments to test the proposed method. The reported results show that the proposed method performs quite well for the tested problems.