A Hybrid Algorithm For Solving Nonlinear Least Squares Problems

Nonlinear least squares problem is one of the most important branches of non-linear optimization; it's widely used in chemistry, spectral data, neural networks,robotics, signal analysis, medical and biological imaging. Some general methodsare described for nonlinear least squares, such as Gauss-Newton method, Levenberg-Marquardt method, Quasi-Newton method and tensor method.F. Lampariello proposed a algorithm which is based on the Gauss-Newton step.The standard Gauss-Newton equation is suitably modified after a subsequence of theiterates, then the Levenberg-Marquardt step is used as the search direction. The step-size is computed by nonmonotone line search technique, which can guarantee theglobal convergence. The superlinear convergence is proved for the zero residual prob-lems.We propose a new hybrid algorithm for nonlinear least squares. At each iteration,we compute either a Gauss-Newton step or a Levenberg-Marquardt step. The trustregion technique is used to describe whether to accept the trial step and how to adjustthe Levenberg-Marquardt parameter. The new algorithm converges globally undersome standard assumptions and the quadratic rate of convergence is proved for thezero residual case. Numerical results for some standard test problems are reported,which shows that the new algorithm is promising, especially for the rank deficiencyproblems.