Improved Levenberg-Marquardt Method And Its Application

The solution of nonlinear equations is a hot topic in optimization research,and it has a very wide range of applications in engineering technology,production management,economic planning and other fields.This paper mainly studies the Levenberg-Marquardt(referred to as LM)method.By improving the LM method,the nonlinear equation system problem is solved,and the modified LM method is applied to the BP algorithm of the neural network for fitting nonlinear functions.The following are descriptions of some of the main chapters:Firstly,the LM method based on an extend LM parameter is proposed.In this chapter,there are an improved LM algorithm by combining an extend LM parameter and an approximate LM step.The algorithm has global convergence,and its convergence rate is cubic based on local error bound condition.In the numerical experiment part,the algorithm will test two types of singular nonlinear equations with medium and low statistical dimensions,by the calculations of function,the calculations of Jacobi matrix and the total of calculations.It can be see two numerical tables,and the results in the table show that the improved algorithm proposed in this chapter is feasible.Secondly,based on the LM parameters given in Chapter 3,an improved Shamanskii-like LM method is proposed.This method uses the corresponding line search before each approximate LM step to make the algorithm achieve the effect of speeding up.By proving,The improved algorithm has global convergence and based on the local error bound condition,its convergent rate is m+1 order.In the numerical experiment part,the algorithm will test two types of singular nonlinear equations with dimensions of 3000 and 10000 respectively.The computations of function,the calculations of the Jacobi matrix,the final value of the function norm,and the time spent in the running process are lie in two tables.The numerical results show the feasibility of the algorithm.Finally,it is to apply the improved LM method proposed in the fourth chapter to the BP algorithm,and then obtain the improved LMBP algorithm.In order to verify the effectiveness of the improved LMBP algorithm,the algorithm is used to test and fit nonlinear functions,and the number of neural network layers is 3,the weights of each layer are 1-10-1,and the selection range is [-1:1] take 21 sample points at an interval of 0.1.After testing,the corresponding mean square error convergence graph and curve fitting graph of the improved algorithm and the other three algorithms are given,And the comparison of numerical effects from the number of iterations,the mean square error at convergence,and the running time is given,and the results show that the algorithm is feasible.Combining adaptive technology,random sampling and large-scale numerical experiments is the further research direction of the improved LM algorithm in this paper.