Analysis Of Deep Neural Network Optimization Process Based On Dynamic System

We are entering the era of deep neural networks.Deep neural networks have been successfully applied in a wide range of fields,from image recognition to target location,from speech recognition to machine translation.Although deep neural networks have reached or exceeded human levels in many tasks,until now,deep neural networks have been faced with black box intelligence and poor interpretability.There are still many disputes in the understanding of the optimization process of deep neural networks.The interpretability of deep neural networks has always been the focus of academic research.With the deepening of relevant research,the importance of the optimization process of deep neural networks has gradually emerged.Explaining the optimization process of deep neural networks from a mathematical perspective is a very important step to open the black box of deep neural networks.At present,people have proposed many different related theories by combining with different mathematical methods.Nevertheless,these theories are still not deep enough to understand the optimization process of some deep neural networks.This paper mainly explains the optimization process of deep neural network from the perspective of dynamic system.On the one hand,in the context of LandScape's conjecture,the gradient of the loss function of the deep neural network is introduced into the Lipschitz continuity hypothesis,and the relationship between the full gradi-ent descent method and the dynamical system equation is established by the dynamic system manifold stability theorem.In this way,the convergence of the full gradien-t descent method is analyzed.On the other hand,combined with the transfer density two-discrimination theorem of stochastic dynamical equations and the numerical calcu-lation analysis of dynamical system equations,the stochastic gradient descent method and the momentum gradient descent method are studied.Based on these,a series of new optimization algorithm are proposed.In the experimental part,this paper builds a three-layer fully connected neural network model and a seven-layer convolutional neural network model,and then analyzes the new optimization algorithm based on the Fashion-Mnist data set.Finally,the results are compared with the results of some existing traditional optimization algorithms.