Design And Application Of Fractional Order Gradient Method Based On System Theory

With the rapid development of modern scientific research,the idea of optimization has been widely used in various scientific and engineering fields,such as optimal control,machine learning,and adaptive filtering.Gradient descent method is popular for its simple structure and ease of implementation,and is the most general and classical algorithm for solving unconstrained optimization problems.Recent studies have shown that introducing the concept of fractional calculus into the traditional gradient optimization algorithms can effectively improve their convergence performance.The fractional order gradient method has achieved excellent applications in many research fields due to the flexible order and inherent nonlocal property of fractional calculus itself.Moreover,considering the commonality between optimization theory and system theory,scholars have drawn inspiration from the field of system theory,and then designed a variety of new optimization algorithms for analysis and application.However,the research on fractional order gradient method based on system theory is still in its initial stage.Therefore,this dissertation will be devoted to explaining and analyzing the existing fractional order gradient method from the perspective of system theory,and then designing and demonstrating fractional order optimization algorithms based on system models and the event triggered mechanism,which is further applied and improved in the least mean square(LMS)algorithm.Firstly,from the perspective of system theory,the existing fractional order gradient method is explained and analyzed,and the system model is summarized,and then the convergence performance of various fractional order gradient methods is clarified and the related theories are improved.At the same time,based on the theory of system model and the stability analysis method,the fractional order momentum method and the fractional order particle swarm optimization algorithm are designed,and the equilibrium point and stability of the corresponding closed-loop systems are analyzed,so as to draw the convergence conclusion of fractional optimization algorithms.In addition,the system model interpretation and general correspondence of the gradient method for non-convex problems are given.Secondly,in order to reduce redundant gradient calculation in the optimization algorithm,a gradient descent method with fractional order iteration based on event triggered mechanism is designed.To achieve this,a non-uniform sampling mechanism is established to help the system save computational resources.Meanwhile,the stability of the corresponding closed-loop fractional order system is analyzed strictly to ensure that the event-based optimization algorithm can converge to the optimal solution.Moreover,a positive lower bound of the trigger interval is given to exclude the possible Zeno behavior.Finally,to further improve the convergence performance of traditional LMS algorithms,the system theory based fractional order gradient method is applied to the LMS algorithm,and a class of fractional order LMS algorithm with relatively simple iterative form is designed,and the relationship between the convergence performance and the order α is strictly analyzed.Furthermore,a fractional order LMS algorithm based on convex combination mechanism is proposed to take into account both the convergence speed and accuracy.In addition,the general conclusion of order,truncation number and convergence performance in fractional order iterative LMS algorithm is given.