Research On Continuous-time Algorithms For Several Distributed Optimization Problems

Thanks to the inspiring progresses in multi-agent systems,the research on distributed optimization has drawn a great deal of attention in recent decades,and it's abundant re-search results have been widely applied to various science and engineering fields such as machine learning,smart grid,artificial intelligence and so on.Distributed optimization takes the advantages of network communication to divide a large-scale and complex opti-mization task into several small tasks,and assigns these small tasks to multiple agents with certain perception,communication,calculation and execution capabilities.A multi-agent network is formed and then distributed optimization calculations are performed based on the network.Distributed continuous-time algorithms based on multi-agent systems also have attracted the interest of many researchers due to their advantages in computing effi-ciency,model stability and so on.In this thesis,we presents different types of continuous-time algorithms based on multi-agent systems for several kinds of distributed optimization problems and also analyze the convergence of these algorithms.The details are as follows:1.Based on the idea of penalty,we propose a penalty-like continuous-time algo-rithm to solve a nonsmooth distributed convex optimization problem subject to equality and inequality constraints.The most existing distributed continuous-time algorithms usu-ally own high dimensions due to introducing too many auxiliary variables,which is not suitable for large-scale problems.The proposed continuous-time algorithm in this paper has a lower solution space dimension,and the state solution of the proposed algorithm reaches consensus and converges to an optimal solution to the considered distributed con-vex optimization problem.Moreover,several illustrative examples are given to show the effectiveness and practicality of the proposed algorithm.2.Based on the idea of approximation,a continuous-time algorithm is presented to solve a distributed nonconvex optimization problem with affine equality and convex in-equality constraints.The convergence of the existing continuous-time algorithms mostly rely heavily on the convexity assumption of the objective functions,and these algorithms are not suitable for solving such distributed nonconvex optimization problems.Firstly,we relax the consensus constraint of the considered distributed optimization problem and ob-tain a related approximate distributed optimization problem.It is proved that the optimal solutions of the approximate distributed optimization problem(i.e.sub-optimal solutions of the considered distributed optimization problem)are able to approach the optimal solu-tions of the original distributed optimization problem.Next,a continuous-time algorithm is presented to solve the approximate distributed optimization problem,and the conver-gence of the state solution for the presented algorithm is obtained.Several illustrative examples are shown to validate the effectiveness of the presented algorithm.3.Based on the theory of negative gradient systems,a negative gradient system is designed to solve a quadratic distributed convex optimization problems with affine equal-ity and linear inequality constraints.The main difficulty is how to judge the convergence rate for the state solution of the constructed algorithm while the local objective functions are generally convex.Based on the smooth penalty idea,this paper first obtains a new quadratic distributed convex optimization problem,and proves that the optimal solution of the new quadratic distributed convex optimization problem can approach the optimal solution of the original quadratic distributed convex optimization problem.Next,a nega-tive gradient system is designed to solve the new quadratic distributed convex optimization problem.With the help of?ojasiewicz inequality,it is proved that the state solution of the negative gradient system converges at an exponential rate.Finally,several numerical examples are shown to verify the effectiveness of the designed negative gradient system.4.Based on the theory of inertial systems,a second-order acceleration algorithm is designed to solve a distributed convex optimization with inequality and set constraints.The most existing distributed continuous-time algorithms are usually first-order ones,and it is usually hard to judge the convergence performances for these first-order algorithms.Due to the control design for the acceleration,the second-order continuous-time algo-rithms can often achieve faster convergence rate.Moreover,the existing second-order continuous-time algorithms are mostly designed to solve unconstrained distributed con-vex optimization problems,and are not suitable for solving such constrained distributed convex optimization problems.It is acquired that the state solution of the designed algo-rithm in this paper converges to the optimal solution of the considered distributed convex optimization problem and an error function which demonstrates the performance of the de-signed algorithm is convergent to zero with a rate of o(t^-2).Several numerical examples are given to show the effectiveness of the presented second-order accelerated algorithm.