Research On Adaptive Dynamic Programming-based Optimal Cooperative Control Of UAV Formation

This thesis studies the optimal cooperative control of UAV formation based on the adaptive dynamic programming method and the consensus theory.Firstly,the consensus error dynamic model is established.Based on this model,the corresponding cooperative control techniques are formulated and developed from three aspects: the consensus problem of UAV formation system with input constraints,partially unknown dynamics and completely unknown dynamics,respectively.The main work is summarized as follows.(1)A distributed adaptive optimal control scheme is presented to solve the problem of nonzero sum differential game for UAV formation in the presence of input saturation limitation.In order to circumvent the problem of incontinuity caused by input limitation,a suitable non-quadratic functional is selected to transform it into the problem of solvable optimization.A single network structure for each UAV is designed to approximate the solution of the coupled Hamiltonian-Jacobi equation,and then the distributed optimal cooperative control law is obtained.For the UAV formation system,the superiority of using a single network rather than the typical dual network structure of ADP is more prominent due to it can not only reduce the memory requirements but also alleviate the computational burden of the UAV.In addition,the updates of all UAV neural network weights are simultaneous and continuous,which makes the control law smooth.(2)To cope with the cooperative control of UAV formation with partially unknown dynamics,an online adaptive optimal control scheme based on identification-critic structure is developed.In general,the key to the optimal cooperative control of nonlinear nonzero sum differential games is the solution of the coupled HJ equation.Because of the uncertainty for each UAV,an identification neural network is utilized to estimate the unknown dynamics and a critic neural network is used to approximate the solution of coupling HJ equation(the optimal value function),and then the optimal cooperative control law is derived.These two neural networks can update their weights synchronously based on the proposed identification-critic structure.(3)A new data-driven ADP algorithm is developed to conquer the situation of completely unknown dynamics and external disturbance in system.Firstly,a model-based policy iteration algorithm is presented.Then the iterative sequence of the value function and control strategy is proved to converge to the optimal function.In order to relax the algorithm's dependence on the accurate information of the system model,the model free iterative equation is derived by combining the previous model-based algorithm and integral reinforcement learning techniques.We further put forward a data driven iterative ADP method which use the generated system data to solve the model free iterative equation.Furthermore,it is proved that this model free iterative equation is equivalent to the model-based iterative equation.This means that the data driven algorithm can approximate optimal value function and control strategy.