Event-Triggered Adaptive Dynamic Programming And Reinforcement Learning Control Methods

Adaptive dynamic programming and reinforcement learning(ADPRL)is an intelligent control method that combines traditional dynamic programming with neural networks(NNs)approximation technology.This method can effectively solve the problem of "curse of dimensionality" in traditional dynamic programming methods,and is suitable for complex large-scale dynamic systems.In addition,with the development of communication network and computing data,the time-triggered ADPRL method of traditional periodic sampling mechanism has the problem of high computing cost and network transmission cost,which is gradually difficult to meet the requirements of scholars for computing efficiency and resource utilization.The signal sampling and data transmission in the event-triggered control are carried out according to the requirements of the system.It can avoid unnecessary waste of communication resources and computing resources,and is an effective way to reduce computing costs.Aiming at the problems of limited network bandwidth,high computing cost and high network transmission cost when solving the optimal control problem,robust control problem and tracking control problem,this paper proposes a new event-triggered ADPRL optimization algorithm to improve the utilization efficiency of computing and communication resources and optimize the consumption of algorithm resources.The main works are as follows:First,three event-triggered ADPRL algorithm for optimal control problems.(1)An event-triggered ADP method based on integral reinforcement learning(IRL)is presented for input saturated continuous-time nonlinear systems.This method does not require knowledge of drift dynamics and the initial learning process is not subject to admissible control.(2)For the continuous-time non-zero-sum game control problem,an event-triggered ADPRL method is proposed.In this method,we first derive the nonlinear Hamilton-Jacobi equation for the non-zero-sum game control problem,the event-triggered ADP algorithm is used to solve this complex equation.The temperature control problem of multi-zone heating,ventilation,and air conditioning system was taken as an example to conduct simulation experiments.The simulation results show that this method not only reduces the burden of network transmission,but also makes each zone meet the required temperature.(3)An event-triggered IRL algorithm is presented for asymmetric input saturated non-zero-sum game.The derived coupled Hamilton-Jacobi equation based on IRL can reduce the requirement for complete knowledge of game problems.Players’ learning is based on event-triggering.A weight adjustment method based on experience replay is proposed.Second,two event-triggered ADPRL algorithm for robust control problems.(1)An event-triggered ADP method is presented for robust control of mismatched uncertain systems.This method transforms the robust control problem with mismatched uncertainties into an optimal control problem for an auxiliary system,the designed algorithm can avoid both initial admissible control and persistence of excitation conditions.It is proved that the event triggered ADP controller guarantees the robustness of the uncertain system and the uniform ultimate boundedness of the NN weight estimation error.(2)For an unknown continuous nonlinear system with control constraints and external disturbances,an event-triggered H∞control method based on experience replay is presented.This method uses NN-based system identification technology to identify completely unknown systems.It designs event-triggered control law and time-triggered disturbance law,and applies experience replay to weight update rules.The validity of constrained event-triggered H∞ method is verified by comparing the time-triggering method with event-triggering method.Third,three event-triggered ADPRL algorithm for tracking control problems.(1)A constrained optimal tracking control method based on IRL for event-triggering is presented.This method converts the constrained optimal tracking control problem into an optimal regulation problem by using an augmented system with discounted value function.No drift dynamics and reference dynamics knowledge are required to solve the Hamilton-Jacobi-Bellman equation.(2)An event-triggered ADP method is presented for tracking control of partially unknown constrained uncertain systems.Using the constructed augmented system,the optimal tracking control problem of the uncertain system is transformed into the optimal regulation problem of the nominal augmented system with discounted value function.The IRL method avoids the need of augmented drift dynamics and proves that the tracking error and the weight estimation error are UUB.(3)An IRL algorithm based on NN is presented for event-triggered tracking control with input constraints.By constructing an augmented system and using non-quadratic discount function,the tracking control problem is transformed into a regulation problem.The IRL technology relaxes the need for complete knowledge of the system.The event-triggered control reduces the execution frequency of the controller.The algorithm guarantees the non-negativity of the threshold.The experience replay technology improve data utilization.The simulation results verify the effectiveness of the algorithm.For each of the above works,(1)an adaptive event-triggered threshold condition is designed,and the controller only executes when this triggering condition is violated.The event-triggering mechanism can reduce the execution frequency of the controller.(2)In order to avoid the controller triggering infinitely in a limited time,which leads to the convergence of the system can not be guaranteed.Through theoretical analysis and simulation validation,this paper illustrates that the event-triggering ADPRL optimization algorithm can ensure that the event interval has a positive lower bound,which can effectively eliminate the Zeno behavior.(3)Based on the powerful nonlinear approximation ability of NN,it is used to approximate the optimal performance function in the control problem.The learning rules of the weights are designed,and the properties of the control system and the weight estimation error are analyzed.