| Recently, the optimal control problem based on approximate dynamic program-ming (ADP) is one of the research hot topics of control field. Approximate dynamic programming, which combines dynamic programming with reinforcement learning, utilizes function approximation structure to approximate the cost function and the control policy of the dynamic programming equation such that they satisfies the principle of optimization, then, the optimal cost function and the optimal control policy can be obtained. So ADP can obtain the optimal control without the "curse of dimensionality" of dynamic programming, and then receives a large amount of attentions. However, the theory of approximate dynamic programming is far from perfect. Many theoretical and technical issues of optimal control of dynamic systems based on ADP have yet to be addressed. Under the support of the National Natural Science Foundation of China (50977008), in this dissertation, several optimal control problems are investigated by using ADP. The main contents and contributions of the dissertation can be briefly described as follows:1. A novel optimal control scheme based on ADP is developed to solve online the optimal tracking problem of the continuous-time linear system with unknown dynamics. First, the optimal tracking problem is converted into designing an optimal regulator for the augmented system. And then, the solutions for the optimal control problem of the augmented system are proven to be equal to the standard solutions for the optimal tracking problem of the original system dynamics. Moreover, a new on-line algorithm based on ADP technique is presented to solve the optimal tracking problem of the continuous-time linear system with unknown system dynamics.2. A novel adaptive optimal control scheme ADP-based is presented for a class of discrete-time affine nonlinear systems. First, two neural networks (NNs) are used as the online parametric structures to respectively approximate the cost function and optimal control law, which are named as the critic network and the action network, respectively. Considering neural networks approxima- tion errors, Lyapunov theory is utilized to demonstrate the uniform ultimate boundedness of the system states and the weight estimation errors of all NNs.3. A new online adaptive policy learning algorithm is proposed to solve the H∞ control problem for a class of discrete-time affine nonlinear system with ex-ternal disturbance, which can solve online the Hamilton-Jacobi-Issue(HJI) e-quation by using the online information of state and input and to obtain the realtime optimal controller. Three neural networks are used as the online para-metric structures to respectively design the critic network、the action network and the disturbance network, and the novel weight update laws for the crit-ic、the action and action networks are derived. Lyapunov theory is utilized to demonstrate the uniform ultimate boundedness of the system states and the weight estimation errors of all NNs while explicitly considering the NNs approximation errors.4. A novel iterative two-stage dual heuristic programming (DHP) algorithm is proposed to obtain the optimal hybrid control policy for a class of discrete-time switched nonlinear systems subject to actuators saturation. First, a novel non-quadratic performance function is introduced to confront control constraints of the saturating actuator, which guarantees that the obtained control law is a smooth in the saturated controller. And then, a novel iterative two-stage dual heuristic programming is proposed for finding the solution of the constrained HJB equation, the convergence of the iterative algorithm is also given.5. For the optimal tracking control problem of a class of discrete-time nonlinear switched systems, an iterative ADP algorithm is proposed for obtaining the optimal hybrid control policy. First, the optimal tracking control problem is converted into the optimal regulation problem for a tracking error switched system. And then, the HJB equation of the error switched system is given, an iterative two-stage ADP algorithm is proposed to solve the HJB equation, At last, the convergence of the iterative algorithm is also given, which guarantees that the obtained tracking hybrid control policy is optimal.6. An iterative two-stage ε-ADP algorithm is presented to solve the finite-time optimal control problem of a class of discrete-time nonlinear switched systems. First, an iterative two-stage ADP algorithm is proposed to solve the HJB equation, the convergence analysis of the proposed algorithm is given. And then, an e-optimal control scheme is given to make the cost function close to its optimal value within an e-error bound, the finite-time optimal hybrid control policy is obtained.7. A new online H∞ robust load-frequency controller design scheme is proposed for the load-frequency control problem of the unknown power system. First, the H∞ control method is used to solve the system uncertainties. And then, the two-person zero-sum differential game is utilized to solve the H∞ control problem. And, a new data-based online ADP algorithm is proposed by using the ADP technique and Kronecker product theory, which can solve online the game algebraic Riccati equation (GARE) by using the online information of state and input of the system, without requiring the a priori knowledge of the system matrices. The online H∞ robust load-frequency controller can be obtained online using the proposed algorithm. |
PDF Full Text Request