Researches On Approximate Optimal Control Of Nonaffine Nonlinear Systems Based On Neural Networks

The optimal control problem of nonlinear systems is one of the principal and dif-ficult domain in the control field. The most existed optimal control methods due to their respective limitations, the analytical optimal solution is hard to gain. Hence, approximate dynamic programming as an effective way to deal with the optimal control problem of nonlinear systems, can overcome the "curse of dimensionality", and meanwhile obtain the approximate optimal close-loop feedback control law, has gained much attention from a lot of researchers. So, it is of great importance on nonlinear optimal control for the further research on the theory and algorithm of approximate dynamic programming. By employing the approximate dynamic programming, this dissertations makes the further research on the optimal stabi-lization and tracking control of unknown discrete-time nonaffine nonlinear systems, the optimal tracking control of discrete-time nonaffine nonlinear systems with non-symmetric dead-zone inputs, the optimal tracking control of unknown continuous-time nonaffine nonlinear systems, the zero-sum games of continuous-time nonaffine nonlinear systems. The main research of the dissertation can be briefly described as follows:1. A novel neuro-optimal control scheme is proposed for unknown discrete-time nonaffine nonlinear systems by using adaptive dynamic programming method. A neuro identifier is established by employing recurrent neural networks model to reconstruct the unknown system dynamics. By using Lyapunov theory, it is proved that the identification error converges to a small neighborhood around zero by adjusting the design parameters. Based on the established recurrent neural networks model, the approximate dynamic programming method is utilized to design the approximate optimal controller. Two neural networks are used to implement the iterative algorithm. The action neural network error and weight estimation errors are proved uniformly converge to a bounded region near the origin.2. Proposed a novel online optimal tracking control scheme for unknown general nonlinear discrete-time systems by using approximate dynamic programming method. First, an online neuro-identifier is established by employing a re-current neural network model to reconstruct the unknown system dynamics. The convergence of the identification error is proved. The optimal tracking controller is composed of the steady-state controller and the optimal feedback controller. An approximate dynamic programming method is proposed to solve the optimal feedback controller forward-in-time using online approximations. Two neural networks are used. Novel weight update rules for the critic and action neural networks are derived, and the weights are tuned online. The uniform ultimate boundedness of closed-loop system is demonstrated while considering the neural network approximation errors.3. A novel adaptive-critic-based neural network(NN) controller using reinforce-ment learning is presented for a class of nonlinear systems with non-symmetric dead-zone inputs. The adaptive critic NN controller uses two NNs:the critic NN is used to approximate the strategic utility function, and the output of action NN is to approximate the unknown nonlinear function and to minimize the strategic utility function. The tuning of the NNs is performed online with-out an explicit offline learning phase. The uniformly ultimate boundedness of the close-loop tracking error is derived by using the Lyapunov approach.4. For the first time, a novel robust approximate optimal tracking control scheme is proposed for unknown general nonlinear systems by using approximate dy-namic programming method. By a recurrent neural network model to recon-struct the unknown system dynamics. Via adding a novel adjustable term related to the modeling error, the resultant modeling error is first guaranteed to converge to zero. Then, the approximate dynamic programming method is utilized to design the approximate optimal tracking controller, which consists of the steady-state controller and the optimal feedback controller. Further, a robustifying term is developed to compensate for the neural network ap-proximation errors. Based on Lyapunov approach, stability analysis of the closed-loop system is performed to show that the proposed controller guaran-tees the system state asymptotically tracking the desired trajectory.5. Proposed a new iteration approach to solve the optimal strategies for finite-horizon continuous-time nonaffine nonlinear system quadratic zero-sum game. Through iteration algorithm between two sequences which are a sequence of state trajectories of linear quadratic zero-sum games and a sequence of cor- responding Riccati differential equations, the optimal strategies for the non-affine nonlinear zero-sum game are given. Under very mild conditions of local Lipschitz continuity, the convergence of approximating linear time-varying se-quences is proved.6. Proposed an approximate dynamic programming approach for a class of un-known nonlinear zero-sum game. A neuro-identifier based on a recurrent neu-ral network is used to approximate the unknown system dynamics. A novel adjustable term related to the modeling error is added to the recurrent neural network model, which guarantees the modeling error convergent to zero. Then, an approximate dynamic programming approach is given to solve the optimal performance index and the optimal control pair under the saddle point of the zero-sum game exists or not.Finally, concluding remarks are given. Some unsolved problems and develop-ment direction for the approximate dynamic programming are proposed. Further-more, the prospects of the further study are given.