Constrained Nonlinear Optimal Control Stochastic Systems With Time-delay

The nonlinear time-delay stochastic system with some control constraints is considered in this paper. The objective is to find the optimal control law that minimizes the cost function. The algorithm for locally-optimal control of this stochastic control problem is presented in this paper. By time discretization, Euler method and Taylor expansion, our original problem is converted to a Linear-Quadratic-Gaussian problem in terms of the deterministic control and state deviations. Then using the Bellman dynamic programming principle, the locally-optimal control law and the corresponding optimal state trajectory are obtained. In particular, it is proved that Hessian matrix H is positive definite when the original cost function has a quadratic form. So there is an optimal control of the Linear-Quadratic-Gaussian system. Finally, we simulate a2-link6-muscle arm model to show the feasibility and the effectiveness of the algorithm. And sensibility analysis is made to some parameters in the model.