Research On Adaptive Dynamic Programming And Its Applications To The Missile Interception Guidance

Adaptive dynamic programming(ADP)technique,as a novel approximate optimal control method in recent years,is actually derived from the reinforcement learning(RL)principle.It integrates the control theory and computational intelligence,and utilizes the function approximation structure to deal with the optimal control and differential game problem of nonlinear systems.Therefore,the ADP technique has attracted much attention for the researchers.In addition,with the rapid development of the aerospace technology,various novel intelligent aircrafts have emerged.Thus,the traditional interception guidance methods have been difficult to cope with the future aerospace requirements,such as the rapid task change,multi-objective decision and trajectory adjustable ability adaptively under the complex circumstance.In order to improve the missile’s abilities of online task adjustment,such as cruise,penetration and attack,thus realizing the requirements of intelligent and autonomous guidance,in this paper,a promising adaptive optimal interception guidance scheme is developed by combining the ADP technique and missile guidance,which integrates the robust control,adaptive control and intelligent control.This adaptive optimal method brings the intelligent principle into the design of guidance law,which enriches the ADP theory.Furthermore,it will provide the referable theoretical guidance and prospective exploratory experience for the development of intellignet guidance systems in the future.The main contributions are presented as follows.(1)Considering the effects of uncertainties of interception guidance systems,such as internal perturbation and external disturbance caused by the changes of flight velocity and flight altitude,we study the differential game-based robust adaptive optimal interception guidance problem.By introducing the upper bounded function of uncertainties into the performance index function,the robust control problem of original uncertain system is transformed into an optimal regulation control problem of its nominal system.The asymptotically stability and optimality of the original system with uncertainties can be guaranteed theoretically by modifying the differential game strategy.Then,the critic network is constructed to implement the developed differential game strategy online via ADP technique,and the stability of the closed-loop system is ensured.The effectiveness of the proposed method is verified by applying it to the missile interception guidance problem.(2)In the terminal guidance phase,considering the large maneuvering of target for escaping,the missile’s terminal time and terminal state are both required,and further the overload saturation may occur,we study the differential game-based finite-horizon adaptive optimal interception guidance problem with input saturation.First,by introducing a nonquadratic positive functional into the performance index function,the differential game strategy with input saturation is derived,and the associated Hamilton-Jacobi-Isaacs(HJI)equation is presented.Subsequently,we construct the time-varying critic network,and then,the time-varying optimal cost function is approximated online by designing a critic network weight updating law.Meanwhile,the terminal constraints are satisfied.The stability of the closed-loop system and the boundedness of critic network weight estimation error signal are both guaranteed.The missile interception guidance problem is utilized to demonstrate the effectiveness of the proposed method.(3)Considering the fact that missile may suffer multiple constraints under a bad battlefield environment,such as overload saturation,terminal angle constraint,terminal velocity constraint,and so on,we investigate the differential game-based adaptive optimal interception guidance problem with multiple constraints.For the nonlinear differential game systems in strict-feedback form,the feedforward+feedback control architecture is proposed.First,based on the backstepping method,the feedforward control input is given,in which the input saturation and output constraint are overcome by introducing an auxiliary dynamic system and barrier Lyapunov function,respectively.Thus,the tracking control problem of original system is converted into an optimal regulation control problem of the equivalent tracking error dynamic system.Subsequently,the critic network is built to approximate the optimal cost function online by using ADP technique,and thus implementing the proposed differential game strategy.Owing to the feedforward+feedback control architecture,the developed scheme can not only guarantee the stability of the closed-loop system,but also drive the system output track the desired reference signal.Meanwhile,the cost function is ensured to be minimized.Finally,the adaptive optimal guidance is realized by applying it to the interception guidance problem.(4)Considering the requirements of high dynamic characteristic and fast response of interception guidance systems,to reduce the computational complexity of the online adaptive optimal control method,the adaptive predictive optimal interception guidance problem based on ADP technique is studied.For the nonlinear systems in block strict-feedback form,the offline-online control architecture is developed by combining the ADP technique and continuous-time predictive control method.First,based on the backstepping control structure,the virtual control input is approximated online via ADP technique.Subsequently,the optimal control input in an analytical form is derived offline by using the continuous-time predictive control method.The proposed offline-online control structure not only guarantees the optimality of the closed-loop system,and improves the robustness of the controller,but also reduces the computational burden of controller effectively.The simulation of interception guidance demonstrates the effectiveness and superiority of the proposed scheme.(5)Considering the scenario where multiple missiles intercept a single maneuvering target,we investigate the distributed optimal cooperative interception guidance problem of multi-missile systems.For the strict-feedback nonlinear multi-agent system with input saturation,based on the leader-follower mode,the backstepping+ADP compound control structure is developed.First,by utilizing the command filtered backstepping control method,the distributed feedforward control input is designed,in which the auxiliary dynamic system is designed to deal with the input saturation,and the neural network is used to approximate the unknown nonlinear function.Thus,the cooperative tracking control problem of original system is transformed into a distributed optimal regulation control problem of the equivalent local cooperative tracking error dynamic system.Then,by using the ADP technique,the critic network is constructed to approximate the cooperative cost function online.The stability of the closed-loop system is guaranteed by utilizing Lyapunov method.Furthermore,all followers are able to synchronize to the leader signal cooperatively.Finally,the developed scheme is applied to the multi-missile cooperative interception guidance problem.(6)Considering the fact that the traditional time-triggered mechanism may consume tremendous computation resources and energy,even leads to the obstruction of communication channel,and thus destroying the system stability,we study the event-triggered distributed optimal cooperative interception guidance problem of multi-missile systems.For the affine nonlinear multi-agent systems,the event-triggered distributed optimal control law is proposed.Then,an adaptive triggering condition is given to guarantee the asymptotically stability of the closed-loop system.In order to implement the proposed controller,the critic network is built to approximate the cooperative optimal cost function online.By using Lyapunov method,the stability of the closed-loop system is analysized.Furthermore,the infamous Zeno behavior is avoided theoretically.By applying the proposed scheme into the multi-missile interception guidance system,the missiles are able to attack the target simultaneously,which verifies its effectiveness.