| Ballistic missile is the dominant weapon of medium and long range attack,improving the penetration capability of ballistic missile is very important for the war of future.With the continuous development of artificial intelligence and other technologies,it has been a hot in the research of midcourse maneuver penetration to make ballistic missiles penetrate intelligently and autonomously.In this paper,the penetration antagonism between ballistic missile and interceptor is studied and Reinforcement Learning is proposed to improve the penetration capability of missile.The discount factor of Reinforcement Learning is analyzed and improved,and the fixed discount factor is replaced by increasing discount factor.Then,based on Deep Q Network(DQN)and Proximal Policy Optimization(PPO),the applicability of Deep Reinforcement Learning in midcourse maneuver penetration is verified.The specific research contents are as follows:(1)Discounting factor of Reinforcement Learning is analyzed and improved.Firstly,the basic principle of discount factor is studied and the research progress of flexible discounting in recent years is introduced.Then the increasing discount factor is proposed and the convergence of flexible discount is analyzed and deduced.Finally,the performance of increasing discount factor is simulated based on Open AI Gym.(2)Maneuvering penetration control model was established based on DQN and PPO.Firstly,based on the penetration scene of missile and interceptor,the penetration environment model was built by designing the state,action space and reward function.Then,the DQN model was built under the framework of Tensorflow,and the DQN agent was trained to analyze the applicability and convergence of the DQN model in the scene.Then,the PPO algorithm model is built in Tensorflow framework,and PPO agent is trained in the three-dimensional environment to achieve the approximation of the penetration control model.Meanwhile,the advantages and disadvantages of the two algorithms for penetration control model are analyzed and compared.(3)Penetration simulation system is constructed and the penetration interception simulation of PPO missile and TPN guided interceptor is implemented.Considering environmental error and time delay,the penetration simulation system of missile and interceptor in three-dimensional space is constructed.In the penetration simulation system,PPO intelligent missile and TPN guided interceptor are used for penetration interception simulation,and several simulations are carried out by changing the proportional navigation coefficients of interceptor.The results show that PPO missile has ideal penetration ability to TPN guided interceptor. |
