Research On Air Combat Situation Analysis And Decision Making Based On Deep Reinforcement Learning

In the process of modern air combat,due to the technological progress of both sides,some key information will be missing in the acquired situation data,thus causing the problem of incomplete information.Traditional situation analysis and decision methods are usually completed by knowledge reasoning,but they cannot solve the incomplete problem in situation data,while deep reinforcement learning has significant advantages in dealing with incomplete information and situation analysis and decision.Combining the characteristics of deep reinforcement learning technology,this paper proposes two solutions to the situation analysis and decision problem under incomplete information.One method is to complete the incomplete situation data first,and then conduct the situation analysis based on the complete data.Another method is to skip the reconstruction process of complete data and build an end-to-end situation decision model based on incomplete data by using multi-agent decision algorithm.The specific research contents of this paper are as follows:(1)Establish the air combat situation factor model.Aiming at the air combat scene,the index system of air combat situation elements is perfected.Combining with Markov theory,the composition of situation elements is expressed in the form of sextuple.The main tasks of situation awareness,situation understanding,situation analysis and situation decision in battlefield intelligent decision-making process are analyzed.(2)The incomplete information compensation method based on improved generative adversarial network is proposed,and combined with DQN model,the processed complete data is used for situation analysis.By inputting the position information of missing data and the camp information of situation data into the generator,the generator model can learn more association rules between data and improve the accuracy of the complete data after filling.The results show that the improved algorithm has the highest data filling accuracy,and the completed data can be used for situation analysis,which verifies the feasibility of the improved algorithm.(3)A multi-agent decision algorithm for multi-projectile cooperative scenario is designed and an improved multi-agent depth deterministic gradient algorithm is proposed.The multi-bullet cooperative combat model is established,and the state space,action space and reward function of the decision-making model are designed.Aiming at the problem of sparse reward in multi-bullet cooperative scenario,MADDPG algorithm was applied to multi-bullet cooperative scenario.By adding curiosity module based on state similarity in MADDPG algorithm,the problem of insufficient and excessive exploration of agent was avoided.Compared with the model trained by the original algorithm,the missile agent trained by the improved algorithm has a greater degree of improvement in reward value and mission completion rate,which verifies the effectiveness of the improved multi-agent decision algorithm.