Research On UAV Maneuver Decision-Making Method Based On The Game Model

As the core force of the future battlefield,UAV plays a vital role in seizing air supremacy,and its autonomous maneuver decision-making ability is the key to play the combat effectiveness.Although existing air combat decision-making methods such as differential game and expert system have made some achievements,they still have some limitations such as long time spent searching decision results and poor adaptability.Therefore,how to make fast and accurate maneuver decision in the highly dynamic and highly competitive UAV confrontation environment is the main problem of this paper.With the background of close confrontation,game theory as the basis and intelligent algorithm as the tool,this paper studies the UAV maneuver decision-making method based on the game model.The specific research work is as follows:(1)The control parameters were designed based on F-16 UAV,and the basic maneuvering action library was enriched and improved.The maneuvering space of UAV was designed,and the maneuvering strategy set of UAV was constructed.Simulation experiments are conducted to test the designed control parameters and maneuver space respectively,and the results meet the design requirements.(2)Aiming at the problem that the basic swarm intelligence algorithm is inefficient and easy to fall into the local optimal value,an improved particle swarm optimization algorithm is proposed to solve the optimal maneuver strategy.Firstly,a one-to-one dynamic game model of UAV is established.Then,the difficult problem of Nash equilibrium is transformed into an optimization problem to search for optimization,and an improved swarm intelligence optimization algorithm is proposed to control the population diversity through the probability selection of particle concentration,so as to reduce the possibility of falling into the local optimal value in the optimization convergence stage.Finally,the algorithm is applied to UAV countermeasure maneuvering decision,and the performance of the improved algorithm is compared with the simulation experiment of single-machine countermeasure.The results show that the improved particle swarm optimization algorithm improves the global search efficiency and optimization accuracy,and improves the computational efficiency and accuracy of solving the optimal maneuver strategy in UAV countermeasure maneuvering decision.(3)In order to solve the problem of dimension explosion of traditional reinforcement learning algorithm in processing high-dimensional state input and the problem of unilateral optimization of one’s own strategy without considering the influence of opponent’s strategy,an improved DQN algorithm was proposed to generate effective antagonistic decisions.Firstly,a two-person zero-sum Markov game model is established in the one-to-one UAV scenario,and the basic state space,action space and reward function are designed accordingly.Then,for high-dimensional state input,a deep neural network is introduced to fit the state action value function,and the convergence and stability of the algorithm are improved by setting the experience playback technique and updating the network parameters by using the loss function.Secondly,for unilateral optimization problem,minimax equilibrium of game decision is introduced to generate targeted maneuvering strategies.Finally,the performance of the improved algorithm is compared by the simulation experiment of single-machine confrontation.The results show that the improved DQN algorithm can generate more accurate and effective maneuvering decisions against the opponent in the strong competitive environment through self-learning,which meets the real-time performance of the confrontation and has a higher decision-making level.