Research On Multi-UAV Cooperative Task Planning Algorithm Based On Reinforcement Learning

With the rapid development of Unmanned Aerial Vehicle technology,the application range of UAVs has been greatly expanded.As the main combat mode of future drones,Multi-UAV cooperative combat has attracted broad attention from many scholars.A good multi-UAV collaborative mission planning method can significantly improve the resource utilizing efficiency and survival probability of UAVs,which has great significance for mission completion of multi-UAVs.However,the multi-UAV collaborative mission planning problem is complicated,and if only relying on the experience and knowledge of the designers,it's difficult to obtain the good adaptability of multiple UAVs in complex environments.As a feasible technical route to achieve good adaptability under the complicated environment of multi-UAV,the reinforcement learning algorithm has made great achievements in the learning of some tasks that are difficult for humans,such as games,Go,and robot control.This thesis uses top-down,bottom-up methods with the reinforcement learning methods as the main optimization tools to solve the task planning of multi-UAV cooperative reconnaissance and cooperative operation,Firstly,study on the multi-UAV cooperative reconnaissance mission planning from the top-down way.Combined with the goal of collaborative task,the key factors affecting the cooperative reconnaissance mission,such as environmental information,drone endurance constraints,survival probability,etc,are summarized by deeply analyzing the cooperative reconnaissance problem to establish a collaborative reconnaissance mission planning optimization model.The model can solve the mission planning problem by establishing a sequential neural network.Combined the optimization objectives of the model with the reward function of reinforcement learning,the performance of the sequence neural network can be improved,and in this way,the sequence neural network can be continuously optimized.Compared the performance of reinforcement method with the traditional group intelligence method in the simulation experiment,the advantages of reinforcement sequential neural network methods in time consumption and generalization capabilities can be verified.Then,study on the multi-UAV collaborative combat planning problem from the bottom-up approach.Regarding multi-UAV as an independent agent,the neural network that reinforces the learning for multi-agents is established to ensure that multiple agents can communicate and cooperate with each other to adjust individual behaviors spontaneously.And the Enhanced Deep Distributed Recurrent Q Network based on sampling of different environment importance is proposed,which enables the drone to improve the learning efficiency by empirical playback mechanism and reduce the interference between multiple UAVs.At the same time,the intensive learning reward function of collaborative operations is shaped to guide the multi-UAVs to complete the overall combat mission.Compared with the enhanced learning network based on Independent Q Learning and Deep Distributed Recurrent Q Network,the simulation results show that EDDRQN has significant improvement in stability and learning efficiencyFinally,the multi-UAV cooperative reconnaissance problem is simulated in the AirSim 3D simulation platform environment,and the effectiveness of the algorithm based on sequence neural network is verified.