Research On DDPG Conflict Resolution Algorithm Based On Multi-dimensional State And Action

In 2019,the annual passenger throughput of domestic airports exceeded 1.3 billion pas-sengers,an increase of 6.9%over the same period of last year.With the increasingly severe airspace congestion problem,the aviation industry faced huge challenges,including the multi-plication of pressure on air traffic controllers,large flight delays,and increasingly serious flight conflicts,etc.Among them,the realization of efficient conflict detection and resolution tech-nology is the primary task of ensuring flight safety,especially for the complex and high-density airspace environment.This work is also of great significance for maintaining flight order,pre-venting aircraft collisions,alleviating air traffic pressure,and ensuring air traffic safety.Faced with huge air traffic pressure,scholars in the field of civil aviation have put forward many algo-rithms,including classical algorithms and AI algorithms,but they all have certain limitations.The main contents of this paper include:according to the state and action dimension of DDPG model can not meet the demand of real control scenarios,a short-term conflict resolu-tion model of single agent DDPG-net based on multi-dimension state and action is proposed.By building an environmental model with OpenScope,an open source air traffic control simula-tion platform,including Gym,an open source reinforcement learning environment library,and BADA aircraft performance models,a set of simulated air traffic control environment "At-cEnv”is designed.Through the interaction of the simulation system,the model is evaluated and analyzed by the related evaluation index.In the model,the uncoupling action of the agent includes three degrees of freedom,such as altitude,velocity and heading angle,and the state in-cludes 10 degrees of freedom,such as position information,minimum distance to other aircraft and velocity,etc.Firstly,we registered the simulation environment with Gym,and made functional modifi-cations to OpenScope,introducing the latest fixed point data in the approach area of Chengdu Shuangliu Airport,including each sector parameter,approach and departure route data,and approach area jurisdiction data,etc.We designed a simple approach area ATC control en-vironment.This environment includes independent approach and departure scenarios,fully considering the conflicts in various complex situations.Secondly,we design corresponding conflict scenarios for different airspace complexity,such as cross-conflict as well as head-to-head conflict on the flight path,head-to-head conflict and overtaking conflict during approach and departure,etc,constructing the complete conflict set.In order to reduce the complexity of the model,it is assumed that the minimum turning radius is not considered when the aircraft is turning.Finally,considering the continuity of the uncoupling action and the complexity of the agent’s state,this paper investigates the aircraft’s resolution strategy with the premise of ensuring flight safety.The interactive training task between the agent and other aircrafts is implemented through the constructed ATC simulation environment,the reward function of the conflict resolution model is designed,and the classical algorithm DDPG in DLR is used to learn the resolution strategy.The simulation experiment results show that the algorithm can search for a better resolution strategy for a variety of conflict environments,and the success rate of con-flict resolution is over 89%,which can be used as one of the reference schemes for controllers to avoid conflicts.The research work of this paper includes data processing,simulation environment con-struction,conflict scenario construction and DLR techniques.This paper provides a new mod-eling idea and solution to solve the air traffic conflict problem in civil aviation field.The effec-tiveness and superiority of the algorithm are proved through repeated validation of simulation experiments.