Research On Urban Traffic Control Algorithm Based On Deep Reinforcement Learning

With the continuous increase of China's urbanization rate and per capita motor vehicle population,many cities are facing the dilemma of traffic congestion.In recent years,research in areas such as big data,artificial intelligence and deep reinforcement learning has made great progress.To profit the advantage of these technologies,urban intelligent transportation combining these new AI technologies has also become a research hotspot.In addition,the gradual improvement of the informatization of urban transportation provides data-level protection for the realization of urban intelligent transportation.This thesis has conducted in-depth research on the optimization of urban traffic signals to make traffic smooth and fast.The main work is divided into the following two parts.At the engineering level,this thesis designs and implements two traffic signal control simulation platforms based on Vissim and SUMO simulation software respectively.These two platforms are designed and optimized for reinforcement learning algorithms,and re-developed on the basis of professional simulation software,which improves the credibility of simulation experiments.At the algorithm level,this thesis proposes a traffic control algorithm based on a deep Q network algorithm.It uses matrix representation method to extract traffic state information,and performs one-hot encoding of the joint signal phase of a small road network as the action of agent.The feasibility of deep reinforcement learning algorithms for traffic signal control optimization on small-scale networks is analyzed,and the advantages and limitations of DQN algorithms in this field are analyzed through multiple experiments.At present,signal control methods for regional traffic networks always use distributed control solution,for this framework,the coordination of strategies relies on manual adjustment,while the centralized traffic control is limited by the action space,and it is impossible to conduct efficient exploration in the state action space to learn good strategies.To solve this problem,this thesis proposes a hierarchical regional control framework: the upper-level controller observes the global information and coordinates the control strategies of intersections.Each lower-level controller only focuses on traffic signal optimization at each local intersection.This structure combines the advantages of distributed control and centralized control,reduces the dimension of the action space,and can effectively coordinate the traffic flow of the regional traffic network.With the advantages of hierarchical structure,this method can adapt to larger regional traffic networks and performes robustly during algorithm training.In addition,this article shows a large number of simulation experiments.Through the analysis and summary of the experimental process and results,this article proves that this method achieves significant performance advantages over traditional methods in typical scenarios.