Regional Signal Cooperative Control Algorithem Based On Deep Reinforcement Learning

With the rapid development of intelligent transportation,the number of cars continues to grow,and the problem of urban traffic congestion is becoming increasingly prominent,which seriously affects people’s travel experience.As an important part of unclogging traffic,traffic signal control plays a crucial role in improving the efficiency of intersection traffic and improving congestion.However,traffic signals in the road network need to be coordinated to effectively dredge the traffic flow,and the large-scale road network in the city faces the challenge of complex calculations when coordinating as a whole,so this paper designs an overall scheme for the cooperative control of traffic signals in the urban large-scale road network,including sub-district division of the traffic network and cooperative control of regional signals,to improve the traffic efficiency of the overall road network.For the sub-division of traffic road network,this paper proposes a rapid sub-area division algorithm based on Macroscopic Fundamental Diagram(MFD).Firstly,according to the MFD theory,the correlation degree between intersections and regional quality are defined as the basis for sub-region division,and the homogeneity and aggregation of the region are evaluated at the micro and macro levels,respectively.Then,a heuristic Holonfication-based rapid sub-region division algorithm is introduced,which can divide the large-scale urban road network into multiple traffic sub-zones with uniform traffic flow distribution through lightweight calculation.Finally,through experiments,the sub-districts of the real road network in Chengdu are divided,and the MFD is introduced to test the homogeneity of the sub-district division results,the mean absolute percentage error is reduced by about 10%.For regional signal cooperative control,a regional signal cooperative control algorithm based on hierarchical deep reinforcement learning architecture is designed.Through the architecture of double layer agent,the control of the signal scheme is split into two interrelated tasks:the selection of the passage phase and the determination of the passage duration.For basic agents,a long short-term memory network(LSTM)is introduced to model the temporal correlation of traffic flow,and a graph attention mechanism(GAT)is introduced to model the spatial correlation of traffic flow,as to achieve multi-agent cooperation and improve the overall traffic efficiency of the region,we design the global reward of the region.For higher-level agents,the corresponding state information is obtained according to the passage phase output by the grassroots agent,and output the appropriate passage time.Finally,the experiment is carried out by the traffic simulation software Simulation of Urban Mobility(SUMO).Compared with other reinforcement learning algorithms,the algorithm proposed in this paper has a significant improvement effect on various traffic performance indicators,in which the average vehicle transit time is reduced by more than 20%.Finally,the research content of the two parts is combined into a system scheme that can be applied to the cooperative control of traffic signals in large-scale urban road networks.Under the real road network and traffic flow environment in Chengdu,the scene simulation is carried out through SUMO,which proves that the scheme can effectively clear the road network and reduce congestion.