Research On Traffic Signal Control Method Based On Deep Reinforcement Learning

The control of traffic signals is always a hotspot in intelligent transportation systems research.The purpose is to enable traffic signals to coordinate traffic through specific algorithms and improve the efficiency of the traffic network.In order to coordinate traffic more timely and effectively,a novel traffic signal control algorithm based on deep reinforcement learning is proposed to realize coordinated control of traffic signals.The traffic signal control of single intersection is the foundation of the traffic network coordination control.The basic structure of the single intersection agent is first constructed.After integrating the discretization of the high-dimensional real-time traffic information at intersections with waiting time,queue length,delay time and phase information as states and making appropriate definitions of actions,rewards in the algorithm.Then,a deep distributional reinforcement learning algorithm based on prioritized experience replay with reward weight is proposed.Based on the deep Q network,the reward weight mechanism is introduced into the priority sampling to improve the utilization efficiency of the samples in the experience replay.The convergence of the algorithm is accelerated.By combining with double Q network and distributional reinforcement learning,the algorithm performance is further improved,and the algorithm is applied to the coordinated control of traffic signals.Finally,The results show that the deep distributional reinforcement learning algorithm based on prioritized experience replay with reward weight is more efficient with lower average delay,travel time of vehicles.The traffic signal control algorithm of single intersection is extended to the field of multiple intersections,and a coordinated control algorithm of traffic network signals based on deep reinforcement learning is proposed.Through model-based transfer learning,the traffic signal control model at a single intersection is transferred to agents at multiple intersections to speed up the convergence of the algorithm.In the process of policy learning,the coordination of each intersection is promoted through the transfer of value distribution in the neighborhood.The space discount factor is introduced and the value distribution information of the neighborhood decreases with the increase of the distance between the agents.The effectiveness of the proposed signal coordination algorithm at multi-intersection is verified by experiments,which improves the overall traffic efficiency.