Researches On Intelligent Traffic Signal Control Based On Deep Reinforcement Learning

The rapid growth in car ownership has outstripped the capacity of existing infrastructure,causing massive traffic congestion.Traffic congestion is an increasingly serious problem,which has brought about negative effects on society.The increased travel time and accident rate bring a lot of economic losses,and the exhaust gas generated by traffic congestion aggravates the environmental pollution.Intersections in cities,where roads meet and where vehicles and pedestrians gather,are also the worst-hit areas of congestion.Traffic signals at intersections guide vehicles in all directions to pass orderly according to the set scheme,and the control strategy of traffic signals can directly affect traffic efficiency.Intelligent control of traffic signals is a hot issue in intelligent transportation system,which aims at coordinating the process of vehicles at intersections to optimize traffic state.Traffic data is more abundant and computing power is greater than ever,but the traffic signal control systems in use today still rely too much on simple information and conventional methods.In order to improve the efficiency of traffic signal control,it is necessary to collect and process traffic data in real time and adopt more effective control strategies.Firstly,a traffic signal control method based on deep reinforcement learning was proposed to solve the problem that the existing traffic signal control system could not adjust signal strategy according to real-time traffic conditions.In order to represent the traffic state more effectively,a road division strategy based on Fibonacci sequence is designed.The strategy divides roads at intersections into cells of corresponding lengths according to the rules,with the goal of reflecting the difference in importance of vehicles at different positions.The states of all the cells constitute the state of the intersection,which is represented by an array.An action is defined as a corresponding traffic phase,and the motion space contains four independent traffic phases.The reward is defined as a weighted linear combination of two traffic performance indicators.The state array is used as the input of the deep neural network,and the output is the Q value of the four actions,which is used to measure the value of the corresponding actions.The agent obtains experience through sample learning and improves decision making ability continuously.In this paper,the effect of traffic signal control is evaluated through Simulation of Urban Mobility(SUMO).Experimental results show that the proposed method can effectively improve the traffic efficiency of the intersection.Secondly,due to different levels of urban roads and other reasons,there will be a big difference in the traffic flow in different directions at some intersections under the condition of high vehicle density,which is called unconventional intersections.A problem may arise when reinforcement learning is applied at irregular intersections: the potential rewards of different actions vary greatly,so that the direction with more vehicles will obtain the right of way for several times in a row,while the direction with less vehicles will be kept in a waiting state for a long time.Aiming at this problem,a traffic signal control algorithm is proposed for unconventional intersections.A motion determination mechanism is introduced in the algorithm,which determines the choice of the same motion in succession according to the occupancy of the road in each direction,and balances the passage opportunities of vehicles in each direction.Road occupancy is also taken into account in the definition of incentives.Experimental results show that the traffic signal control algorithm at unconventional intersection can effectively control the waiting time of vehicles in all directions.