Research On Urban Traffic Signal Control Via Deep Reinforcement Learning

Since the beginning of the 21 st century,Chinese urbanization and motorization have been accelerating,and the number of vehicles has maintained a rapid growth,which has led to problems such as traffic congestion,energy crisis,and environmental pollution.As traffic throats,intersections are the main areas where traffic congestion occurs.Urban traffic signal control,as a method of collaborative optimization of time and space resources at intersections,is important for intelligent transportation systems.Reinforcement learning,an exploratory machine learning method,is an effective means to improve traffic capacity at intersections.The traffic signal control method based on deep reinforcement learning studied in this paper is as follows:1.We proposed an improved algorithm based on deep Q-network(DQN)by using a multi-process method.The training neural network and the simulation environment are performed in parallel,and multiple simulation environments are constructed at the same time.These environments will generate transitions in parallel.Compared with the traditional DQN algorithm,this method will speed up the amount of data generated for the replay memory and improve the training speed of the neural network.2.the improved DQN algorithm was applied to the single-point signal control model.In reinforcement learning,we proposed the representation methods of state,action,and reward.LSTM(long short term memory)network was used to fit the state and Q-value function.Compared with the fixed signal timing,the results showed that the traffic capacity at the intersection of the algorithm is increased by 14.28% under certain traffic conditions.3.Combined the single-point signal control algorithm,the multi-agent deep reinforcement learning algorithm was applied to the area signal control.On the basis of the single-point control method,the representation methods of state,action,and reward are further improved and applied to the area signal control.At the same time,a controller???????.??is proposed to solve the problem of coordinated signalcontrol at different intersections.Each controller independently controls the changes of the corresponding intersection's green or yellow signals due to the duration of each signal may be different.The results showed that comparing with the control of fixedsignal timing,the multi-agent signal control method performs better in capacity,vehicle delay and average queue length at the intersection.