Study On Area Coordination Control Of Urban Road Network Based On Deep Q-Learning

In recent years,a series of problems caused by urban traffic congestion have become more and more serious in China.The development of intelligent traffic control technology has effectively solved some of the traffic problems.Area coordination signal control can significantly increase the traffic efficiency of road network.Therefore,it is the key research direction of intelligent traffic control technology.In this work,the area coordination control is studied to reduce the average delay of road network.The main work of this dissertation is as follows:1.Area average delay modelingBy analyzing the correlations among intersections and traffic flow characteristics within the road network,the area average delay is divided into external entrance delay and internal entrance delay.According to the integrality and discreteness of the traffic flow,the delay of the internal entrance lane within road network is divided into uniform delay and random delay.Thus we establish the area average delay model.Finally,the accuracy of the model was validated by the simulation experiments.2.Single intersection optimization control studyThe control of single intersection is the foundation of area coordination control.We study the optimal control algorithm of single intersection based on Deep Q-learning theory.In order to improve the performances of Deep Q-learning when dealing with the area traffic control which is Partially Observable Markov process,Deep Recurrent Q-learning is introduced.The vehicle at the entrance of the intersection is represented as the position,velocity and acceleration matrixs,which makes the state required for Q-learning more in accord with the actual state.By introducing the recurrent neural network LSTM into the deep Q network and using the time axis information obtained by the LSTM network,it enables the deep Q network extract the dyna,mic information of the vehicles.The simulation experiments show that the Deep Recurrent Q-learning algorithm effectively reduces the average delay of vehicles at the intersection.3.Area coordination optimization controlThe Deep Recurrent Q-learning algorithm at a single intersection was applied extendedly to the local road network.The joint-action method of multi-agent reinforcement learning was adopted to realize coordination and optimization control for the road network.In order to speed up this algorithm,the distributed Max-plus algorithm is used to obtain the optimal joint action.In addition,the transfer learning is used to perform Q-learning initialization process for multiple intersections to reduce the training time.Finally,an actual case analysis shows that the proposed area coordination optimization algorithm effectively reduces the area average delay and improves the road network's capacity.