The Research For Adaptive Urban Traffic Signal Control Method Based On Reinforcement Learning

The urban road has been building and expanding constantly, the investment of infrastructure has been growing, but urban traffic jams is getting worse. This situation mainly due to the system of urban traffic signal control (TSC) cannot achieve optimal control and management for traffic flow fully. Therefore, how to design TSC system through the optimal of traffic signal control, ensure safe and smooth transportation, and increase road efficiency are the key to solve the traffic congestion problem.In this thesis, Single-Agent control systems based on Q-learning algorithm, Multi-Agent systems based on distributed Q-learning and Green Light District open source systems simulation platform were selected for TSC system optimization, the following main works including:(1) The TSC system with Agent architecture framework of single intersection and well-shaped region were designed to realize the simulation of urban traffic control. For the single intersection, real-time traffic data in different directions was achieved by an intelligent Agent. After fuzzy logic of the traffic flow data, enter the intersection Q-learning designer for finding the optimal control strategy. For area traffic control, distributed Q-learning algorithm was combined with MAS, and adjacent intersections Agent coordination model was given to realize sharing information among the adjacent intersections.(2) Solved the key issues on Q-learning and distributed Q-learning, such as the state set S of traffic environment, policy set A, and reward function R. For the selection of state space, fuzzy logic was used to calculate the queue length; policy set A includes increases green light time, preserve the green light time, reduces green light time; the reward function R regards queue length as criterion for the purpose of minimum vehicle queue length.(3) To solve the problem of signal coordinate control in the areas, the application of distributed Q-learning algorithm to urban TSC systems was implemented. The combination of distributed Q-learning algorithm and MAS makes urban TSC system optimal controlled. Metropolitan area network can be abstracted as a distributed multi-Agent network. And Multi Agent modeling framework on distributed Q-learning was established. The detailed steps of distributed Q-learning algorithm were given. Finally, this thesis analyzed the algorithm performance of intersection urban TSC optimization on Q-learning and areas urban TSC optimization on distributed Q-learning. The simulations on random time, fixed time, longest-queue, traffic controller 1 (TCI), ACGJ-1 and Q-learning algorithm were tested in the GLD, and the experimental results show that Q-learning and distributed Q-learning are outperform other algorithms in the urban TSC system.