Single-Agent Reinforcement Learning For Intelligent Traffic Light Control

Modern cities have grown larger and larger in recent years.In the meantime,traffic became worse as well.Existing traffic light control methods are mostly based on artificial analysis,human experts study the traffic data and give a static traffic signal timing plan.The plan may be efficient in the beginning but cannot dynamically optimize the traffic flow in the long run.Fundamentally,the problem of analyzing traffic flow data and optimizing traffic lights corresponding to interacting with the environment and making decisions,respectively.And reinforcement learning is designed to solve this problem.The more intelligent methods using reinforcement learning usually focus on single intersection,which is effective but lack consideration of the influence of upstream and downstream flows.In this thesis,we propose a single-agent deep reinforcement learning method to control multiple traffic lights and apply it in the Simulation of Urban MObility(SUMO).The agent based on Convolutional Neural Networks(CNN)takes every intersection’s state as input and select the best action to reduce every cars’ waiting time.Considering the influence of different intersections,we make every CNN in different intersections share parameters and connect together in the end to interact with each other.We apply several stateof-art reinforcement learning methods such as dueling network and double Q-learning to improve performance.We compare two kinds of state inputs: grid expression and image.Simulation results show our model can control multiple traffic lights effectively.An automatic traffic light control system is also designed to conduct simulating and data training under instructions to the graphical interface commands.