Adaptive Traffic Signal Control With Multi-agent Reinforcement Learning

With the developement of social economy,urban traffic congestion problem grows quickly,which propose higher demand for traffic signal control.By real-time decision making,intelligent traffic signal control methods have the potential to solve traffic con-gestion problem.Hovever,traditional adaptive signal control methods heavily depend on a well-defined traffic model,which makes them cannot fit time-variant traffic well.Reinforcement learning is a technique to learn the optimal control policy by the interac-tions between agents and the environment.It could conduct optimal decision according to real-time traffic flow situation to import traffic by applying reinforcement learning technique into traffic signal control tasks.Nevertheless,the control problem in traffic network is harder to be solved by straightforwardly using reinforcement leraning,as traffic network’s more complex traffic situation,more control objects and larger con-trol region.By introducing multi-agent reinforcement learning into this prolem,the coordinated joint-policy could be learned effectiveness.In this paper,we first describe the traffic signal control task in the form of an opti-mazition problem,and model the problem with Markov decision process and stochastic game.Besides,we design the scenario of traffic network signal control task.Secondly,we propose two decentralized multi-agent reinforcement learning algorithm based on communication mechanism,which are information exchange DQN and depthwise con-volution for multi-agent communication with enhanced mean-field approximation,re-spectively.Then the simulation experiments are conducted in the scenario designed above.Both the two algorithms have better performance in the metrics of average queue length and average delay time,which prove that the algorithms could learn more coor-dinational joint-policy and have better ability to reduce traffic congestion.