Multi-Agent Traffic Signal Control Algorithm Based On Mean Field In Large-Scale Road Network

Traffic congestion has become a challenging issue for sustainable development in major cities.Exploring more advanced and intelligent traffic signal control methods has become a popular trend due to the current road resources and infrastructure conditions.With the rapid development of multi-agent reinforcement learning technology,traffic signal control methods have also made significant progress.However,existing research on multi-agent traffic signal control methods usually focuses on controlling traditional traffic parameters such as traffic flow and lane occupancy rate,while ignoring the problem of dimensionality disaster caused by the rapid growth of joint action space dimensions with the increase of intelligent agent quantity,making it difficult to adapt to large-scale road networks.Therefore,in order to improve the scalability and reliability of multi-agent reinforcement learning algorithms in large-scale urban road networks,this paper studies a multi-agent traffic signal control algorithm based on mean field theory in large-scale urban road networks,which can alleviate traffic congestion and improve road network efficiency.Firstly,this article summarizes and introduces the basic concepts of traffic signal control and the relevant theories of multi-agent reinforcement learning and mean-field approximation.Secondly,with regards to the traffic signal control method for small-scale urban road networks,this article proposes a weighted mean field Q-learning control algorithm(WMFQ)based on a cooperative mechanism.This method shares state information and redistributes reward information among intelligent agents at adjacent intersections,encouraging them to learn cooperative action strategies.The method also assigns weighted attention coefficients to intelligent agent action information at different distances,allowing for more accurate and effective dimensionality reduction of joint action information.Simulation experiments in a basic road network environment demonstrate the effectiveness of the proposed method.This article proposes a dynamic control algorithm for phase duration of traffic lights in large-scale urban road networks,called the Mean Field Double Q-learning Control Algorithm with Dynamic Timing Control(MFDQL-DTC).This method uses the average effect of the action information of other agents to simulate the action information of agents near the intersection,which can greatly reduce the dimension of the joint action space of agents in the road network,thereby improving the training scalability of this signal control algorithm in large-scale road network scenarios.In order to reduce congestion at intersections,the dynamic timing control module combines a recursive neural network(RNN)with an improved Webster timing formula to predict traffic flow and adaptively control the duration of the phase.To verify the traffic control performance of MFDQL-DTC,experiments were conducted in different scenarios,including a synthetic four-way intersection network and a large urban road network in Chengdu with 315 intersections.Compared with the current state-of-the-art multi-agent reinforcement learning methods,MFDQLDTC shows great advantages in scalability and significantly improves traffic performance in terms of average parking times,average waiting times,and average road speeds.