The Research Of Traffic Congestion Control Method Based On Reinforcement Learning

Due to the sustainable growth of car ownership,traffic congestion has become a worldwide public issue in urban development.Simple infrastructure can ease traffic congestion in a certain extent,but it may be restricted by cost,land,time,etc.With the development of science and technology,collecting information about traffic environment becomes more and more accurate and comprehensive.At the same time,due to the continuous research and development of the artificial intelligence algorithm,how to alleviate urban traffic congestion intelligently has become a hot topic of the research around the world.Reinforcement learning is frequently used to get the traffic control strategies and it can achieve good control effect.But in multi-intersection traffic control,with the increase of intersections number,the system state number increased exponentially,which can result in the difficulty of system calculation.Hierarchical control is one of the typical methods to solve the problem of huge system state.The general layered structure consists of two layers,the first layer is the intersection control layer,the second layer is the regional control layer.The region can be got by the merger of intersection and the rule of merger affects the outcome of regional control.Therefore how to divide region is a question which must be considered in the hierarchical control.In multi-intersection traffic control,the intersection are connected.When a single intersection is optimized,its adjacent intersections will be affected.Therefore,the optimization of intersection must be considered from a global perspective and the intersection control action can be obtained in the case of the global optimization.Collaboration control between the intersection turn into how to evaluate the global optimal solution under the condition of considering intersection influence each other.To solve the above two problems,this paper puts forward a improved hierarchical control method which mainly reflected in two aspects as below:First of all,in a layered combination control,the region of regional control layer need to be formed by the junction of intersection.The traditional partition is mainly on the basis of distance,which can merge intersection of a certain range into one region,the partition does not take into account the real time traffic change and is not conducive to regional control.In this paper,a dynamic road network partitioning method based on improved spectral clustering algorithm is proposed.This method can merge intersection of high similarity into one region according to distance,traffic flow and signal period.What's more,it can adjust region according to the change of traffic flow and make a better preparation for region control.Then,the problem of solving global optimal will be turned into the problem of getting optimal solution in coordination graph.Coordination graph is a kind of undirected graph structure which demonstrates relationship among nodes.We can get crossroads global optimal solution by calculating the global optimal solution of coordination graph.Because of the large calculation of coordination graph,this paper puts forward the variable elimination method based on training and pruning to reduce the calculated amount.Through simulation experiment compared general hierarchical control method based fixed partition,the proposed method in this paper can effectively gain good result when the number of intersection and the traffic flow are big.