Research On Intersection Signal Control Model And Optimization Method Based On Deep Learning

Urban traffic network is an important part of the city,which provides a strong guarantee for residents’ lives.With the development of national economy and urbanization,the number of motor vehicles in China has increased sharply which is accompanied by the intensification of urban congestion,and the problems of traffic safety and pollution are becoming increasingly serious.Traffic congestion and other hazards arising therefrom have become one of the ”urban diseases” faced by most cities.Urban traffic problem has become a bottleneck restricting economic and urban sustainable development.It is the most economical and feasible scheme to alleviate congestion by improving traffic management level,making full use of existing road resources,using advanced control technology and coordination methods,and improving road network operation efficiency.Installing waiting areas at intersections is a common practice used in Chinese cities to improve traffic capacity,alleviate traffic congestion and curb oversaturation.However,the research on waiting areas still needs to be further improved.In addition,the previous model-based optimization methods may have reached the”theoretical limit”.On the contrary,the methods and technologies based on machine learning have been successful in many fields.Therefore,in-depth research on traffic signal control theory and method based on reinforcement learning and deep reinforcement methods a prerequisite for improving the level of traffic management,improving traffic efficiency,alleviating traffic congestion,and improving residents’ travel satisfaction.Based on the basic theory and method of traffic control,this dissertation analyzes the setting of waiting area,vehicle operation characteristics,signal optimization of oversaturated intersection,and carries out an in-depth study on the application of reinforcement learning method in the field of signal optimization.Firstly,the dissertation analyzes and reviews the previous studies at home and abroad,and determines the research ideas,research methods and technical route based on China’s national conditions.Secondly,on the basis of analyzing the operation characteristics of oversaturated traffic flow,the delay model under the oversaturated state is established by using graphic analysis method,and the signal cycle optimization model applied to the oversaturated intersection is deduced with the optimization objective of the minimum average delay.Thirdly,the traffic conflict theory is used to analyze the setting method of waiting zone and vehicle starting characteristics widely used in large and medium-sized cities in China.The capacity model of intersections with waiting zone is established,and the signal optimization strategies and methods applied to intersections with waiting zone are put forward.Fourthly,based on the in-depth analysis of reinforcement learning and deep reinforcement learning methods,the queue length is selected as the state space,and the signal optimization problem of oversaturated intersections is transformed into Markov decision process.Then,a signal optimization method of oversaturated intersections based on reinforcement learning method is proposed,and a training-testing platform is built to prove the effectiveness of the proposed method.Lastly,according to the arterial intersection coordination control theory and method,the state-action relationship is determined,and a distributed arterial intersection coordination method similar to the k armed bandit optimization problem is proposed.The state is gradient descent by using neural network,and an urban trunk line with five intersections is tested by using the built training-testing platform.The results show that the method proposed is better than the mathematical analysis method.The followings are the main conclusions of this dissertation.(1)By drawing into the continuous delay modeling method and describing the arrival-departure law of traffic flow,a delay model of oversaturated intersection based on graphical analysis method is established.Then a cycle optimization model is derived with the goal of delay minimization.Empirical analysis shows that the model is effective and the case shows that the model can reduce the vehicle travel delay by 4.22 % and the stops delay by 7.58 %.(2)Based on traffic conflict method and combined with the behavior characteristics of drivers,a design method of intersection with waiting areas is proposed.Using the comparative analysis method,the impact of installing waiting areas on the vehicle start-up process and the clearance process is quantitatively modeled.Then,the capacity model of intersection with waiting areas is constructed by leading into the modeling method of HCM.For the oversaturated traffic state,the optimization strategy is proposed to maximiz traffic capacity.On the contrary,for the undersaturated traffic state,two optimization methods are proposed to reduce intersection delays.The simulation results show that and the two optimization methods for unsaturated intersections can reduce the the average delay by 5 %-11 % and 10 %-14 % respectively.(3)Referring to the modeling and analysis method of discrete delay model,the operation mechanism of how the Bang-bang control strategy can reduce the intersection delay is revealed,and the reward function based on the average delay model is established.By characterizing the optimization process of reinforcement learning algorithm and Markov decision process(MDP),taking the intersection queue length as the state space and the green-signal ratio as the action space,the oversaturated intersection signal optimization problem is transformed into MDP.On this basis,the two signal optimization methods of oversaturated intersection based on Q-Learning and Double DQN are proposed.In order to avoid vehicle overflow,constraint conditions and penalty factors are designed.A training-testing platform is built using Python programming language and Tensor Flow toolkit to verify the cases of two-phase and multi-phase schemes.Simulation results show that the two optimization methods can effectively reduce the average delay of oversaturated intersections.Without queuing constraints,the two methods can achieve the same average delay as the two-stage Bang-bang control method,and the maximum queue length is reduced by 1.7%.Correspondingly,the maximum queue length is reduced by about 13% with queuing constraints.Moreover,the proposed deep reinforcement learning optimization method has better adaptability than reinforcement learning optimization method.(4)The key parameters such as intersection distance,system speed,cycle length and upstream intersection offset are set as system state,and the offset of this intersection is set as action by analyzing the optimization mechanism of coordinated control of arterial intersections.Since the interaction process of state and action is difficult to convert into MDP,a distributed learning algorithm similar to k-arm bandit optimization method is proposed.Due to the upstream offset is creatively drawn into the state space,and the weighted average stops rate of the upstream and downstream vehicles at the intersection is used as the reward function,the twoway distributed coordination optimization is realized.A training-testing platform is built using a trunk line with 5 intersections in Zhengzhou City as an empirical case.The test results show that the distributed deep reinforcement learning optimization method is better than the mathematical analysis method,the overall stops rate is reduced by 3 %-37 %.moreover,the distributed coordination method has better adaptability and scalability than other methods.