Reinforcement Learning For Urban Traffic Signal Control

With the development of economy,the vehicle ownership continue to escalate while living standards improve.The massive growth of traffic is followed by the great increase in frequency and size of traffic congestion.Traffic jams has become a common problem existing in cities and its surrounding areas.Traffic signal control is a major method of adjusting the traffic flow in the network,whose control scheme determine whether the traffic is good or not and play an important role.So the optimization of traffic signal control is the most effective way to solve the problem of traffic congestion.Traffic flow is dynamic,random and uncertain.The traffic signal control problem itself is a sequential decision problem,so that one select:ion of a control scheme is only related to the current traffic condition.Because of this characteristic,reinforcement learning is rmore applicable to solve urban traffic signal control problem.Based on reinforcement learning,we focus on the optimization of urban traffic signal control problem.The main research work are as follows:Firstly,the application of reinforcement learning in urban traffic signal control problem is first introduced.Under the background of 'Ali intelligent transportation algorithm competition',reasonable simplification and hypothesis for complex conditions in the traffic control are made based on real road network and traffic flow data.An application model of Q learning is established to control traffic signal.Through 'offline training and online applying'method,the required result is calculated.Experiment result shows that Q learning algorithm is effective in traffic signal control problem.Secondly,Sarsa algorithm is combined with eligibility traces to study the same control problem.The effectiveness of Sarsa(?)in signal control is verified by simulation.Through large number of experiments,two methods in this thesis are compared.The result shows that Sarsa(A)has faster convergence speed than Q learning due to the introduction of eligibility traces.Besides,parameters for eligibility traces which is more suitable for the background of this thesis is found out and analyzed.Finally,the work of research in this thesis are summarized.Problems and deficiencies exist in this thesis are studied and prospect for future research is made.