An Intelligent Traffic Signal Control System Based On Q-Learning Algorithm

With the rapid development of economy and the progress of science and technology,there are billions of vehicles on the roads,which bring great convenience to people’s travel.However,the increasing number of cars leads to a lot of inevitable environmental and social problems,e.g.,noise pollution,air pollution caused by automobile exhaust,traffic congestion and so on.These problems not only bring annoyance to people’s life,but also cause great economic loss.Especially,traffic congestion,has been an inevitable problem with the rapid growth of the number of vehicles.In order to solve the problem of traffic congestion,researchers introduced artificial intelligence into the traffic control system,named intelligent traffic systems.The artificial intelligent algorithm was used to optimize the traffic system,so as to make the reasonable allocation of traffic resources.In traffic control systems,signal control system is the most important one.Traffic signal system can reasonably allocate traffic resources in the traffic system to relieve the traffic congestion.Nowadays,the fixed time signal control strategy is still dominant in the practical signal control system.However,the fixed time signal control strategy can no longer reasonably use traffic resources with the changeable traffic mode,resulting in the waste of traffic resources.This paper presents an intelligent traffic signal control system based on the Q-Learning algorithm.In this system,digital infochemicals(DIs)is introduced as the carrier of traffic information on the lane.Assuming that the vehicle leaves DIs in the lane,the system formulates the traffic light duration strategy for the next cycle according to the amount of the DIs in the lane.Since DIs will be retained in the lane,they can not only retain real-time traffic information,but also preserve historical traffic information.Therefore,when allocating the time of traffic lights in the next cycle,real-time traffic information is not only considered,but also historical traffic information.The signal control system can control the influence of historical traffic information on the duration of the next traffic light cycle by changing evaporation rate of DIs.Q-Learning algorithm is applied to dynamically change the evaporation rate of the DIs according to the real-time traffic environment to reduce traffic congestion.Firstly,we analyze the influence of evaporation rate and cell size on the performance of the traffic signal control system in different traffic demand mode.According to the result of analysis,we define a residual queuing length as the state space of Q-Learning algorithm,and evaporation rate as the action space.Also,we define two kinds of reward functions for Q-Learning algorithm,namely,queuing length and waiting time.Finally,the intelligent traffic signal control system based on Q-Learning algorithm is compared with the fixed time traffic signal control system.And the effects of two kinds of reward functions on the performance of intelligent traffic signal control system were analyzed.The results show that the performance of the intelligent traffic signal control system based on Q-Learning algorithm under the two kinds of reward functions is better than the fixed time traffic signal control system,that is-a signal control system with a reward function of queuing length and waiting time has a better performance on reducing the average queuing length,and on reducing the average waiting time,respectively.