The Research On Traffic Signal Timing Based On Risk-sensitive Q-learning

At present, the urban traffic problem has become the important influence factors restrict-ing the development of urban economy. How to solve the traffic congestion, guarantee traffic system is smooth and orderly operation has become a top priority in the government’s work.But, Limited and confined space, the demand of the economy and environment, traffic infrastructure expansion is impossible. At this time, the development of intelligent transportation to solve the traffic congestion has become the only way.Summary based on the research of intelligent transportation system both at home and abroad, this paper is sensitive to risk theory and the Q learning theory applied in traffic signal control optimization is studied. The main research contents of this dissertation are listed as follows:1. Research on based on risk-avoiding Q-learning online signal timing optimization modelMost of the existing signal timing models apply risk-neutral reinforcement learning model. The disadvantages of these models are instability and low robustness. Also computing time of these models is long. For solving these problems, the paper formulates an on-line risk avoidance reinforcement learning model. The queue length difference is the performance index. Then, through VISSIM-Excel VBA-Matlab simulation platform, we analyze the effect of risk avoidance parameter on signal timing and convergence. Also we compare the proposed model with risk-neural reinforcement learning model. The results show that the proposed model has quick convergence, better stability and almost the same performance. Lastly, we propose incremental risk avoidance reinforcement learning method is suitable to signal timing optimization, that is, risk avoidance parameters should increase in a small step.2. Research on based on risk-seeking Q-learning online signal timing optimization modelConsidering the traffic randomness, uncertainty, impossible to transport planners expect. So sometimes we must fully consider the situation that may arise, even if there may be a higher risk. This paper seeks to further build the based on risk-seeking Q-learning online signal timing optimization model. The queue length difference is the performance index. In order to better contrast with the based on risk-avoiding Q-learning online signal timing optimization model, all sorts of model establishment conditions are consistent. Then, through VISSIM-Excel VBA-Matlab simulation platform, we analyze the effect of risk avoidance parameter on signal timing and convergence. Also we compare the proposed model with risk-neural reinforcement learning model. The results show that the proposed model has quick convergence. Lastly, we compare the proposed model with risk-avoiding reinforcement learning model. The results show that the exioring of model is wider,and the number of behavior training is larger. but the performance of the timing plan is not very stable,when good or bad.