DQN-based Signal Optimization Model And Applicability Study At Isolated Intersection

Traditional intersection signal control methods are based on data-driven models,including fixed timing signal control methods and inductive control methods,which optimize signals within a cycle and have limitations in coping with complex traffic demand changes.Recently,the application of artificial intelligence technology to traffic signal control has become a research hotspot,and numerous deep reinforcement learning methods(DRL)have been applied to urban traffic signal control problems.The related research results conclude through experiments that DRL methods are more flexible than traditional adaptive signal control,and can meet the current complex traffic time-varying demands,and have better development prospects.There are still some problems to be solved in the implementation of machine learning related methods to urban road networks in real traffic systems.In this paper,we will study a set of training methods with the practical situation,which will help the models to be deployed in the actual urban road networks and have stronger applicability and generalization.The specific research problems include two aspects,on the one hand,how to construct the main parameters of the model under different constraints so that the model can achieve the best performance;on the other hand,how to solve the problem of time and energy consumption caused by repeatedly training multiple models for each intersection in a city,so that the model can be used in new traffic demand scenarios and new intersection scenarios without repeated training..Therefore,the main research work of this paper is as follows:(1)Most of the current studies mainly focus on algorithms to optimize the DQN signal control model without an in-depth discussion on the parameter settings of the model.In order to provide a more reasonable and efficient parameter setting reference for the model to give the best performance effect,this study further investigates the state representation and action selection parameters of the model.First,based on the application development of existing traffic sensor technology,several different traffic state observation indicators are proposed under the constraints of realistic traffic detection technology,and these traffic indicators and their combinations are used to construct different traffic state observation matrices as the state representation of the model,and the performance of the model under different traffic state observation matrices is compared through experiments,so as to draw the experimental conclusion that adopting more advanced traffic The performance of the DQN signal optimization model is better by adopting more advanced traffic sensor technology as the data source of the model state representation.Secondly,the design of signal control schemes in real situations is combined with the classification of possible signal decision actions,and the performance of the model under different signal decision actions is compared through experiments,which finally leads to the conclusion that the larger the range of solution space of signal decision actions,the better its flexibility and correspondingly the better performance of the model.Finally,based on the above experimental results,the most efficient state representation parameters and decision action parameters are selected for the model.(2)In many studies,when the signal optimization model is trained under a predefined traffic demand scenario,the model can only gain experience through the limited knowledge in the set scenario,which results in a very limited set of control strategies,and this set of control strategies is often unable to cope with unexplored traffic demand scenarios.In order to solve the problem of limited learning experience of the model,this study aims to obtain a set of control strategies with transferability and generalization,and to design more comprehensive traffic demand scenarios to enhance the applicability of the model in unexplored spatial domains.Several traffic scenarios containing different size and number of traffic states are designed to analyze the relationship between the traffic demand and the actual traffic operation state,train the DQN model under the designed traffic demand scenarios respectively,and then apply the model to the new traffic demand scenarios without additional training to compare and analyze the performance of the model,and compare it with the traditional signal control methods and the currently popular SAC methods to verify the feasibility of this training method compared with traditional signal control methods and the currently popular SAC methods.Therefore,this study focuses on the main parameters of the model to compare and select the state representation and decision actions that make the model perform best;then a training method for traffic demand scenarios is proposed to make the model applicable in unexplored areas as well.