Research On State Estimation And Control Of Urban Traffic Systems Using A Dynamic Data Driven Approach

With the continuous advancement of urbanization and the improvement of living standards,the population and traffic demand in the urban are growing rapidly.The imbalance between traffic supply and demand has caused many problems such as traffic congestions,traffic accidents and energy waste.In terms of alleviating traffic congestions,the method of simply restricting traffic demand or constructing traffic infrastructure is faced with tremendous pressure and poor potential.The Intelligent Traffic System(ITS)employs scientific means to balance traffic supply and demand,plays an irreplaceable role in improving the efficiency of traffic system,ensuring traffic safety,and reducing exhaust emissions.It is the future direction of urban traffic system.Real-time,accurate monitoring of traffic system is the basis of intelligent management and control of traffic system.Traffic state estimation refers to the process of the inference of traffic state variable using the partial and noisy traffic data,which is an indispensable part of the management and control of traffic system.Intersections are the main source of traffic congestions and delays.Signal control is the most common way of traffic control.Improving the efficiency of traffic signals and maximizing the capacity of intersections are the key to improving the performance of traffic system.Utilizing the traffic data available in current traffic system,we employs data assimilation and reinforcement learning techniques to conduct traffic state estimation and traffic signal control,respectively in this paper.They are theoretically significant and of application value.The main contributions are:(1)A mesoscopic traffic data assimilation framework for vehicle density estimation on urban traffic networks is proposed.Considering the characteristics of urban traffic and the real-time requirements of vehicle density estimation,a mesoscopic data assimilation framework,which can be applied in relatively large urban traffic networks,is proposed.This framework uses the platoon-based model(PBM)to describe the dynamics of urban traffic,which balances the details and computation cost of urban traffic modeling.The measurement data is the pass time of vehicles which contains much useful information,and it is assumed to be noisy where both missing detection and false detection exist.Since the mesoscopic traffic model is nonlinear,and the measurement data is non-Gaussian noisy,particle filters are used to assimilating them to estimate vehicle densities.(2)Considering the two types of traffic data available in current traffic system,we propose the reinforcement learning-based traffic signal control system(RL-TSC)using event data and using vehicle densities on segments,respectively.Among them,we propose to uses the high-resolution event data to define traffic state and utilize deep neural network to extracts features automatically in order to make full use of the measured data.When defining traffic state with event data,we propose a discrete time traffic state encoding method to encodes the event data into state vectors.Thus,realizing an efficient deep reinforcement learning-based traffic signal controller that uses measured data completely.Regarding the use of estimated data,the result of vehicle density estimation is used to define the state.As a result,the RL-TSC system using vehicle densities on segments is obtained.(3)The influence of some factors such as the way of organizing traffic state using traffic data,the phase extension interval in RL-TSC systems is studied quantitatively.In the process of designing the RL-TSC system using event data and using vehicle densities,the difference between state definitions using the lane data and the phase-based data,the effect of the restriction of maximum green time,and the choice of different extension intervals are studied.In addition,Q-learning and 3DQN(Double Dueling Deep Q Network)are compared in the RL-TSC system using vehicle densities.Conclusions about the design of RL-TSC systems are obtained.(4)The influence of data quality on both the vehicle density estimation and the deep reinforcement learning-base traffic signal control is analyzed systematically.In the data assimilation framework for vehicle density estimation,the effect of the missing detection and false detection are analyzed respectively.The results show that the proposed framework is robust.In the traffic signal control algorithm,the effects of the noise in the state data and reward data are studied in the RL-TSC system using event data and vehicle densities,respectively.The experimental results show that both of the two proposed control algorithms are insensitive to these noises.