Research On The New Train Control System Interval Control Method Based On Reinforcement Learning

As the core of public transportation in large and medium-sized cities in China,urban rail is the first traveling choice for urban commuters.In recent years,with the rapid development of urban rail,the passenger volume of urban rail has increased sharply.Under the condition that the urban rail operation network has been formed,using advanced technology to further shorten the train tracking interval,improving the traffic density,and fully releasing the line capacity are important means to meet passenger demand.The existing urban rail uses Communication Based Train Control System(CBTC)to realize the moving block technology of "concrete hard wall".This method only considers the static position of the train ahead,which limits the release of line capacity.The VehicleBased Train Control System(VBTC)realizes the ideal moving block technology of "concrete soft wall",which considering more dynamic information such as the position,speed,and acceleration of the train ahead,and the train operation interval can be further reduced and the system capacity can be improved.The thesis focuses on the train interval control method,based on algorithms such as Reinforcement Learning and Fuzzy Control.The main work of this paper is as follow:(1)Research on the system structures and the main function of VBTC and analyze the differences between VBTC and CBTC in system structures,communication mode and train control.Establish the train dynamics model according to the characteristics of urban rail train,and analyze the ATO control performance to provide reference for the formulation of train interval control index.(2)Building the model of the train interval control system based on the VBTC "concrete soft wall" technology,and use Monte Carlo Tree Search algorithm(MCTS)to research the control problem of train interval.Using MCTS algorithm to generate train action sequences,and to realize the prediction of the future track of trains;using Dynamic Programming algorithm(DP)to process the action sequences to obtain a far-sighted train control strategy;Judging the safety of the train operation and adopt the train control strategy.Establish the train interval control index,and evaluat the train interval control algorithm.(3)Introducing Fuzzy Control to improve the problem that MCTS algorithm has low computing efficiency when the current state of the train is far from the control target.Designing Fuzzy Control algorithm by using a 2-D Mamdani fuzzy controller and generating fuzzy rule table based on the calculation data of MCTS algorithm.Forming an integrated intelligent train interval control method with reinforcement learning and fuzzy control as the core.(4)The simulation compared three different control strategy generation methods,and decided to use DP algorithm to generate MCTS train control strategy.The simulation of different tree depths of MCTS algorithm shows that the algorithm has the best efficiency and control precision when the tree depth is 7.Firstly,in order to verify the actual performance of the control algorithm,multiple running scenarios of train interval tracking are simulated.Then,based on the actual running data of a train,the tracking interval is simulated by using the control algorithm.Finally,the proposed control algorithm and the Fuzzy Control algorithm are used to compare the simulation of the train tracking interval in the same tracking scene.The thesis research results show that the new interval control method based on Reinforcement Learning in the VBTC is suitable for train interval control.Compared with the traditional control algorithm,it has the characteristics of fast response speed,short adjustment time,and high control precision in solving the control problems such as partially observable,high dimensional,and complex nonlinear control problems.Figures 42,Tables 8,References 80...