A Multi-Train Cooperative Formation Method Based On Reinforcement Learning

The Communication-based Train Control(CBTC)technology has been widely applied in urban rail transit systems in order to increase operation efficiency.The departing time interval has reached the design threshold to deal with the increasing passenger flow pressure.However,due to the imbalance of passenger distributions of urban rail transit systems and the uniformity of the CBTC departing time interval,the mismatch between the train flow and the passenger flow is serious,which means the shortage of transport capacity in the area with large passenger flow density and the waste of transport capacity in the area with small passenger flow density.It is urgent to build a new flexible transportation organization mode based on CBTC to increase the service quality of urban rail transit and improve the satisfaction of passengers.It is significant to the sustainable development of urban rail transit that trains are running with high speed and closer distance between adjacent ones.A multi-train formation architecture based on cooperative control is proposed in this thesis according to the characteristics of passenger flow distribution in urban rail transit.With the application of the artificial potential field method and the reinforcement learning,we build a novel multi-train tracking mode based on the "soft wall" mechanism to improves the operation efficiency of urban rail transit and enhances the flexibility of operation organization mode.The main works of this thesis are as follows:(1)The feasibility and necessity of multi-train cooperative formation are evaluated based on the actual passenger flow data.A solution of multi-train cooperative formation is proposed according to the tidal passenger flow data of Beijing Metro Batong Line.The optimization ability of the multi-train cooperative formation is quantitatively analyzed based on train load rate and line operation efficiency,and demand-driven multi-train cooperative formation scenarios are summarized.(2)A multi-train cooperative formation approach based on the artificial potential field is proposed.The leader-follower model is introduced,and we design an artificial potential field function according to the operation characteristics of urban rail transit,where parameters such as the transmission delay,the response delay and motor efficiency are considered comprehensively.The proposed approch can achieve the high efficiency and punctualityunder the transportation plan.(3)A multi-train cooperative formation approach based on the reinforcement learning is developed.The artificial potential field method and the reinforcement learning are integrated to achieve fast convergence and accurate control of the multi-train cooperative formation,and the "hard wall" mode and the "soft wall" mode are both considered.The proposed approach can improve the tracking ability,parking accuracy,punctuality,the comfort level and safety.(4)The verification method of the multi-train cooperative formation is proposed.Two algorithm verification mechanisms are developed,including the ROS(Robot Operating System)simulator of cyber space and TB3 Robot of physical space.The feasibility of the multi-train cooperative formation is evaluated based on actual operating parameters and communication performance,and the availability of the proposed algorithms is verified.The algorithm simulations and verification results show that the multi-train cooperative formation approach designed in this thesis can provide a flexible transportation organization mode,achieve the matching of train flow and passenger flow,which is an approapriate solution to transporation problems of urban rail transit such as the tidal passenger flows.