Energy-efficient Train Operation Control Of Automatic Driving Based On Q Learning And Deep Q Learning

In recent years,China’s urban rail transit has been developing rapidly,and the total operating mileage of metro,light rail,maglev and other rail transit systems has increased year by year.At the same time,the increase in operation miles is accompanied by huge energy consumption,so the energy-efficient operation of automatic train driving is closely related to the sustainable development of urban rail transit.In this thesis,the train operation control method based on reinforcement learning is established by taking urban rail transit as the research object.The actual subway line information is selected,and the above method is verified through simulation.The main work of this thesis is as follows:Firstly,according to the research on train energy-saving operation at home and abroad,the force analysis of train operation process is carried out,and the train kinematics model is established.Then,the structure of reinforcement learning algorithm is represented,and the basic principle of reinforcement learning is described by Markov decision process.Combined with the train motion analysis,the train energy-saving operation reward is set based on Q-learning algorithm,and the value function iterative reasoning of train operation control is carried out.According to the reinforcement learning algorithm,an energy-saving control method of automatic train operation is designed based on Q-learning algorithm.Furthermore,this dissertation introduces the basic principle of deep reinforcement learning.In order to improve the training speed of the algorithm,an improved dueling dqn network structure is used.Regarding to the application of deep reinforcement learning method to train operation control,the train operation environment based on deep q-network(DQN)framework is established,and the agent and train environment are constructed.According to the analysis of train state,the value functions of state value function and dominance function are designed to evaluate the train state and action,and the action strategy is optimized.Finally,simulation experiments are performed.Based on the data of Yizhuang line of Beijing Metro,the algorithm is verified by simulation.The simulation results show that,on the basis of meeting the basic operation conditions,the algorithm has better control effect than the actual train operation energy consumption.Q-learning algorithm has obvious advantages in calculation speed,while DQN algorithm is more intelligent and has better optimization effect on train energy consumption.Thus,the effectiveness of the proposed model is verified for automatic train operation.38 Figures,10 Tables,and 51 References.