Eco-driving Strategy Design Of Intelligent Vheicle At Signalized Intersection Based On Deep Reinforcement Learning

Connected and autonomous vehicles(CAVs)can make use of the road traffic information obtained by connected communication technology or the prediction of future driving conditions,and through economic driving technology,that is,reasonable motion or speed planning,can greatly improve the energy utilization rate of vehicles.However,the existing economic driving technology has some problems,such as high safety risk,lack of robustness and timeliness of control algorithm,when driving at random dynamic traffic lights.Therefore,in order to improve energy efficiency of connected-automated electric vehicles and realize safe and efficient driving decisions,this paper starting from the energy saving mechanism and researched on eco-driving strategy of electric vehicles at signalized intersection based on deep reinforcement learning and the constrained reinforcement learning respectively.The main research work of this paper is as follows:Firstly,the vehicle dynamics model of four-wheel independent drive was established to construct the optimal control problem of electric vehicle with minimum energy consumption including longitudinal dynamics,hub motor,power battery,etc.,and the optimization of energy efficient driving strategy of electric vehicle was realized through dynamic programming method.The control law of energy saving speed and the influence mechanism of energy consumption of electric vehicles under different initial and terminal conditions are studied.Secondly,a traffic control strategy based on Deep Deterministic Policy Gradient(DDPG)is proposed.The environment model of signalized intersection was established,the reinforcement learning framework of intelligent vehicle based on speed guidance was constructed,and the reinforcement learning reward function was designed which took longitudinal dynamics of vehicle into account and aimed at minimizing energy consumption,so as to realize the training of economical driving strategy of typical intelligent vehicle controlled by signal lamp.The simulation of typical signalized intersection shows that the proposed DDPG eco-driving strategy can improve the energy efficiency by more than 15% in dynamic random driving scenarios compared with the traditional "accelerating-uniform-braking" strategy.Then,an eco-driving strategy based on constrained reinforcement learning was proposed for the following scenario at signalized intersection.The relationship between safety constraints and signalized intersections is studied;The dynamic safety boundary based on the control barrier function is established;The reinforcement learning strategy optimization problem satisfying control security constraints is reconstructed;A learning strategy iterative method satisfying the safety constraints such as car following is explored;And the efficient training of decision control strategy of safety assurance reinforcement learning is realized.The simulation results show that compared with the traditional DDPG,the designed eco-driving strategy can improve the energy efficiency by 10.2% while ensuring the safety.Finally,the digital twin test platform of CAV is built based on MATLAB/Simulink and Unreal Engine,in which the CAV is real car and the road traffic environment is virtual system.The virtual and real combination test of signalized intersection is carried out by MATLAB/Simulink and ROS.Experimental results show that the proposed strategy has good speed tracking performance and algorithm generalization performance.