Research On The Lane-changing Trajectory Planning Of Intelligent And Connected Vehicles Based On Deep Reinforcement Learning

Lane-changing is a basic task in the process of driving,and improper operation may easily lead to traffic accidents.Intelligent and connected vehicles are equipped with advanced sensing devices and integrated wireless communication technology to realize information interaction between vehicle and vehicle,vehicle and road.Using reasonable decision-making,planning,and control methods,can improve traffic efficiency,enhance vehicle safety,and reduce fuel consumption.In order to improve the solution efficiency of the optimal lane-changing trajectory planning problem under the multi-objective constraints of intelligent and connected vehicles,and obtain an optimized lane-changing trajectory that takes into account both traffic safety and energy efficiency,this paper investigates the deep reinforcement learning based intelligent and connected vehicles lane-changing trajectory planning method with the following main research work:(1)A lane-changing decision model based on a dynamic game with complete information is designed.The characteristics of vehicle lane-changing scenarios are analyzed,and a lanechanging decision model based on a dynamic game with complete information is designed.Considering the safety and timeliness in the decision-making process,the calculation of game profit is completed,and the Nash equilibrium strategy is solved to provide decision information for trajectory planning.(2)The twin delayed deep deterministic policy gradient(TD3)based lane-changing trajectory planning algorithm for intelligent and connected vehicles is proposed to solve the multi-objective constrained vehicle lane-changing trajectory planning problem with complex temporal interaction characteristics.Taking vehicle position and velocity as input states,acceleration as output action,and comfort and fuel economy as reward functions,a lanechanging trajectory planning model based on TD3 algorithm is constructed to obtain optimized vehicle lane-changing trajectories.(3)Optimization of TD3 lane-changing trajectory planning model by introducing long short-term memory(LSTM)network,and the LSTM-TD3-based lane-changing trajectory planning model is proposed.The historical driving data extraction module is added to the TD3 policy network and evaluation network,and the policy selection and policy evaluation are carried out in combination with the current information,which improves the completion rate of lane-changing for intelligent and connected vehicles trajectory planning and further optimizes the lane-changing trajectory.(4)Typical lane-changing scenarios are constructed to simulate and verify the above model.The lane-changing decision model based on complete information dynamic game obtains safe lane-changing decisions under different driving scenarios and provides a basis for trajectory planning;the trajectory planning model based on TD3 algorithm achieves safe,energy-saving and comfortable lane-changing trajectory planning,the average fuel consumption is reduced by63% and 44% during left and right lane-changing,respectively.Average training speed improvement of about 10.5% compared with the deep deterministic policy gradient(DDPG)algorithm.The lane-changing trajectory planning model based on the LSTM-TD3 algorithm further optimizes the lane-changing trajectory,increases the average gain,and reduces the fuel consumption.The average fuel consumption in the left lane-changing experiment is reduced by13.71% compared with the TD3 algorithm,3.02% compared with the LSTM-DDPG algorithm,and 14.52% compared with the DDPG algorithm;the average fuel consumption in the right lane-changing experiment is 34.80% less than TD3 algorithm,30.05% less than the LSTMDDPG algorithm,and 39.66% less than the DDPG algorithm.In summary,the proposed deep reinforcement learning-based intelligent and connected vehicles lane-changing trajectory planning method can effectively improve the efficiency of optimal lane-changing trajectory planning solution under multi-objective constraints,and can effectively improve the lane-changing trajectory planning of intelligent and connected vehicles.