Research On Driving Decision-Making Of Intelligent Connected Vehicle In Mandatory Lane Changing Scenarios Based On Deep Reinforcement Learning

Mandatory lane changes caused by road construction,vehicle accidents,etc.occur frequently on expressways and expressways.The traditional mandatory lane-changing decision-making model can only make decisions based on limited driving experience,so it often does not perform well in improving traffic efficiency in the face of complex environments.Deep reinforcement learning is considered to be an effective solution to the traffic and driving decision-making problem in complex environments due to its ability to fully perceive the surrounding environment information and its powerful decision-making ability.In this regard,this paper studies and proposes a set of decision-making models for ICV driving in mandatory lane changing scenarios based on deep reinforcement learning on the basis of designing and conducting real vehicle experiments in mandatory lane changing scenarios.The main work includes the following three aspects:(1)The real car experiment in the mandatory lane change scene.In order to solve the problem of insufficient observation data for the mandatory lane change scene,a mandatory lane change experiment including 30 real vehicles was organized and carried out using the circular two-lane test site with a perimeter of 411 meters in the highway test site of the Ministry of Communications.On the one hand,the queuing phenomenon in the bottleneck area of mandatory lane change is studied through experiments,and on the other hand,the reasons for the decrease of traffic efficiency and driving comfort are analyzed.(2)The mandatory lane change simulation model based on the real vehicle lane change trajectory.In order to simulate the mandatory lane change decision-making behavior of real vehicles,a mandatory lane change simulation model based on real vehicle experimental data is proposed.Firstly,4 mandatory lane change trajectory libraries with different vehicle densities are constructed according to the experimental data of 8 rounds and a total of 200 minutes.Secondly,randomly select a trajectory in each simulation,and use the least squares method to perform polynomial fitting on the trajectory,thereby forming the mandatory lane-changing trajectory of the simulated vehicle;finally,during the lane-changing process of the simulated vehicle along the trajectory,The safe car following mechanism is introduced to realize the interaction between the simulated vehicle and the surrounding vehicles.The model provides a realistic simulation environment that can interact with surrounding vehicles for deep reinforcement learning training.(3)A decision-making method for forced lane change driving in ICV based on deep reinforcement learning.Based on the MAPPO(Multi-Agent Proximal Policy Optimization)algorithm,an ICV decision-making model suitable for forced lane changing scenarios is proposed: the multi-dimensional state information of the vehicle and surrounding vehicles is considered in the state space design;the action output includes acceleration and The vehicle control quantity of the two continuous spatial dimensions of the driving angle;factors such as overall traffic efficiency,safety,and comfort are considered in the design of the reward function.And using the Python-Traci interface to develop and build a forced lane change simulation environment consistent with the real vehicle experiment on the SUMO traffic simulation platform,the proposed ICV decision-making method is verified.Experiments show that this method can significantly improve the overall traffic efficiency on the basis of ensuring safety and comfort.The decision-making method for mandatory lane change driving of ICV based on deep reinforcement learning proposed in this paper provides a useful solution to the problems of low traffic efficiency,low safety level,and poor driving comfort experience caused by mandatory lane change behavior.area.