Research On Lane Changing Of Autonomous Vehicles Based On Reinforcement Learning

Autonomous vehicles play an important role in solving traffic congestion and safety,and the study of lane changing behavior of autonomous driving is known as an important research content for higher level autonomous driving decision planning systems.Compared to other driving behaviors,the complexity of lane changing behavior is higher.Lane changing vehicles need to consider the location,speed,and other information of all vehicles in their observable surroundings.The existing research on automatic driving lane change behavior models and observes the results based on real trajectory data,simulating human drivers’ lane change decisions through training,but it is not possible to obtain drivers’ behavioral motivations during the lane change process from training data.Reinforcement learning can learn the driving strategies of other vehicles in the traffic flow,thereby obtaining the optimal strategy and maximizing the flow of traffic flow.This provides a solution for the study of autonomous driving decision-making.This paper conducts modeling research on the decision-making process of automatic driving lane change.The existing research on autonomous driving reinforcement learning takes autonomous vehicles as the main body of training,and inputs all other vehicles as environments to autonomous vehicles,which ignores the interaction between autonomous vehicles.Therefore,the main research content of this article is the cooperative lane change of autonomous vehicles in the case of multiple agents,and on this basis,each agent will conduct a separate study of driving style and lane change strategies.The specific work arrangement includes the following two aspects:(1)The lane changing problem of mixed traffic expressway vehicles is described as a multi-agent reinforcement learning problem,in which autonomous vehicle collaborative learning is a strategy adapted to human driving vehicles to maximize traffic throughput.This article extends the SACDiscrete algorithm,which was previously studied,to the MASAC-Discrete algorithm using a framework of centralized training and non centralized execution.In addition,this article also proposes a motion prediction safety controller that includes a motion predictor and motion replacement module to ensure driving safety during training and testing.The motion predictor estimates the trajectory of the autonomous driving vehicle and surrounding vehicles according to the kinematics model,and predicts the trajectory of the autonomous driving vehicle to determine whether there is a potential collision.The action replacement module replaces dangerous actions based on the safe distance and sends them to the low-level controller.This article trains,evaluates,and tests the proposed method on a highway simulator similar to Gym under three different levels of traffic modes.The simulation results show that even in high traffic density situations,this method can significantly reduce collision rates while maintaining high traffic efficiency.In the highway scenario established in this article,its performance is superior to several state-of-the-art benchmark algorithms.(2)In order to explore the impact of driver driving style on lane changing decisions for autonomous vehicles,this paper uses the classification results of driving styles studied by previous researchers,and uses DQN reinforcement learning algorithms to interact with autonomous vehicles and three types of human driving styles,respectively,to maximize traffic throughput.The experimental results show that under the conservative driving style condition,the driving strategies of autonomous vehicles tend to be conservative.Under the normal driving style condition and the aggressive driving style condition,different parameters are used for K-Means clustering analysis.The driving strategies of autonomous vehicles are generally similar to the driving style,but there are also a few cases where the driving strategies tend to be conservative.