Research On Stylized Car-following Decision-making For Autonomous Driving Based On Deep Reinforcement Learning

In recent years,the number of motor vehicles in China has grown rapidly,leading to increasingly severe problems such as traffic safety and congestion.Vehicle following,as an important part of traffic scenarios,can improve traffic efficiency and alleviate traffic pressure by optimizing the driving behavior of drivers alone.Currently,research on vehicle following behavior is mainly divided into traditional following models based on mathematical and physical methods and data-driven following models.However,these models have a single design pattern and lack consideration of personalized following requirements and multiobjective optimization.To address these issues,this paper proposes a vehicle following decision-making method that combines driving styles based on deep reinforcement learning,aiming to meet the following needs of drivers with different styles and optimize the following performance.The specific research contents are as follows:(1)To address the issues of dataset extraction and identification of stylized data in the vehicle following model,this study proposes a comprehensive approach.Firstly,the NGSIM(Next Generation Simulation)I-80 dataset is carefully selected as the primary data source,and an in-depth error analysis is conducted.The data is then reconstructed using the Savitzky-Golay filtering algorithm to ensure the exclusion of any erroneous samples.Secondly,focusing on small and medium-sized vehicles,following events are meticulously extracted based on specific selection criteria.The clustering features of relative velocity,headway time,and following distance are utilized.The K-means algorithm is employed to categorize the data into three distinct styles: aggressive,smooth,and conservative.Finally,leveraging multiple classification identification models,the stylized following data is meticulously trained and tested in a 7:3ratio.The outstanding results highlight the significant effectiveness of the K-means clustering technique,with the three SVM(Support Vector Machine)models achieving an impressive recognition rate of up to 98.8% for the stylized data.(2)To address the drivers’ demands for safety,driving efficiency,fuel economy,and comfort during the vehicle following process,a cutting-edge solution is proposed: a multiobjective automatic driving decision-making model for following based on the Deep Deterministic Policy Gradient(DDPG)algorithm.Firstly,an in-depth analysis and design of the state space,action space,and model network of the following vehicles are conducted.Recognizing the diverse objectives in the vehicle following process,a comprehensive reward function is formulated within the DDPG framework,incorporating safety,efficiency,fuel economy,and comfort metrics.Furthermore,a collision avoidance strategy,based on a following distance threshold,is devised to ensure absolute safety during the following process.Subsequently,a simulation environment is created using SUMO(Simulation of Urban MObility)to simulate the car-following process.Experimental data demonstrates that vehicles using the DDPG decision-making model exhibit higher driving efficiency and significantly improved comfort compared to human-driven vehicles during car-following.With the assistance of the safety collision avoidance strategy,the car-following process becomes safer.Moreover,when compared to the Intelligent Driver Model(IDM),vehicles employing the DDPG decisionmaking model achieve an average fuel consumption reduction of 12.28%,with improved driving efficiency and comfort.(3)To address the issue of the limited range of autonomous driving vehicle following modes and the lack of consideration for personalized following demands from passengers,a driver-style-based autonomous driving following strategy is proposed.Firstly,using the clustered following data representing three different styles as a starting point,the feature parameters related to safety,driving efficiency,fuel economy,and comfort are calibrated,and reward functions for different style DDPG following models are designed.The stylized data is divided into training and testing sets in a 7:3 ratio,and model validation experiments are conducted on the SUMO platform.The experimental results demonstrate that the multiobjective reward values for the three styles all reach a converged state,and there are noticeable differences in the distribution of following motion parameters among different styles.Furthermore,considering the varying emphasis of each style’s following model on different indicators,a DDPG stylized following optimization model that integrates the Particle Swarm Optimization(PSO)algorithm is designed.By optimizing the weights of the multiple objectives,the model aims to find the optimal combination of weights,achieving a balance in multiobjective rewards for each style’s following model,and further optimizing the model based on stylization.The experimental results show that the DDPG following model with the integrated PSO algorithm not only achieves rapid convergence but also enhances the adaptability of the model to driving styles.The rewards for various indicators are effectively balanced and optimized,achieving maximum benefits under the premise of stylization.