Anthropomorphic Decision-making For Automated Driving Vehicle Based On Deep Reinforcement Learning Theory

Automated driving is one of the core technologies in vehicle engineering research,which can efficiently improve road safety,relieve traffic congestion,and reduce environmental pollution.On the one hand,the automated driving decision-making system should meet the requirements of driving safety and traffic efficiency,on the other hand,it should meet the different driving styles to improve the driving experience.At present,it is difficult to improve the decision-making system performance because of the high dimension and different driving characteristics for the research on driving safety and traffic efficiency improvement.Therefore,aiming at the vehicle automated driving decision-making problem,this research proposes an automated driving anthropomorphic decision-making model based on GAIL-DDPG and the difference of driving style.Then the effectiveness of the model is verified.The main research contents of this paper include:(1)Analysis of driving style characteristics and database constructionBased on the purpose of this research,the analysis and expression of the driver’s driving style is the basic requirement of anthropomorphic decision-making research.Different drivers have great differences in the running state of vehicles in the same traffic environment,so the driving style can be classified according to the relevant running state parameters of vehicles.Therefore,based on the detailed analysis of the driver’s driving style evaluation method and the combination of test data to quantify and characterize the driver’s driving style,this research constructs a data set for the construction of an anthropomorphic decision model.(2)Reward function design of reinforcement learning based on artificial potential field theoryBuilding a basic reinforcement learning model is the basis of anthropomorphic decisionmaking in this research.Among them,the reward function needs to be designed according to the running characteristics of the vehicle.Based on this,due to the successful application of artificial potential field theory in path planning and the in-depth analysis of vehicle safety,efficiency,and comfort in this article,an agent-centric vehicle operation potential field function design method is proposed,and then combine experimental data to train and verify the constructed reinforcement learning model.(3)Transfer strategy of imitation reinforcement learning training modeImitation learning based on expert data can effectively improve the training efficiency of reinforcement learning models.How to achieve a smooth transition from imitation learning to reinforcement learning is one of the keys to constructing the GAIL-DDPG model.Among them,the transfer strategy should consider maximizing the distribution of expert data,and on the other hand,it should take into account the requirements of reinforcement learning to ensure that the agent’s decision does not deviate from the expert data distribution and explore more advanced decision-making behaviors.Based on this,based on the in-depth analysis of the characteristics of various transfer functions,this research proposes a training mode transfer strategy based on the Sigmoid function,thereby achieving a smooth transition from imitation learning to reinforcement learning.(4)Construction of anthropomorphic reinforcement learning model for automated drivingBased on the GAIL-DDPG model constructed above,how to introduce the driver’s driving characteristics is the key to realizing anthropomorphic decision-making.Therefore,based on the reliable identification of driver’s driving style,this research realizes the reasonable introduction of different driver’s characteristics through the dynamic adjustment of the weight of vehicle operation potential field function,and then establishes a deep reinforcement learning automated driving anthropomorphic decision-making model considering the individual characteristics of the driver.Focusing on the decision-making problem of vehicle automated driving,this research carried out related research work on the driver characteristics classification and anthropomorphic decision-making model construction.Wherein,related breakthroughs in this research are as the following:(1)Aiming at the problem of efficiency improvement in reinforcement learning training,this research solves the problem by introducing imitation learning and designing training mode transfer strategy,and realize the effective improvement of the early training efficiency of the reinforcement learning model;(2)Aiming at the problem of anthropomorphic decision-making,this research solves the problem by constructing APF for vehicle driving and dynamically adjusting the weight of the reward function,and realize the construction of an automated driving anthropomorphic reinforcement learning decision-making model.