Research On Autonomous Driving Human-like Car-following Decision Algorithm Based On Deep Reinforcement Learning

Autonomous driving decision making is the process of making a decision on the ideal action of the vehicle based on the current motion state and environmental information.At present,automatic driving decision can maintain vehicle safety and driving efficiency,but different drivers have different personalities,skills and styles,and the uniformly calibrated control strategy cannot meet the driving habits of different drivers.Considering the integration of driver’s behavioral preferences into the autopilot tracking decision to improve the human friendliness will promote further development in this field.To improve the personalization of autonomous vehicle following,it can adapt to different drivers’ driving styles and improve occupant acceptance of autonomous driving functions.In this paper,we propose a follow-along decision algorithm that considers driver characteristics,and based on this,we propose an anthropomorphic follow-along decision algorithm for autonomous driving based on supervised signal-guided reinforcement learning to meet the needs of more drivers.The details of the work are as follows:(1)A driving data collection platform is built using Carla automatic driving simulator and Logitech G29 driving simulator.The following scenario is built by the automatic driving simulator,and the driving data of different drivers are collected,and the driving behavior features are extracted.Secondly,the driver data were enhanced using time series-based variational self-encoder generated data for subsequent driver reference model training to improve the driver reference model accuracy.(2)To address the problem that the current automatic driving decision methods cannot satisfy different driver driving habits,a dual-delay deep deterministic policy gradient algorithm is used to implement the automatic driving car-following decision function.And different reward functions are designed using according to the driver statistical information.The driving behavior characteristics of the decision model with different reward functions are analyzed,and the experimental results show that the design of reward functions according to different driver characteristics can meet the needs of drivers with more stable following distance and headway time distance.(3)In order to improve the generalization ability and Human-like effect of the model,meet the needs of more drivers,and reduce the convergence time of the model,this paper combines the advantages of imitation learning and deep reinforcement learning,and designs a deep reinforcement learning Human-like following decision algorithm using supervised signal guidance.During the training process the driver reference model receives information about the temporal environment and generates a supervisory signal,which guides the deep reinforcement learning model to learn from the supervisory signal.Finally,the results of the trained model are compared in terms of its driving characteristics and convergence speed during the tracking process.At the same time,the convergence speed is 30% faster compared to traditional deep reinforcement learning.Therefore,the decision model using supervised signal guidance can converge faster.