A Decision-making Method For Autonomous Driving Based On Deep Reinforcement Learning

The existing decision algorithms in the field of autonomous driving are often difficult to meet the performance requirements of complex traffic scenarios.Based on the existing reinforcement learning algorithms,this study proposes two new algorithms to address the difficulties encountered in the application of current reinforcement learning algorithms in the field of autonomous driving,and solves the decision problem of autonomous driving in scenarios where the dynamics of the observed vehicle set changes and the intention of other vehicles is not directly observable.The main work of this paper is as follows:First of all,a simulation platform for reinforcement learning algorithm training was built using SUMO.Use SUMO simulation software to build a simulation platform,establish communication between Python and the simulation platform,and realize the training of deep reinforcement learning algorithms based on SUMO platform;based on the highly customizable characteristics of SUMO,a three-lane crossroad without signal lights is built.,two-lane no-signal T-intersection scene,irregular no-signal intersection scene and other scenarios to meet the training and verification requirements of the algorithm.Second,based on the attention mechanism,this study proposes a new state characterization method to solve the state characterization problem under the dynamic change scenario of the peri-vehicle set of autonomous vehicles.Using the self-attention mechanism,a state characterization module of the autonomous vehicle for the environment is built,which can input any number of self-vehicle and other vehicle state information and output a fixeddimension state vector as the observation vector of the autonomous vehicle;meanwhile,the performance difference between the new proposed state characterization method and the existing state characterization methods is verified by comparing them in simulation.Finally,based on the long and short-term memory network and attention mechanism,a new decision algorithm for autonomous driving is proposed to solve the decision problem in partially observable environments in autonomous driving tasks.The decision algorithm predicts the hidden states of other vehicles from their historical trajectories by means of a long and shortterm memory network or an attention mechanism network,and outputs the decision actions by means of a reinforcement learning algorithm.The performance of the proposed algorithm is compared with the basic algorithm in a simulation scenario to verify the effectiveness of the proposed algorithm in this study.