Research On Driving Decision-making Algorithms Of Intelligent Vehicles For Structured Road Environment

With the development of Intelligent Transportation,Intelligent Vehicle has gradually become the focus of research in the automotive industry.Driving decision-making is the key point for Intelligent Vehicle to be different from traditional Intelligent Driving Assistant Vehicle.Therefore,the development of driving decision algorithm has greatly affected the development of intelligent vehicle technology and industrialization.However,the application of traditional driving decision-making algorithm is limited,so the establishment of a new driving decision-making algorithm that can meet the requirements of the development of Intelligent Vehicle in the new era has become an urgent problem to be solved.This paper gives a brief analysis of the driving decision-making algorithms adopted by industry and academia at home and abroad.Considering the difficulties and key points in the development of driving decision algorithm,a driving decision algorithm based on deep reinforcement learning algorithm is proposed.The main research contents are as follows:(1)Research on the method of deep reinforcement learning for driving decision-making.Based on the analysis of the specificity of driving decision-making,a deep reinforcement learning method is selected as the basis of constructing driving decision-making algorithm.This paper briefly introduces the basic contents of deep reinforcement learning,including the basis of reinforcement learning,learning methods and function fitting method based on neural network.Considering the characteristics and actual needs of driving decision-making,hybrid learning is chosen as the basic method,and Deep Deterministic Policy Gradient(DDPG)is chosen as the basic method.Aiming at the problem of high training time and cost of DDPG algorithm,an improved DDPG algorithm(DDPG-G)is established by introducing optimized sequencing sampling and accurate action evaluation to improve the algorithm.(2)Research on factors influencing driving decision-making oriented to reward function.The construction method of reward function is studied and the selection criteria of influencing factors of driving decision are analyzed.Four categories(safety,efficiency,comfort,altruism)and eight sub-categories of influencing factors are selected by combining the analysis method with the actual driving decision.The influencing factors are quantified and given thresholds,which lays the foundation for the construction of reward function and the design of the overall algorithm.(3)Design of driving decision algorithm and construction of training and testing platform.Based on the above research,TORCS is selected as the software platform for training and testing,and the architecture of algorithm training simulation platform is introduced.At the same time,the selected vehicle dynamics model is embedded.Intensive learning and neural network function fitter are designed,and state,action and network topology are selected.(4)Driving decision algorithm training and test analysis.The environment of training and testing is configured,and the driving decision algorithm is designed to conduct training and testing tests respectively.In the training experiment,the improved DDPG algorithm and the improved DDPG algorithm are trained under the same map,and the data obtained are compared.The influence of the improved DDPG algorithm on the convergence of the driving decision algorithm and the cost control of driving decision training time is analyzed.In the test,the effect of DDPG-G algorithm is analyzed from four aspects: reliability,stability,realtime and generalization.The experimental results show that the DDPG-G algorithm saves about 20% of the training time without reducing the convergence;the DDPG-G algorithm has good reliability on the training road;the passing rate in the stability test is 98%;the real-time performance is close to the level of skilled drivers;in the generalization test,the passing rate of complex maps is 74%,mainly because part of the state of complex road environment is untrained.Training can be solved by increasing training.This paper adopts the method of deep reinforcement learning,and establishes a driving decision algorithm based on improved depth deterministic strategy gradient.Compared with the traditional driving decision algorithm,it avoids the theoretical defects;compared with the improved algorithm,it has shorter training time cost without sacrificing convergence,and has good reliability,stability,real-time and generalization.