Research On Intelligent Decision Control Of Autonomous Driving Based On Deep Reinforcement Learning In Hierarchical Architecture

Decision-making control of autonomous vehicle is a key problem in the research of autonomous vehicle.With the development of artificial intelligence technology,the introduction of learning-based artificial intelligence method into decision-making control has become a hot topic in the world.At present,the decision-making control of autonomous vehicles based on learning method mostly adopts the end-to-end model structure,and its decision-making process lacks certain interpretability,and the model is mostly for a single driving behavior.Deep learning methods rely on a large amount of annotated data in the training process,and the deep reinforcement learning method can learn through interaction with the environment,which is more suitable for intelligent decision control problems.However,the existing decision-making control models of autonomous vehicles are less combined with deep reinforcement learning methods.Therefore,based on deep reinforcement learning algorithm,this paper proposes a hierarchical modeling scheme for decision-making control of autonomous vehicles to solve the problem of decision-making control of autonomous vehicles facing multiple driving behaviors.The specific research contents are as follows:First,the study of decision-making control model of autonomous vehicle based on deep reinforcement learning method under hierarchical architecture.Firstly,the change of driving state of autonomous vehicle in the process of driving behavior execution is studied,the hierarchical structure framework of " action decision-vehicle control" of autonomous vehicle decision control is proposed,and the overall decision-making process of the model is analyzed.Secondly,based on the consideration of environment and other factors in the decision-making of autonomous vehicle,two sublayer models are abstracted,and a universal end-to-end model framework in line with the decision-making process of deep reinforcement learning algorithm is proposed for the sublayer model.Finally,the scalability of the model for multi driving behavior is analyzed.Second,the vehicle control model based on deep deterministic policy gradient algorithm.Firstly,the vehicle control layer is modeled based on DDPG algorithm.Secondly,by analyzing the key action points when the vehicle state obviously changes during the execution of driving behavior,a reward function based on the target driving state is designed for the vehicle control model as the model optimization objective.Finally,the vehicle control network model of conditional control structure is proposed.Third,the action decision model based on deep Q network algorithm.Firstly,the action decision model was modeled based on DQN algorithm.Secondly,for the three typical driving behaviors of lane-changing,overtaking and car-following,the reward functions corresponding to the characteristics of the target driving behaviors were designed respectively under the premise of considering the safe co-driving.Finally,the network model of action decision making is proposed.Finally,the simulation training and verification analysis of the decision-making control model of autonomous vehicle are carried out.Firstly,The TORCS（The Open Racing Car Simulator）simulation platform is modified as the autonomous vehicle interactive environment in this paper.Secondly,the overall training process of vehicle control model and motion decision model is designed,and the random exploration process of autonomous vehicle is optimized by combining the function curve and probability distribution.Finally,in the dynamic scenario including other traffic participants,the decision-making control ability of this model for multi driving behavior is verified and analyzed.