Research On Autonomous Driving Decision Control Based On Deep Reinforcement Learning

Autonomous driving is one of the research hotspots in the field of artificial intelligence.Decision control technology is the core technologies of autonomous driving and it is an important factor to ensure the safety of autonomous vehicles.It is of great significance to study decision control technology.How to make decisions quickly and accurately in various scenarios is a key issue in automatic driving decision control technology.As one of the core technologies in the field of artificial intelligence,Deep Reinforcement Learning(DRL)is an effective method to solve the problem of automatic driving decision-making control.The agent interacts with the environment and learns driving strategies based on the feedback of the environment to achieve the goal of independent decision-making.This paper mainly focuses on the needs of end-to-end automatic driving decision control technology,and studies the method of automatic driving decision control based on Deep Deterministic Policy Gradient(DDPG).New algorithms are proposed in this thesis to solve the problems of DDPG based automatic driving decision-making control methods,such as poor strategy at the initial stage,inability to effectively learn human driving skills,and inability to apply to gradient-free scenes.Main contributions of this thesis are as follows.1.Aiming at the problem of poor strategy in the initial stage of DDPG algorithm and the inability to effectively learn human driving skills.,one new algorithm named DDPGwE(DDPG with Expert)algorithm combined with expert experience is proposed.DDPGwE algorithm uses expert experience to pre-train the actor network,so that the agent has some "prior knowledge",and the LSTM module is added to the actor network to improve the ability of the autonomous vehicle to predict the future situation.At the same time,the loss function of the critic network is improved,so that the expert experience can be used as an auxiliary guide for the critic network to evaluate the actor network strategy.Experimental results show that the DDPGwE algorithm has learned a better strategy,the learning speed is faster,and it has better generalization ability and robustness.2.Regarding the problem of difficulty in defining the reward function of the DDPG algorithm guided by the reward value.This paper proposes a DDPG algorithm(Generative Adversarial DDPG,GADDPG)that combines generative adversarial learning.GADDPG algorithm uses expert experience to directly learn strategies from expert experience and obtain reward value through the way of generating confrontation,instead of the original DDPG algorithm obtaining reward from the environment for learning.Experimental results show that the GADDPG algorithm can learn the strategies of human experts very well,and the learning speed is fast.3.In order to solve the problem that the existed deep reinforcement learning algorithms cannot be applied to the gradient-free scenes,this thesis proposes a gradient-free Co-evolutionary Reinforcement Learning(COERL)Algorithm.COERL Algorithm uses an evolutionary approach to design the neural network structure and uses the co-evolutionary algorithm to optimize the neural network weights,which solves the parameter optimization problem in the gradient-free scenes and improves the strategy learning ability of autonomous vehicles.The experimental results show that COERL Algorithm is competitive with deep reinforcement learning algorithms,and its performance is better than other deep reinforcement learning algorithms on certain tasks.