Research Of The Lane Following Decision-making Of Autonomous Vehicle Based On Deep Reinforcement Learning

As a product of the fifth stage of the information technology revolution,autonomous vehicle technology is of great significance for improving urban traffic situation and eco-friendly development.On the other hand,it plays an important role in driving safety and reducing the unnecessary incidence of traffic accidents.At present,the mainstream solution of autonomous vehicle technology is the stack of the perception-decision-control technology module.The decision-making module relies on the results of the perception module and outputs the control strategy to the control module.However,there are still some difficulties in the decision-making module of this traditional solution:1?the process of designing those artificial rule-based strategy is complex and highly expensive.2?these sophisticated strategies are often very fragile,unable to adapt to the dynamic environment.3?heavily rely on perception module.If there is a fault in the perception module,it will directly affect the operation of decision-making,and then affect the safety of vehicles and passengers,resulting in disastrous consequences.In order to solve the problems of traditional decision-making module,we propose an End-To-End decision-making system solution based on deep reinforcement learning.With the test of the solution in the lane-following task in CARLA simulator,we evaluated the effectiveness of this solution,which can provide some references and improvements to the traditional decision-making methods.According to the features of unmanned driving tasks,in this paper,we have improved the DQN algorithm and DDPG algorithm in following aspects:(1)Using priority experience replay method(PER)to improve the utilization rate of DQN algorithm and DDPG algorithm for experience data;(2)Double DQN method is adopted to improve the overestimation problem of DQN algorithm;(3)OU noise more suitable for inertial system is used to enhance DDPG algorithm's ability to explore the environment;(4)Regularization is applied to the target policy network of DDPG algorithm,and the update speed of the target policy network is delayed to solve the problem that DDPG algorithm is easy to be limited to local optimization.Finally,a deep reinforcement learning algorithm library is designed and implemented.The algorithms described above are respectively implemented(DQNAgent and DDPGAgent)and applied to the lane-following task in this paper.According to the universal assessment scheme in the experimental platform,we make a comparison of these two algorithms' evaluation:in a stationary environment and a dynamic environment,DQNAgent achieved 98%and 82%on average task completion,respectively;DDPGAgent respectively achieved 97%and 78%of the average task completion.Compared with the lane-following task completed by the A3C method in CARLA's paper in 2017,the method adopted in this paper greatly improves the average task completion degree and reduces the cost of development and training.