Research On Local Path Planning In Vehicle-Road Cooperation

With the continuous development of social economy and technology,road traffic has gradually been a hot spot in daily life.Since the development of public construction,we have put forward higher requirements for traffic.In order to solve the problem of reasonable utilization of traffic resources,traffic congestion and vehicle driving safety problems,our government proposed the concept of vehicle-road coordination in 2011.There is great value in the combination with the automobile industry,road construction with communications and the internet,which leads to build a modern vehicle-road cooperative road by using artificial intelligence.Therefore,this thesis works on the path planning problem in the vehicle-road collaborative system based on the study of reinforcement learning.It proposes an improved retrospective Q-Learning algorithm and a multi-layer empirical AC algorithm for global and local aspects of unknown environments in the vehicle-road collaborative system path planning issues.The main contributions are listed as follows:(1)We study and analyze the current situation in the field of reinforcement learning and path planning at home and abroad,and sort out the advantages and development of various algorithms.Then we introduce reinforcement learning methods to provide theoretical foundation and strategy design foundation for agent strategy methods,along with the Actor-Critic framework as the research basis for the strategy and method of this article.Meanwhile,the concepts of global path planning and local path planning are introduced to decompose the path planning problem in the vehicle-road collaborative system,and the algorithm is improved and optimized in both cases.(2)We demonstrate the principles of traditional reinforcement learning and path planning algorithms,and propose an improved retrospective Q-Learning algorithm based on the Q-Learning algorithm for global path planning problems in static and unknown environments.Then experience playback and experience pool mechanisms is adopted to improved the convergence speed of the algorithm to quickly obtain the optimal value function and optimal strategy.The Q-Learning algorithm,SARSA algorithm,and improved retrospective Q-Learning algorithm are compared and verified on the designed map.It demonstrates the effect of the agent on path planning through three indicators,cumulative reward,success rate and running time per round.Experiments show that the algorithm can accelerate the agent converge to find the optimal strategy and path.(3)We introduce the asynchronous framework,independent target network and the idea of experience playback on the Actor-Critic framework to propose a multi-layered empirical AC algorithm.The experiments are verified on the built-in map of the simulation software CARLA,and the algorithm feasibility is evaluated through the reward function map,success rate,and collision rate indicators.Simulation experiments show that the improved multi-layer empirical AC algorithm can effectively solve the end-to-end local path planning problem in the unknown environment of a single agent,and has better performance than other deep reinforcement learning algorithms.