Application Research Of End-to-end Model-free Deep Reinforcement Learning In Autonomous Driving

Autonomous driving technology has always been one of the directions that people continue to explore in the automotive field,and fully autonomous vehicles are the ultimate goal of autonomous driving technology.In recent years,with the development of artificial intelligence,autonomous driving technology has ushered in a new round of revolutionary improvement,and autonomous driving vehicles are moving from imagination to reality.Driving strategy is the core issue of autonomous driving technology,it requires high robustness to deal with various complex driving environments,but rulebased control strategies are difficult to handle.At present,the deep reinforcement learning algorithm that combines deep learning and reinforcement learning has made great progress in many fields,it learns itself through past experience,has strong robustness to various driving environments,and shows good performance when using highdimensional inputs such as images.In view of these advantages,this thesis applies deep reinforcement learning algorithms to the research of autonomous driving problems,the main research contents are as follows:(1)In view of the importance of the simulation platform to the research of autonomous driving,this thesis compares and summarizes the five commonly used autonomous driving simulation platforms in detail from the aspects of scene information,sensor configuration,vehicle control methods and algorithm application.According to research needs,two typical driving scenarios were selected for follow-up experiments.(2)The driving assistance system has been commercialized and applied to real cars.This thesis is based on the combination of various driving assistance systems to achieve automatic driving in the highway scene,and proposes a driving strategy that uses deep reinforcement learning to coordinate various driving assistance systems.This driving solution uses cameras and radar for environmental perception,and realizes collision-free,safe and high-speed driving of the agent in the simulated highway environment.As a comparison,this thesis also designed a scheme that only uses a camera and only a radar for environmental perception.The experimental results show that the multiple sensing scheme performs better and the system redundancy is stronger regardless of the presence or absence of noise.(3)In order to study the problem of autonomous driving in a more realistic environment,this thesis proposes an end-to-end driving strategy based on environmental information coding combined with deep reinforcement learning.The pre-training encoder compresses the action space of the agent to a reasonable range,and can improve its exploration efficiency.Compared to the pre-training imitation learning method,very little data is required to use the variational auto-encoder.It does not require complex image preprocessing technology,so it is easy to implement.Regarding the problem that reinforcement learning agents cannot turn according to the specified path at the intersection,this thesis proposes two reinforcement learning models based on turning instructions,which are called conditional reinforcement learning models in this thesis,namely command_input model and command_branch model.Experiments have shown that both of these models complete the 1,245 meters cruise mission on the Carla platform,conditional reinforcement learning model has strong robustness and certain practical value.This thesis realizes end-to-end autonomous driving in specific scenarios and unlimited scenarios based on the model-free deep reinforcement algorithm,which shows that the application research of deep reinforcement learning methods in autonomous driving has practical significance.