Research On Imitation Learning And Its Applications In Autonomous Driving

Imitation learning(IL)is crucial to the artificial intelligence theory,which has a promising application in decision making and control areas including autonomous driving.In this thesis,based on the analysis of the research status of reinforcement learning(RL),IL and autonomous driving,both at home and abroad,the author further explores the fundamentals of IL and applies the extended generative adversarial IL and hierarchical IL to the simulations of autonomous driving,which has great theoretical and practical significance.The main work of this thesis is as follows:Firstly,the fundamentals of IL are summarized.The theoretical framework,Markov Decision Processes(MDP)is given.Important concepts such as entropy and divergence in information theory,as well as policy,value function and trust region policy optimization in RL are expounded.Secondly,a class of IL algorithms,namely inverse RL(IRL),is studied and implemented.Three baseline algorithms,including linear-programming based IL,maximum entropy IRL and maximum entropy deep IRL,are elaborated in detail,and validations of those three methods are conducted in the Grid World simulation.Thirdly,the generative adversarial IL is studied along with its extended variant.Integration of generative adversarial nets with maximum entropy IRL can directly learn a policy without explicitly acquiring a reward function which is extremely expensive.Moreover,combined with maximum mutual information regularization and reward augmentation,the generative adversarial IL is extended.The extended method manages to learn the policy to pass another vehicle from two-mode expert demonstrations in TORCS simulations.Finally,the hierarchical IL method is employed to autonomous driving in urban environments.The architecture of hierarchical IL is presented,which decomposes the learning process into two stages,i.e.,coach IL and apprentice IL.A simulation is carried out in the CARLA benchmark,whose results illustrate prominent advantages of the hierarchical IL approach.