Research On Virtual Unmanned Vehicle Control Based On Deep Reinforcement Learning

As the bottom-level module of self-driving system,control system is the key for the safety and comfort of self-driving.Popular control method is model-based with control theory,and-needs-tune parameters by hand.Model-based control method has been widely researched and is known as its interpretability.However,for complex road environment,the parameters would be very complex and hard to tune.What's more,traditional controller is enable to adaptive learning.Althrough there are some adaptive method for parameter tuning,they are always limited for their representation ability,and are not robust for the change of road.To solve above problems,this paper makes two contributions to improve the control performance:First,traditional control method is substituted by deep reinforcement learning,and this paper proposes to use model-free,self-adaptive DDPG for lateral control and longitudinal control.DDPG learns control policy by trial-and-error without vehicle dynamic model and environment model.DDPG interacts with environment and is more robust for the change of environment.This paper validates the performance of synchronal control of lateral and longitudinal direction with DDPG on TORCS driving simulator.Results show that the tracking error is in reasonable bounds.Second,deep reinforcement learning needs large number of trials and some of them are dangerous,especially for self-driving task.This paper proposes to accelerate the training with little prior knowledge to reduce the number of trials.Particularly,a supervisor designed with little prior knowledge will guide the training of DDPG.This model is called Supervised DDPG.This paper compares three models:Supervisor(here is feedback controller in this paper),DDPG,and Supervised DDPG on the task of lateral control.The experiment results show that Supervised DDPG has better control performance than Supervisor.Compared with DDPG,Supervised DDPG has a faster convergence rate and reduces the trials.