Research On The Decision Of Changing Lanes Of Driverless Vehicles In Urban Environment

Autonomous driving technology has always been an important future development direction in the automotive field,and with the rapid development of artificial intelligence technology,the realization of unmanned driving is one step closer.At present,unmanned driving decision-making is one of the most important problems in unmanned driving technology,especially driving decision-making on complex and unknown urban roads.Therefore,in order to solve this problem,it is necessary to adopt a kind of self-learning and in the complex and changeable.Safe and reasonable decision-making methods can be made under the driving environment,and according to the characteristics of deep reinforcement learning with independent learning ability and strong generalization ability,therefore,this thesis uses deep reinforcement learning to study the decision-making of lane-changing driving in urban environment problem.This thesis is based on the Deep deterministic policy gradient(DDPG)algorithm.The original DDPG algorithm has a large number of blind explorations in action exploration and low utilization of important sample data,resulting in poor lane-changing driving strategies and slow learning speed.Based on the original algorithm,a Deep deterministic policy gradient with expert guidance(DDPGWEG)algorithm was designed.First of all,the guidance for the output actions of the actor network has been added.It is through the expert network to obtain expert actions to guide the output action of the actor network and the update process of the critic network parameters at the same time,so that the output action of the actor network is more directional,can reduce the blind movement exploration in the initial training stage,and indirectly improve the quality of sample data.Secondly,the priority playback technology of experience classification is added.The priority playback of experience classification is to design two playback pools.The samples are divided into two pools according to the level of the reward value,and then in the initial training stage,in the high-quality playback Samples are collected in the pool.The samples with the same high reward value are more than the samples with low reward value,and then half of the samples are collected from the two playback pools according to the priority sampling method of TD-error sorting,which improves the utilization of high reward samples.At the same time,the diversity of the collected samples is increased,thereby improving the quality of the driving strategy learned by the self-driving car,so that the self-driving car can complete reasonable decisions through this algorithm.Finally,the design of each part of the DDPGWEG algorithm is completed by analyzing the characteristics of urban roads and the characteristics of vehicle lane changing.The simulation experiment part uses the TORCS simulation platform as the experimental platform to realize the simulation experiment of lane changing driving decision.The experimental results show that compared with the original DDPG algorithm,the DDPGWEG algorithm has improved the degree of the learned lane-changing driving strategy and the safety and driving efficiency of the vehicle lane-changing.The algorithm also has strong generalization and can complete lane-changing driving tasks.It is verified that the improved algorithm can make decisions safely and reasonably on the task of changing lanes.