Automated Lane Changing Of Intelligent Vehicle Based On Reinforcement Learning

In recent years,along with the problems of traffic congestion and road safety caused by the growth of automobile ownership,automobile intelligence has become a key research direction for the future development of the entire automobile industry.The current research on smart cars is mainly based on the rule design method,which establishes the behavior rules in the driving process of the vehicle through a priori "expert knowledge",and tests and verifies in specific scenarios.But for some complex scenarios or unexpected accidents,this method is difficult to verify through testing.Therefore,in order to cope with more complicated scenarios,with the help of agents' self-learning intelligent control algorithms that constantly interact with the environment,the intelligent vehicle has the ability to respond autonomously in complex scenarios.Reinforcement learning(RL)is a typical experiencedriven,self-directed learning method,which enables agents to find the optimal strategy for completing tasks through continuous "trial and error" and feedback learning in interaction with the environment For solving practical engineering problems that can be modeled as a Markov decision process.At the same time,deep reinforcement learning combines the perception capabilities of deep learning(DL)with the decision-making capabilities of reinforcement learning to form complementary advantages,which provides new solutions to the perception decision of complex problems.At the same time,it makes it one of the feasible solutions to solve the intelligent car's autonomous lane change.This article relies on the national key research and development plan "Key Technologies for Supporting Reliability and Environmental Adaptability Evaluation of Autonomous Driving Electric Vehicles"(No.2018YFB0105205),and proposes an intelligent lane change control method based on reinforcement learning.At the perception end of the agent,the state boundary is formed by radar detection lane boundary and the distance of surrounding vehicles,forming an intelligent vehicle autonomous lane change control "end-to-end" reinforcement learning framework with radar detection data as input.A reinforcement learning autonomous lane changing simulation environment that satisfies the requirements of this paper is established by Python,and the training and verification of the intelligent lane autonomous lane changing under the horizontal,horizontal,and vertical control are completed under this environment.Focusing on the research objectives of this article,the following research work has been carried out.Firstly,an "end-to-end" reinforcement learning framework for autonomous lane change control of Intelligent Vehicles Based on radar detection data is proposed.In the built simulation environment,the lane boundary line and the distance information of surrounding vehicles are detected by lidar to form the driving area of vehicle lane changing,which is directly used as the original data of reinforcement learning state space input.Secondly,use Python to build an experimental simulation environment and visualize the simulation interface through Pyglet.By comparing with the current mainstream reinforcement learning simulation platforms,in order to facilitate the acquisition of training data from the environment,an independent lane change reinforcement learning simulation experiment environment is built.According to the division of automatic driving scene elements,the road geometric information and road motion targets in the process of lane change are selected as the main research subjects.The agent in the environment solves the distance between the detected lane line and the surrounding vehicles through the basic theoretical method of the intersection of two straight lines in the two dimensional plane.By adding the kinematics bicycle model,the geometric motion constraints of the agent are satisfied.In addition,it also includes the detection of the abnormal behavior of the agent.Thirdly,through the custom-made lane-changing task goal,task goal switching logic and reward function,the design that comprehensively considers safety,comfort and efficiency.A deep deterministic strategy gradient(DDPG)algorithm is used.In the DDPG algorithm,the actor-critic network structure uses a fully connected neural network to output continuous action values.State space design uses the lidar in the simulation environment to obtain the distance between the agent and the surrounding vehicles and lane boundary in the scene as training data.The action space adopts continuous steering wheel angle and longitudinal acceleration as the output of decision action.Regarding the reward function,the consideration of safety avoids vehicle collisions and maintains a fixed distance from the vehicle in front,the comfort considers the steering wheel angular velocity and jerk value of the vehicle,and the efficiency of changing lanes and the minimum speed limit.Finally,for the problem of lane change under the condition of free traffic in the city,an autonomous lane change method considering the lane change moment in vehicle lateral control is proposed by reinforcement learning.Realize the anthropomorphism and performance requirements of the agent during the lane change process.At the same time,it also completes the complete phase of autonomous lane change by the lane maintenance,autonomous lane change,and lane maintenance after the lane change during the automatic driving process..Aiming at the problem of lane changing under the complicated traffic state of urban roads,a comprehensive control method of horizontal and vertical vehicle autonomous lane changing is proposed.The control of longitudinal speed is added on the basis of horizontal control,the safety,comfort and efficiency of the agent's autonomous lane change process are fully considered in the design of the reward function,and the agent training is completed in a dynamic scene.Experimental verification shows that the agent can reflect the differences in driving styles of different types of drivers at the time of changing lanes at different lateral speeds,and it has good robustness under the control of different speeds in the horizontal and vertical directions and different average speeds of road traffic,The DDPG-based model and the designed reward function can achieve higher safety,comfort and efficiency requirements during the lane change process.