Research On Automatic Parking Algorithm Based On Reinforcement Learning

In recent years,with the rapid development of the automobile industry,the wave of "new four modernizations" has swept the entire automobile industry.As a key part of the "new four modernizations","intelligence" has naturally become a current research hotspot.Automatic parking technology is one of the research directions of vehicle intelligence.Its main function is to assist the driving behavior or replace the driver to park in the process of parking,It enables the driver to avoid the difficulty of parking due to the narrow parking space or the complex parking environment.In the environment where the parking environment becomes more and more complicated with the increase of car ownership year by year,the automatic parking algorithm has gradually become one of the research focuses of enterprises and universities.First of all,this paper expounds the current research status of automatic parking productization and automatic parking control strategies at home and abroad.Based on the current control strategy of automatic parking,an automatic parking algorithm that combines reinforcement learning with the control strategy of automatic parking is selected as the main research goal of this paper.In order to quantify the description of vehicle position and attitude in training and testing,the parking coordinate system selected in this paper combines the simplified Gaussian coordinate system with the traditional vehicle coordinate system.This paper simplifies the vehicle model based on the Ackerman steering principle,and establishes the vehicle parking kinematics model.The minimum turning radius is calculated using the established parking kinematics model,which can limit the layout of the parking lot in the following paragraphs.Secondly,this paper firstly introduces the basic theory of reinforcement learning from four aspects: agent,action,environment,observation and reward,and then introduces the basic theory of deep learning from the aspects of model and learning criteria.Since the action space of automatic parking is a continuous value,this paper chooses DDPG and SAC as the automatic parking algorithm in this paper.In order to make the description of the agent more accurate,this paper selects the real-time abscissa and ordinate of the vehicle,the real-time speed of the vehicle,and the heading angle of the vehicle as the vehicle state in the reinforcement learning elements.Based on the behavior of the vehicle in automatic parking,the front wheel steering angle of the vehicle and the acceleration and deceleration of the vehicle are defined as the vehicle action space in the reinforcement learning element.The reward function in the reinforcement learning element is defined from the perspectives of safety,attitude of parking in the berth,and comfort.Finally,this paper improves the highway＿env based on Open AI Gym to meet the simulation requirements of automatic parking.Then the characteristics of the environment and the vehicle are defined and the method of data collection is briefly introduced.In order to facilitate the evaluation of the training and testing process,this paper defines the evaluation indicators of the training,namely the total cumulative reward and the success rate,and also defines the evaluation indicators of the test,that is,the parking trajectory,the speed change of the vehicle,the acceleration change of the vehicle,the front of the vehicle.Rotation angle changes.The training in this paper adopts a step-by-step training method,firstly training in a simple scene,and then inheriting the trained agent into subsequent training.After completing the training of the agent,put the agent in the built automatic parking simulation driving simulator for testing,and then evaluate the test effect according to the evaluation indicators defined above.The test results show that both DDPG and SAC agents can be The parking is completed,and the effectiveness of the automatic parking control strategy is verified.