Research On Reinforcement Learning Algorithm For Longitudinal Control Of Autonomous Vehicles

Autonomous driving system is a comprehensive system that integrates functions of perception,planning and control.In recent years,with the development of artificial intelligence technology,machine learning has been gradually introduced into the design of autonomous driving system.This research relies on the National Ministry of Science and Technology project "Research and Demonstration of Key Technologies of Electric Autonomous Vehicles",which aims to further improve the longitudinal control algorithm by combining reinforcement learning methods and autonomous driving technologies.The main research contents are as follows:(1)Longitudinal decision framework for self-driving carsFirstly,the rules-based model in longitudinal control of autonomous driving is explained,and then two reinforcement learning methods based on value function and policy are introduced.Based on this,two kinds of deep reinforcement learning algorithms suitable for autonomous driving task scenarios are introduced.Finally,based on the basic theory of autonomous driving longitudinal control and reinforcement learning,the design of a longitudinal decision framework for self-driving cars based on reinforcement learning is completed and applied to the sequential decision-making tasks in a high-dimensional state action space.(2)longitudinal control algorithm based on deep reinforcement learningIn order to show the humanity and personality of the system as much as possible,a driving simulator is selected as a data collection platform.The driver's real driving data,processed by Kalman filter,can represent driving characteristics for final test verification.Markov decisionmaking process is first modeled for the longitudinal control task.Several features are selected as elements of the state and action aggregation.Based on the inverse reinforcement learning algorithm,a reward function model is established and used for the action value function and the policy network.Then parameters for the model network structure are designed,and the Deep Deterministic Policy Gradient(DDPG)algorithm for longitudinal control is completed based on the state and action aggregations and reward function.After the expected acceleration is obtained from the upper-level model,the controller in the lower-level is performed based on inverse longitudinal dynamic model for vehicle,which establishes the relationship between the decision model and controller.Finally,a simulation test environment is built in Carsim,which helps to test the reliability of the model.(3)Experiment and verification of longitudinal control algorithmBased on the Simulink/Carsim simulation platform and Haval H7 wire-controlled intelligent platform,the long longitudinal control algorithm itudinal control algorithm designed in this paper is verified.By selecting typical driving conditions in daily traffic scenarios and comparing the actual driving data with the test results under system control,the effectiveness and reliability of the algorithm are verified under several control states such as constant speed cruise and target following.