Research On Behavior Decision-making And Motion Control Methods Based On Imitation Learning For Intelligent Vehicles

With the rapid development of artificial intelligence technology and the increasing demand for intelligent vehicles,intelligent driving technology has become one of the most popular research topics.The key technologies of intelligent driving include environment perception,behavior decision-making,path planning,motion control,etc.Among them,behavior decision-making and motion control technologies are the key indicators to measure the autonomy level of intelligent driving vehicles,which is the focus and difficulty of intelligent driving research.At present,environment perception technologies of intelligent vehicles have achieved a large amount of research results.However,behavior decision-making and motion control methods of vehicles still rely on manual prior knowledge to design expert rules or models,and their adaptability to complex environments is far from competent.In dynamic and complex urban driving environments,traditional methods for behavior decision-making and motion control are facing increasing challenges.At present,the applications of machine learning techniques such as imitation learning and reinforcement learning to driving behavior decision-making and motion control have become a promising trend to overcome the problems that the current behavior decision-making and motion control methods are not highly automated and too dependent on manual experience.Among them,imitation learning can model the rich experience data of human drivers as a driving behavior decision-making or motion control model with good performance.This effectively solves the problems that traditional behavior decision-making and motion control methods rely on manual prior knowledge.Additionally,the shortcomings that reinforcement learning requires too much trial and error training and convergence is difficult are avoided.The most commonly used imitation learning approach is supervised policy function regression methods(such as behavioral cloning),as well as inverse reinforcement learning methods and direct policy interaction learning methods.The supervised policy regression methods are different from general supervised regression methods.It performs regression and modeling on a policy space with continuous action and timing decision characteristics,and needs to achieve efficient feature extraction and iterative optimization of timing regression models to obtain good performance.Therefore,the existing imitation learning methods still need to be improved in terms of learning accuracy and generalization performance.This paper mainly studies behavior decision-making and motion control methods of intelligent vehicles based on imitation learning for the aforementioned problems.Therefore,it has important significance in both theory and application.The main research work and innovations of this paper are as follows:(1)A driving behavior modeling method based on bagging gaussian process regression is proposed.This method not only utilizes the efficient learning characteristic of gaussian process regression method based on bayesian inference and kernel function mapping,but also utilizes the advantages of that fact that Bagging method improves the learning accuracy of driving behavior modeling by using multiple models parallel training and the ensemble policy.Three typical driving scenarios which are lane following,overtaking on a straight road and avoiding obstacle while turning are built to train and test different driving behavior modeling methods on the advanced simulation platform Pre Scan for autonomous driving.The results show that the proposed driving behavior modeling method based on bagging gaussian process regression can further reduce imitation learning errors and improve the generalization performance of the learning model compared with methods like multi-layer back propagation,regression tree ensemble,support vector regression and gaussian process regression.(2)A convolutional neural network gaussian process regression(CNN-GPR)method for driving behavior imitation learning is proposed to tackle the problems that full connection layers of the end-to-end convolutional neural network(CNN)have limited generalization ability and easily converge to local optimization.At the same time,Pilot Net-GPR and Res Net-GPR algorithms based on the CNN-GPR method are designed.The proposed method uses the gaussian process regression method with global mapping capability and better generalization ability to improve fully connected layers of the end-to-end CNN in order to complete the learning from features extracted by CNN to driving actions more efficiently.The verification experiments on the end-to-end autonomous driving dataset named Apollo show that the proposed Pilot Net-GPR and Res Net-GPR algorithms can further improve the imitation accuracy for end-to-end driving behavior learning and promote generalization performance of the learned model compared with the Pilot Net and the Res Net methods.In particular,compared with the Pilot Net network designed by NVIDIA,the proposed Res Net-GPR method reduces the mean square error(MSE)on the test set by 16.27% and the mean absolute error(MAE)by 9%.(3)A deep neural network gaussian process regression(CNN-LSTM-GPR)method of driving behavior learning with time-sequential images is proposed in order to further improve the accuracy of driving behavior learning through time-sequential information.This method utilises the gaussian process regression method to improve the structure of fully connected layers in the cascaded deep neural network(CNN-LSTM)in order to make a more efficient learning approximation for the mapping between the features extracted by the cascaded deep neural network and driving actions.The verification experiments on the Apollo end-to-end autonomous driving dataset show that compared with related end-to-end imitation learning methods based on single image under the same conditions,the proposed method can make full use of the time-sequential information of images which leads to smaller imitation errors.The proposed CNN-LSTM-GPR method for driving behavior learning can further enhance the learning accuracy when compared with the CNN-LSTM method.Additionally,it demonstrates a more satisfying generalization performance.(4)The CNN-GPR method and CNN-LSTM-GPR method for driving behavior learning are tested and verified by using the autonomous vehicle platform built in our laboratory.A data collection module is built on the autonomous vehicle platform HQ E-HS3 and a dataset of driving behaviors on urban roads is collected.Then performance evaluations of proposed methods are performed on the collected dataset,further verifying the effectiveness of the CNN-GPR method and CNN-LSTM-GPR method for driving behavior learning.