End-to-End Driving Decision Algorithm And Simulation Implementation Based On Spatio-Temporal Features

The goal of the intelligent vehicle decision-making system is to ensure safe and stable vehicle operation by integrating the environment and inner-vehicle information.In order to learn to drive on the road,the traditional method utilizes multi-sensor target detection and fusion technology to acquire the surrounding environment,and then sends the processed key information to the decision-making layer,finally,the appropriate path,speed and other information will be sent to the control layer.End-to-end method simplified system structure by learning a single network which maps the sensory information to steering angle,directly calculates the decision value based on the input images and unify the cognitive process into the decision process.Traditional intelligent vehicle decision-making algorithms in feature extraction and the existing end-to-end decision-making algorithm do not consider the vehicle dynamic information.This paper proposes an intelligent vehicle end-to-end decision-making method based on spatiotemporal joint depth residual network,which accurately predicts the steering wheel angle value.This thesis mainly does the following three tasks:(1)Traditional end-to-end driving decision algorithm cannot well get the spatiotemporal joint information of the historical input images,we propose a deep residual network algorithm based on spatio-temporal joint feature.The proposed algorithm combines the spatio-temporal joint depth Residual Network(ResNet)and Convolutional Long Short Term Memory Network(Conv-LSTM),which accurately predicts the steering angle of the current time of the intelligent vehicle by extracting the spatio-temporal joint features of several history frames.(2)Considering of the singularity of the input information of the traditional imagebased end-to-end driving decision algorithm,this paper proposes an end-to-end driving decision algorithm that combines vehicle dynamics information.Based on the abovementioned spatio-temporal joint depth residual network structure,the proposed decision algorithm incorporates historical vehicle dynamics information(such as speed,position information,vehicle attitude,historical steering angle,etc.)into its prediction network,which predicts current moment of the steering angle of the intelligent vehicle more smoothly and accurately.(3)The end-to-end driving decision algorithm proposed in the paper has been tested on the public driving decision database Udacity Challenge-II and the driving decision data set collected in the AirSim simulation environment,and compared with some traditional end-to-end driving decision algorithms(such as PilotNet,CgNet,etc.).The experimental results demonstrate that the proposed decision algorithm shows better performance in the prediction of steering wheel angle of intelligent vehicles.Finally,the driving decision algorithm proposed in the paper verified on the simulation environment,which collects the dataset in Qingshuihe campus of the University of Electronic Science and Technology.The end-to-end driving decision algorithm studied in this paper extracts the spatiotemporal joint features of several historic frames,incorporates the vehicle dynamics information into the predictive network,which improves the prediction accuracy on the current steering angle and provides the rationality of technical basis for the application of intelligent vehicle in complex scenes.