Research On Intelligent Driving For Large Scene With End-to-end Algorithm

The driving environment of intelligent car is complicated and changeable,and the safe decision-making requires a wide perceptual field to obtain rich traffic scene informa-tion.Traditional intelligent driving technology is usually divided into perception module and planning-decision module which involves large number of parameters.This disserta-tion focuses on the key technologies of end-to-end learning method in intelligent driving,whose goal is that the intelligent cars make rational decisions based on actual traffic sce-narios,just like human drivers.By expanding the perception field,the blind spots are eliminated and the moving vehicles and pedestrians are accurately detected with their moving trajectory well judged.Giving different attention to the features detected from the driving scene,the intelligent car is able to achieve safe and stable driving.The main work of this dissertation is summarized as follows:1.A fast video stitching algorithm based on spatio-temporal Bayesian theory for mov-ing objects.In actual driving scenario,the movement of vehicles and pedestrians is highly uncertain.If the view field is too small,the moving direction of the objects cannot be de-tected well.In addition,the movement of the object will bring occlusions,resulting in that the intelligent car can not accurately understand the driving scene ahead.Traditional video stitching method cannot deal with the problems such as motion occlusion,time-domain consistency and computational efficiency,which leads to the phenomenon of ghost,tar-get jumping and high time cost in the panoramic video.This paper proposes a fast video stitching method for moving object which specifically detects the potential occlusion in the overlapping area with optical flow,and the final fusion area was determined according to the occlusion map.With spatio-temporal bayesian fusion algorithm,the video is well stitched.Comparing with state-of-the-art stitching methods such as APAP,AutoStitch and SEAM,the proposed stitching method can effectively eliminate the artifacts,and the output panoramic video is obtained with high quality in real time.2.Large-scene feature fusion based end-to-end learning with spatial control for intel-ligent driving.By using only one forward-looking camera to capture the driving scene,traditional end-to-end intelligent driving algorithm holds the following two difficulties:(1)Blind spots make it impossible to accurately detect the position of moving objects at the edge of the view and fail to predict their trajectory?(2)The spatial position information is not used to give a certain constraint to improve the spatial feature detection ability of the model.To address the above issues,this paper proposes end to end driving model with large view feature fusion,which expands the intelligent vehicle forward view perception by using multiple cameras.By learning the importance of the multiple perspectives,it increases the proportion of key perspective by feature fusion in the model training pro-cess.Furthermore,we set spatial position constraint in the driving model.Comparing with the classical end-to-end driving model such as PilotNet,FCN-LST and DeepSteer-ing,the proposed method can eliminate visual blind areas,improve the ability of intelligent cars to accurately judge risks in time and ensure that intelligent cars can make appropri-ate decisions with more accurate spatial feature extraction.The RMSE of the steer angle prediction of the proposed model is lower than the best end to end method FCN-LSTM by 0.02.3.Surrounding view based multi-mode end-to-end algorithm with visual attention for intelligent driving.In the study of large-scene end-to-end algorithm mentioned above,with only the forward vision expanded,there are still existing serious problems in com-plex scenarios:(1)The lack of information on the left,right and rear view scene makes it difficult for intelligent cars to drive in cross sections and densely populated areas?(2)The lack of attention to the characteristics and positions of the key object makes it hard to im-prove the ability to understand the perceptual information?(3)Lack speed prediction for actual traffic condition.For these problems,this paper proposes surrounding view based end to end algorithm with attention mechanism.With 360-degree view information,the attention mechanism is added to the residual network to extract spatial features.Further-more,the multi-mode prediction including steering wheel angle and speed is realized.Compared with the traditional end-to-end models,which only take the front perspective information as input,the proposed method can drive stable and safely in complex traffic scene where vehicles turn around and pedestrians are more numerous.It can simulate human drivers to achieve accurate prediction for the steering wheel angle and speed.4.Study on the safety of driving decision algorithm for intelligent car.The safety of driving decision is the most basic requirement for intelligent driving.By expanding perception field,intelligent car can well understand and analyze the scene information to realize anthropomorphic driving.However,the accuracy of vision sensors has a great in-fluence on the scene understanding,a large number of visual errors will bring great hidden trouble for safely driving.This paper makes a preliminary study on the safety of intelli-gent driving decision,the driving process is modeled as markov process and the robust quadratic programming algorithm is used to approximately solve the bellman equation to obtain the optimal decision value that can tolerate state noise.From the experiments,we can see that the high operating efficiency of the algorithm can assist the intelligent car to make appropriate decisions in time and avoid decision making error caused by a delay in prediction.At the same time,it is robust to the observation noise and can make safe decisions even with biased observed state.