Research On Uncertainty Estimation For Objects In Autonomous Driving And Its Verification

With the development of artificial intelligence and big data,autonomous driving has become an inevitable trend in automobile industry.Current autonomous driving applications are still facing many challenges.One of the most important technical bottlenecks is the inability to guarantee reliable detection of objects in the scenarios.Current state-of-the-art object detectors inevitably make mistakes or even fail in many driving scenes,especially in open-set or adverse weather conditions.Due to the fact that most of these detectors are not able to provide the uncertainties in their results,information about these mistakes or failures cannot be propagated to the downstream modules of the autonomous driving system,which is unacceptable for safety-critical automoated vechiles,and has already caused catastrophic consequences.An important way to solve this problem is to perform uncertainty estimation for the detected objects in the driving scenes.Previous ways to describle the uncertainties in a detected object,such as classification confidence or observation noises,can only provide limited uncertainty information,in contrast,uncertainty estimation aims to predict comprehensive uncertainties by tracing the causes.With the increasingly prominent safety concerns in the deployment of autonomous driving,uncertainty estimation for objects in the driving scenes is receiving more and more attention.Currently,the uncertainty estimation theory is not very mature,and the research on uncertainty estimation for objects in autonomous driving is still in its infancy.According to the detection results of a given object detector,the objects relative to autonomous driving can be divided into detected objects and missed objects(false negatives)by the detector.For the uncertainty estimation of missed objects,the uncertainty estimation theory is not able to provide their existances or locations,while prior studies on online predicting missed objects rely heavily on empirical assumptions or handcrafted false negative features,and lack generalization facility for different object detectors.For the uncertainty estimation of detected objects,previous studies discussed the 2D object detection task and lidar-based 3D object detection,but has not considered the challenging monocular 3D object detection,despite the extensive need and efforts on achieving 3D object detection with low-cost cameras.These studies merely take complex weather conditions,such as heavy rains,snows or fogs,into consideration in their verification experiments.Besides,application of the estimated uncertainty in objects is still limited to the field of environmental perception,and has not been explored from the perspective of autonomous driving system by for example taking uncertainty into consideration in decision-making or path planning modules.This dissertation proposes an uncertainty estimation method for detected objects in the driving scenes based on Bayesian Neural Network and introspective learning,and verifies its effectiveness under adverse weather conditions,as well as its application in the decision-making module.To this end,this dissertation decomposes the uncertainty estimation problem into two parts: uncertainty estimation of the detected objects,and online prediction of the missed objects.For the uncertainty estimation of the detected objects,this dissertation proposes a merging strategy for MC-Dropout sampling approximation to Bayesian Neural Networks.Considering the huge performance gap between lidar-based 3D object detection and monocular 3D object detection,the proposed merging strategy chooses to gather the sampling results in a soft-clustering way,and then merges these clusters to obtain uncertainties by Bayesian Inference.This avoids the information loss during the uncertainty estimation process,which can be critical for monocular 3D object detectors.Besides,a weighted Monte Carlo Dropout uncertainty calculation method is proposed to extract uncertainties from limited sampling results,which reduces the dependence on sampling times.The effectiveness and its superiority to previous method is verified and discussed on the challenging KITTI dataset.For the uncertainty estimation of the missed objects,this dissertation proposes a general online false negative prediction framework based on introspective learning.In contrast to prior studies,this dissertation proposes to extract false negative features by Deep Convolutional Neural Network instead of using empirical assumptions or handcrafted features.Besides,this dissertation extends the original concept of introspection and proposes an introspective framework to train this network,which allows it to make object-wise predictions,and to have the generalization facility for black-box object detectors.The effectiveness and generalization facility of this framework is verified in both the 2D and 3D object detection tasks on multiple autonomous driving datasets.In order to verify the effectiveness of the above methods under adverse weather conditions,this dissertation utilizes Prescan software to develop synthetic autonomous driving dataset containing different weather conditions,such as heavy rain,heavy snow and thick fog.This avoids the time-consuming process of labeling and data collecting from the real world.Based on the synthetic dataset,the effectiveness of the above methods in improving the safety of autonomous driving is further verified.In order to explore the application of estimated uncertainties of objects in the decision-making module,this dissertation builds a software-in-the-loop autonomous driving system based on a joint simulation platform of Prescan,MATLAB/Simulink and Carsim.Based on Bayesian Decision Network,uncertainties of the objects are taken into consideration in the decision-making process,and the effectiveness of the application is verified in the representative intersection scenerios.