Study On Fault-tolerant Sensing Of Automatic Driving System Based On Generative Adversarial Nets

Typical automatic driving system uses sensors like cameras,lidars and millimeter-wave radars to perceive the circumstances,detect obstacles and extract lane lines.Surroundings of cars are very complex,Sensors,especially visual sensors may get no image data due to malfunction of camera devices,incomplete image data with part of camera lens shaded,or too light image because of sudden high intensity of light and over-exposure.In recent years image generation gains much attention as a new branch of computer vision,but applications and researches of image generation in automatic driving scenes are quite limited.So trying to deal with the accidental ineffectiveness of visual sensors in self-driving system,this paper proposes the concept fault tolerant sensing module of self-driving system and aims to provide some new ideas of image generation based applications in fault-tolerance module of self-driving system with modification of models,loss functions and training methods,specifically the lane line image generation and detection,video frame prediction,image restoration and SLAM fault tolerance based on view synthesis.The main researches and results of the paper are listed as follows:(1)Firstly a new and more robust lane line image generation and detection method is introduced.This method is mainly used in circumstances where there are no lane lines or they’re corroded and discontinuous.In these eases many traditional lane line detection algorithms are ineffective.To solve the problem,the Pix2Pix network is applied,then the generated image is utilized to detect lane lines.This is the first application of image generation technique in lane line image generation and detection.This algorithm can also be embedded into intersections between sensing modules and mature lane line detection algorithms of self-driving system seamlessly and promote fault tolerance of traditional lane line detection algorithms.Experiment on Prescan dataset proves the feasibility of the method and provides some engineering experience.(2)Secondly a new video frame prediction model is presented,dealing with the problem of low-quality images caused by occasional over-exposure or view shade.Anew feature loss function is added to the original model based on image pyramid and GAN network,thus ameliorating the problem of image blurring.Meanwhile,a new pyramid structure is introduced to reduce test time of the network and make real-time test possible,with sharing of feature maps and less parameters and computation.The predicted image realizes the fault tolerance of sensing module which aids the decision module of self-driving system.It can also be used as a predictive signal of the end-to-end deep reinforcement learning system and fine-tune the decision signal.Training and testing results on KITTI and Pacman datasets prove modified models can get cleaner images in shorter time.(3)Lastly a new way of monocular simultaneous localization and mapping is explored based on a modified image view synthesis model.This research deals with the problem of sudden ineffectiveness of one camera in stereo system and proposes a fault tolerant mechanism of generating that broken image to make sure SLAM works when the problem occurs.In this paper,inputs of a depth estimation network and its depth information,image reconstruction loss based on STN net and image inpainting loss based on image inpainting network are gradually added to the classical Pix2Pix net.Tests on the two common datasets KITTI and Cityscapes are performed and better image quality is obtained.In the post-processing part the original and new generated image are utilized as the input of the stereo system.Experiment results prove this method can realize pose estimation in a comparatively small error range and fault tolerance of SLAM based on image restoration is accessible.