Research Of Fault-tolerant Perception And Control Method For Intelligent Vehicle Based On Deep Learning Methods

With the continuous development of hardware devices and the Artificial intelligence(AI)algorithms,artificial intelligence has penetrated into all fields of daily life.AI assistance,even completely autonomous operation become a trend.The Advanced Driving Assistance System(ADAS)equipped on the Intelligent Vehicles(IV)use kinds of sensors to collect data during the cruise.The system senses the environment in real time,detects and tracks the static and dynamic objects,and suggest the risks to human drivers previously with the information from GPS and electronic map,to improve the driving comfort and safety.ADAS is considered as the best way to overtake the powerful automobile country by the Chinese automobile industry.However,conventional ADASs are usually designed with the precondition that all the procedures are proper functioning.There is a lake of fault-tolerant mechanism in the practical application.The Tesla Accident that happened in July,2016 is because the ADAS mis-recognized the whit cart as the sky resion because of the dazzle.The Uber Accident in 201 8 crashed the pedestrian because the terrible visible in the night and the ADAS did not take in the foundation from the Lidar.To solve this issue,we introduce the idea of fault diagnosis,prediction and fault-tolerant control in the conventional ADAS frames,and modify the frame from"data collection—object detection—decision" to "data collection—data restoration/enhancement—scene prediction—object detection—decision based on reinforcement learning".The details are described as follows,1.To solve the issue that the noises in the camera data caused by the bad weather,such as haze and fog.Firstly,we proposed a modified real-time haze removal based on the Dark Channel Prior(DCP).The estimation to the parameters is optimized to achieve them in one traversal.The noises and errors are eliminated according to the channels with the statistics.The experimental results showed that the modified algorithm is 10 times faster than DCP-based algorithm.Secondly,we introduced deep learning method to restore the hazy images.To overcome the data loss in the procedure of the propagation in the deep hierarchical networks,we used shallow network and cascade restorations.Besides,the loss function was modified with the weighted sum of the Peak Signal to Noise Ratio(PSNR)and Structural Similarity Index Measurement(SSIM)The training dataset is constructed by the depth image dataset(Makc3D Dataset and NYU Dataset)and the hazy image model.The experimental results shows that the model achieve the balance between the computation speed and restoration performance Finally,to overcome the problems of small dataset and the weak generalization capability of the end-to-end methods,we use the Conditional Generative Adversarial Networks(CGAN)to restore the hazy images,and modify the loss function beyond the final differences.The experiments showed that the GAN can support enough generalization with small samples.2.To provide more basis for ADAS to make decisions,we use Recurrent Neural Networks(RNN)to predict the scene,including radio signals and video signals.The conventional RNNs has the gradient issue that the gradient explosion or vanishing may happen when learning long-term dependencies.The gradient issues are caused by Backpropagation Through Time(BPTT).Conventional solutions includes modifying the activation function,like ReLU,adding the L2-regularazation for weighted matrices.An improved method uses gates to control the explosions of gradient and memory,like Long Short-term Memory(LSTM)and Gated Recurrent Unit(GRU).In this paper,we proposed a Residual RNN(Res-RNN).The residual learning changed backpropagation from the continuous multiplication to the continuous sum,which makes the propagation as an identity mapping.The experimental results showed that the Res-RNN can provide the competitive performance with LSTM and GRU with less training time and testing time.To predict the scene for ADAS,the radio signal and the video signal were considered as time sequence signals.The full connections in Res-RNN were modified with convolutions to predict the 2D time sequence signals(Frame Prediction)as Res-rCNN.The experimental results showed that the combination of RNN and residual learning can solve the gradient issues,and predict the scenes fast and efficiently.3.To provide more information for ADAS and overcome the issues that the conventional detection algorithms only provided instantaneous information,we combined the sequence learning and object detection.This method can support radar sensors with information redundancy,even work in the radar sensor fails to replace them.We predicted the location information from a slice of a video.We used Kitti dataset and Prescan simulator,and relabeled them with the information of object locations,classes,and relative distances and velocities.To achieve the real-time detection results,we used one-stage model to balance the prediction speed and accuracy,and introduced the Res-rCNN into the model to learn the sequential information.To match up with the residual learning,the inception structures were introduced.The basic model was improved by a series of modification,and achieve a high performance4.In order to improve the decision-making ability of ADAS system,and overcome the issues of traditional decision making methods depends on the rules and end-to-end mapping,we proposed a control method based on the deep learning solutions.First of all,in order to overcome the issue that IVs can not understand the textual traffic laws,and the existing algorithms did not pay special attention on traffic rules,we proposed a reinforcement learning decision control method with textual boundaries.The algorithm combined characteristics of GAN network on unsupervised learning and the features of gradient optimization,and it can make decisions in the continuous domain In addition,the method introduced the concepts of natural language learning,and it make reinforcement learning learn textual traffic rules.In details,the neural network learning traffic rules through the acquisition of the positive and negative samples to understand the traffic rules text features,and it is introduced as a condition to train the neural network decisions made to achieve control target does not violate the traffic rules The experiments showed that the reinforcement learning method with text boundary can dive the car well,and following the text of law.Besides,in order to overcome the issue that the control methods based on deep learning were usually feedforward control,and status and instructions cannot be one-to-one mapping,we use content forecast for Segment 2,and introduce the predict bnetworks into the closed loop network for training to drive the system which provides more information and limitation,improved the driving safety.The experiments showed that the closed loop of end-to-end control method is more accurate,the input noise and stronger robustnessThe experiments in this paper used Prescan,Carla,and Kitti datasets to show the experimental results and performance of this method,and were compared with algorithm based on the current mainstream.In terms of data use,comprehensive use of public data sets and the simulation scenario can improve the training effects.