Research On Deep Transfer Learning And Its Application In Advanced Driver Assistance Systems

As vehicle usage continues to grow,there are increased concerns regarding the problems they create,such as fatalities and injuries.Advanced Driver Assistance Systems(ADAS)provide the early notification necessary for drivers to avoid dangerous or uncomfortable circumstances.Statistics demonstrate that even a simple Driver Assistance Systems,e.g.slowing down the vehicle speed when the pedestrians are detected nearby,could help to prevent drivers from serious traffic accidents.This makes ADAS one of the most popular research topics in the world.In this paper,we research on deep transfer learning and its application in three ADAS,i.e.(1)360?pedestrian detection,(2)traffic sign recognition,and(3)driving maneuver prediction.Recently,deep learning has achieved breakthroughs in many research areas,such as computer vision and natural language processing.However,the essential assumption of traditional machine learning(including deep learning),i.e.the distribution of the training data should be the same as that of the testing data,can not be satisfied in most ADAS applications.Because in the real-world driving circumstances,there are many variations that could cause the discrepancy between the training and the testing data,e.g.the traffic conditions,weather conditions,and the driving styles of the drivers etc..Moreover,it is difficult to train a deep model using the practice driving data that are generally insufficient labeled,extremely imbalance,noisy and multi-modal.Transfer learning(TL)has a much more loose assumption on data than that of the traditional machine learning.TL assumes that there is discrepancy between the source(training)and target(testing)data,e.g.the difference in marginal distribution,conditional distribution and data manifold etc..TL aims to explore the shared knowledge between the source and the target domain for solving the tasks defined in the target domain.Generally,the source data are welllabeled and the target data are unlabeled.In this paper,the crucial problems in the three ADAS applications are modeled into TL problems and addressed using TL methodology and theory.The specific research contents and main contributions of this paper are summarized as follows,(1)In this paper,we introduce a low-cost 360?pedestrian detection system,which is used to collect fish-eye pedestrian images data set.Since translation equivalence has proven to be an important property that could improve the performances of Convolutional Neural Networks(CNNs)in many computer vision tasks.We propose to learn distortion-invariant features for CNNs,which is equivalent to reconstruct the translation equivalence for CNNs in fish-eye images.Motivated by the imaging principle of fish-eye camera and transfer learn theory,the Spatial Focal Loss(SFL)is tailor-made for CNNs to explore the shared knowledge in different distortion domains.The experimental results demonstrate that SFL improved the performance of original YOLO model in fish-eye images through balancing the performances of YOLO in different distortion domains.(2)A traffic sign captured in different driving scenarios could be significantly different in appearance,due to the variations in the light conditions,the weather conditions,image resolution etc..In this paper,the environment factors that cause the shift between the source and target data are considered as the real-world noises,thus learning domain-invariant features are equivalently to learn features that are robust to real-world noises.Motivated by this idea,Stacked Local Constraint Auto-Encoder(SLC-AE)is proposed to reconstruct the clean data from the corrupted data for learning low-dimensional features that robust to two domain-specific noises,i.e.the noises in the source and the target domain.In order to strengthen model’s ability to unknown real-world noises,One-step Generative Adversarial Network(GAN)is proposed to learn low-dimensional features that are robust to a variety of unknown noises that generated by adversarial training.The experimental results evaluated in five transfer learning data sets demonstrate that the proposed methods are effective not only for recognizing traffic signs captured in different driving scenarios,but also for recognizing other general objects.(3)Driving maneuver prediction is one of the most important tasks in ADAS,it provides the early notification necessary for ADAS to detect dangerous circumstances and take appropriate actions.We introduce a driving maneuver prediction system used for collecting driving maneuver data,including the front view images and the necessary OBD(On-Board Diagnostic)data.In order to deal with the high-dimensional multi-modal data,Scene-Model-Transfer Driving Maneuver Prediction system(SMT-DMP)is proposed to transfer the low-dimensional scene and trajectory features for driving maneuver prediction.However,the scene features used in SMT-DMP are meaningless to human being,which makes the diagnose of end-to-end architectures extremely difficult.We propose to transfer driver’s attention as an intermediate concept to guide models in learning causal relation between driving surrounding and driving maneuvers.The experimental results conducted on the data set that contains 1078 driving maneuver samples,demonstrate that transferring scene features and driver’s attention are able to boost the performances of deep models in driving maneuver prediction.