Vehicle And Lane Detection Based On Machine Learning And Multi-Sensor Fusion

The intelligent driving system provides a feasible approach for reducing the driving burden,avoid road accidents and improve driving safety.Being an essential part of the environment perception models for intelligent driving,vehicle and lane detection are becoming a popular research filed recently.Aiming to improve the robustness and accuracy of vehicle and lane detection performance under complex driving environment,this thesis presents an sensing framework based on multi-sensor fusion and machine learning.Given the sensing algorithm framework in this thesis,a car-mounted sensor platform is constructed fusing millimeter wave radar and camera.According to the working principle and mounted position of different sensors,the coordinate systems and their transformation models of different sensing modality were defined,simplified and calibrated.Meanwhile the temporal synchronization method for different sensors has also been designed.Vision based vehicle detection module was designed.A Cascade Adaboost classifier based on Haar-like features was trained for vision based vehicle detection.In vehicle detection the global sliding window scan method was adopted and the raw vehicle hypotheses were then verified by an novel validation method which considers both the vehicle location and dimension according to the prior knowledge and calibration model.Considering that the traditional training sample labelling method is labour-cost and slow,proposed a novel sample labelling method based on visual tracking and semi-supervised training.analysing the quality requirement of training samples and present visual tracking performance,developed an robust accurate long term visual tracker based on sparse optical flow and adaptive template update.Taking this tracker as an kernel,a semi-automatic vehicle sample labelling software was developed.Meanwhile,a semi-supervised training framework was also adopted during the training of the classifier.Based on the raw vision based vehicle detection results,an adaptive multi-vehicle tracker was developed in the image domain.An adaptive Kalman filer with online stochastic modelling was built for vehicle state estimation.An Global Nearest Neighboring(GNN)method with local validation was designed for inter-frame multi-vehicle association.To address the dynamically changing vehicle members,a temporal based Finite State Machine(FSM)was build for track management.The adaptive multi-vehicle tracker efficiently improve the detection performance through compensating the false negative,suppressing the false positive and improving the detection accuracy.Given the traditional lane detection methods are sensitive to interfering noises under heavy traffic scenario,a robust novel lane detection method was proposed.Instead of traditionally used edge feature,the blobs feature with higher dimensional information was used.To fully remove the vehicle interferences,a 3D mask generation method was proposed based on the local dense optical flow analysis and semantic prior knowledge.To enlarge the interclass difference between lane markers and non-lane marker,an associated dual space lane marker classifier was built to extend the dimension of lane marker descriptor.To adaptively estimate the classifier parameters,a From-Near-T-Far segmentation based lane trace method with adaptive K-Means clustering was developed.Finally,the lane marker features was used to fit the lane based on dynamical lane model switching between the straight line and Catmull-Rom spline model.A fusion framework between MMW-radar and vision modules was proposed.A multiobject radar tracker based on 4D Kalman filtering,Gating&Nearest Neighbouring,and a sliding window logic based track management was first built to process the raw radar data generating multi-object trajectories.The different sensing properties of camera and radar were researched through experiments.Based on the strengths and weaknesses of different sensors,the fusion method was designed at the object track level.Through the fusion of radar and camera,the overall sensing performance was efficiently improved in robustness and accuracy.