Heterogeneous Data Fusion-based Road Detection Using Lidar And Visual Information

The economic growth has witnessed a sharp increase in the number of the traffic accidents.It goes without saying that advanced driving assistance system and autonomous driving system play an extremely important role in ensuring traffic safety.Road detection is the basis of vehicle decision-making and execution,which provides a strong guarantee for the subsequent auxiliary driving or automatic driving.However,road detection usually faces much disturbance comes from weather,illumination change and shadow occlusion.It results in getting stable and robust road detection results hardly.With the coming of the big data era and sensor development,it is a trend to obtain more reliable results through multi-source data fusion.This thesis achieves a sable road detection method by fusing LiDAR and Visual image.The main research content is as follows:(1)For the problem of road detection,we preprocess LiDAR point cloud data and image data.Meanwhile,mapping relationship between the two data types is established to accomplish the data-level fusion of the two data types.Then,we reduce the interference of lighting changes on road detection based on image data through color space conversion;(2)To solve the problem that the effect and efficiency of road edge detection cannot be taken into account,a road edge detection method based on LiDAR point cloud turning is proposed.In the method,we combine with the characteristics of road area and the advantages of LiDAR point cloud that is not interfered by the light.We also add the point cloud turning and fitting the road edge adaptively based on the least square method with L2 regularization to realize a simple and efficient edge detection method;(3)According to the structural characteristics and data storage methods of LiDAR point cloud data and visual image,we select and design road features of LiDAR data and image data.In the meantime,we design road feature which combine the advantages of the two data types.In addition,deep abstract features are extracted based on FCN for full data mining.This method improves the robustness of road detection algorithm by not only combine the LiDAR and image,but also combine the shallow hand-made feature with domain knowledge and the deep abstract feature based on deep learning;(4)We build Mixed Conditional Random Field for road detection.In M-CRF,the unary potential of two data types based on the output of random forest.We combine the characteristics of the neighborhood of the LiDAR point and the pixel point to improve the pairwise potential,and design a mixed connection relationship to complete the construction of a mixed condition random field for realizing the decision level fusion of two data sources.Based on M-CRF,the road detection method of heterogeneous data fusion achieves the multi-level fusion of LiDAR point cloud data and visual image data.It is tested and verified on the KITTI datasets.The road detection method based on MCRF can obtain stable and robust road detection effect.