| Road detection is an important part of the autonomous driving systems.Reliable and accurate road detection is a prerequisite to many autonomous driving tasks,e.g.,path planning and decision making.As the basic task of the environment perception,road detection problem has been studied for many years.The existing road segmentation algorithms have already been able to get accurate road detection results.However,many algorithms cannot achieve a good balance between the road detection accuracy and the computational complexity of the algorithm,and the stability of the road detection algorithms also need further improvement.This paper focuses on the studies of structured road detection in the absence of high-precision map.According to the different types of sensor data,it can be divided into Li DAR based road detection and Li DAR-camera fusion based ones.The former is mainly aimed at the current situation that the accuracy of the Li DAR based road detection algorithm is not high.It studies from the two aspects,representation and processing of the Li DAR point clouds,to improve the road detection accuracy.The latter is based on the former,and makes further analysis and research on the Li DAR-camera fusion in road detection task.The main contents of this paper are as follows:(1)Aiming at the problem of low accuracy of Li DAR based road detection algorithm,a road detection method based on the histograms of the normalized inverse depth map and line scanning with Li DAR data only is proposed to improve the representation of Li DAR data and the road detection accuracy.In order to obtain a smooth and dense representation of the Li DAR data,the Li DAR point clouds are projected onto the camera's image plane.We can get a smooth and dense Li DAR data representation and a normalized inverse depth map through the triangulation based upsampling.For road detection,this method makes use of the distribution regularity of the actual road area in the horizontal and vertical histograms of the normalized inverse depth map,and obtain an initial road estimation result quickly.After this,the road detection accuracy is further improved by the optimization method of row and column scanning.Experimental results show that this method achieves the best road detection performance among all Li DAR based traditional road detection methods.(2)To solve the problem of relatively large computational complexity of method(1),a Li DAR imagery and triangulation upsampling based road detection method with Li DAR data only is proposed.By introducing the Li DAR imagery representation and improving the processing of Li DAR data,the proposed method maintains the road detection accuracy while greatly reduces the calculation cost of the algorithm.Li DAR imagery is a kind of image-like representation,which is based on the horizontal and vertical angles of each Li DAR points.In order to reduce the calculation cost in the road detection process,this method conducts road detection based on the Li DAR imagery representation,and gets the road detection result in the Li DAR imagery.Then,the road deteection result is projected onto the camera's image plane,and the dense road detection result in the camera image is obtained by a triangulation based filling approach.Experimental results show that the method can get the road detection accuracy close to the method(1),and the computational cost is greatly reduced at the same time.Under the hardware configuration of this paper,the dense road detection result in the camera image can be obtained at the rate of about 40 frames per second.(3)Aiming at the problem that many parameters need to be set artificially in the traditional road segmentation methods(1)and(2),a two-view fusion based convolutional neural network to estiamte road areas with Li DAR point clouds as input only is proposed.By introducing deep learning techniques,the proposed network reduces the design difficulty of the road detection algorithm and the influence of human factors on the road detection result.The proposed network takes two transformed Li DAR data representations,the Li DAR imageries and the camera-perspective maps,as inputs.It outputs pixel-wise road detection results in both the Li DAR's iamgery view and the camera's perspective view simultaneously,in an end-to-end manner.In order to make better use of the data association characteristics between the two Li DAR data representations,we introduce a novel view transformation layer,which can transform features from the Li DAR's imagery view to the camera's perspective view,so as to realize the data fusion between different perspectives in the convolutional neural network,and enhance the road edtection process in the camera's perspective view.Experimental results show that this network obtains the best road detection performacne among all methods Li DAR data only.Under the hardware configuration of this paper,the average processing time of each Li DAR frame is about 40 ms.(4)Aiming at the problem that most of the existing conditional random field fusion based algorithms are mainly dependent on the camera images and supplemented by Li DAR point clouds,a road detection method via a Li DAR-camera fusion in the multimodal conditional random field(MM-CRF)framework is proposed.By introducing the multi-modal conditional random field framework,the proposed mehod realizes the equal fusion of the Li DAR point clouds and camera images in the road detection task.The whole system consists of two parts,one is single sensor based road detection,the other is multi-modal conditional random field based fusion.In the single sensor based part,the road detection with Li DAR point clouds adopts method(2),and the road detection with camera images adopts a light-weight transfer learning based road segmentation network.In the multi-modal conditional random field ased fusion part,in order to solve the problem of data imbalance between different sensor data,we transform the road detection results based on single sensor to dense and binary ones in the camera's perspective view.After this,the two road detection results are fused equally under the multi-modal conditional random field framework to get the final road detection result.Experimental results show that this method can achieve accurate road estimation results in real-time.(5)Aiming at the problem that the current road detection algorithms cannot achieve a good balance between the accuracy of road detection and the complexity of algorithm calculation,a Li DAR-camera fusion based network is proposed.By improving the processing of Li DAR and camera data,the proposed network improves the road detection accuracy on the premise of real-time performance.The proposed hybrid network takes the Li DAR imagery and camera image as input,and can output road estimation results in both the Li DAR's imagery and camera's perspevtive views.For the Li DAR imageries and the camera images,the network uses two sub-networks with different structures to process respectively,and the data association between the two sub-networks is realized through a view transformation layer.It transfers high-level feature maps with rich semantic information from the Li DAR sub-network to the camera sub-network so as to relize the Li DAR-camera fusion in the camera's perspective view.Experimental results show that this network can achieve the best road detection performance among all real-time methods.Under the hardware configuration of this paper,the average processing time of each frame is about 25 ms. |
PDF Full Text Request