Research On Pedestrian Detection Method Based On Lidar And Image Information Fusion

In recent years,with the rapid development of automobile industry,unmanned driving technology has attracted more and more attention from researchers.Unmanned driving has four main modules: perception,decision-making,planning and control,among which perception is the basis for reliable operation of subsequent modules.Environment perception is to obtain accurate information of the environment by using sensors placed in fixed positions of the car body.Pedestrian detection is one of the most important parts in the environment perception system.Although image-based pedestrian detection has achieved good results,the imaging conditions of the camera are easily affected by illumination conditions,and cannot provide accurate location information.Lidar has good credibility and robustness,and can provide relatively accurate point cloud information.Therefore,in this paper,a pedestrian detection method based on LADAR and image information fusion is proposed to solve the problem of single sensor detection.The main research contents of this paper are as follows:(1)The pedestrian detection algorithm based on Faster-RCNN is studied.Firstly,after analyzing the typical structure of convolution neural network,aiming at the drawbacks of the basic Faster-RCNN algorithm,the algorithm is improved,replacing the original backbone network with the improved Res Net-34 network,introducing FPN(Feature Pyramid Networks)pyramid network to extract multi-dimensions at the end of the backbone network,and clustering the annotation boxes in nu Image data set to get an anchor which is suitable for pedestrian size.Experimental results show that the m AP(mean Average Precision)performance index of the improved method is improved by 10.86%.(2)Realize the point cloud data processing of LIDAR.The preprocessing of point clouds is completed by setting the region of interest,and the ground point clouds are segmented by using the improved RANSAC(Random Sample Consensus)theorem to reduce the number of point clouds.An improved DBSCAN(Density-based Spatial Clustering of Applications with Noise)algorithm is proposed,and the point clouds are clustered by improving the fixed threshold to an adaptive threshold that changes with the distance of obstacles.Experimental results show that the improved algorithm effectively reduces the problem of over-segmentation in the vertical direction of point cloud.(3)Pedestrian detection based on multi-sensor fusion.Firstly,after calibrating the sensor in space and time,the clustering results of LIDAR on pedestrians are projected to the image plane.Then,according to the independent detection results of LIDAR and camera,the independent detection results are fused at decision level by calculating the DIOU(Distance IOU)of the two detection frames.Finally,the point cloud information associated with LIDAR and camera is extracted to obtain the distance between pedestrians and vehicles.This paper uses nu Scenes data set to validate the proposed method in daytime and special driving environment.Experimental results show that the fusion method of LIDAR and camera for pedestrian detection has been effectively improved.