Pedestrian Detection Based On Data Fusion Of Image And LiDAR Point Cloud

The driverless vehicle realizes information interaction with the surrounding environment through a variety of on-board sensors,controllers and actuators to achieve safe and efficient automatic driving.Among them,the environment awareness module,as an important part of the unmanned driving technology,is mainly aimed at timely and accurate perception of the surrounding environment,ensuring the reliability of driving and reducing accidents.In the current environmental awareness schemes,the limitations of a single sensor can no longer meet the growing awareness needs,so multi-sensor data fusion is needed to make up for the deficiencies,so as to obtain more abundant environmental information and improve the robustness of the sensing system.In this paper,we use monocular camera and LIDAR to complete pedestrian detection research based on image and LIDAR point cloud data fusion.First,we jointly calibrate the monocular camera and LIDAR sensor.Firstly,Zhang Zhengyou calibration method is used to calibrate the monocular camera to obtain the camera’s internal orientation parameters.Secondly,the Pn P algorithm is used to solve the corresponding points from 3D space to 2D image,and the rotation matrix R and the translation matrix T between the laser radar coordinate system and the camera coordinate system are obtained through feature point pair matching to complete the external parameter calibration of the two sensors and achieve spatial synchronization.Next,the time synchronization mechanism in ROS is used to match the timestamp according to the nearest neighbor to achieve time synchronization.Finally,the actual field experiment is designed.The re projection error of the LIDAR point cloud data projected onto the corresponding image is 0.38 pixel,and the calibration effect is good.Then,on the basis of Velodyne 16 line LIDAR data acquisition and point cloud filtering,the grid height difference method based on projection is improved,the height variance information is introduced,and a processing scheme for rapid ground point cloud removal is proposed.On the basis of obstacle clustering,the three-dimensional bounding box of pedestrian target point cloud is constructed based on the minimum bounding box algorithm of aerial projection,and pedestrian target detection is realized based on LIDAR data.At the same time,the generated countermeasure network GAN is used to remove the noise of image data.Based on the principle of real-time and accuracy balance,YOLOv3 target detection algorithm is improved from three aspects of expanding the data set,improving the classifier and obtaining the optimal anchor frame to achieve pedestrian target detection based on image data.The field experiment results show that the average accuracy of pedestrian target detection is improved by 2.6% with the improved YOLOv3 algorithm.Secondly,a hybrid data fusion framework based on decision level and pixel level is designed.First,the point cloud data in the three-dimensional bounding box is projected to the image pixels using the rotation translation matrix obtained by joint calibration,and is associated with the image target detection box.Then,according to the data structure of the red black tree,the category information of the image is inserted into the corresponding point cloud data to expand the channel of the point cloud data.Finally,the pedestrian point cloud is identified,and the point cloud data including position,length,width and height information is output.The field experiment results show that the accurate three-dimensional bounding box and category information of target pedestrians can be obtained by pedestrian detection under the fusion of LIDAR and monocular camera.Finally,based on the fusion detection results of image and LIDAR point cloud data,the pedestrian tracking is realized by using Kalman filtering algorithm on the basis of Hungary’s maximum matching.The field experiment results show that the pedestrian ID number of the tracked target has not changed,and its motion track is clear and accurate.