Research On Object Detection Algorithm Based On Image And Point Cloud Fuseion

High-precision environmental perception is the technical basis for intelligent driving,and solving the problem of reliable environmental perception is an important prerequisite for ensuring the safety of people and vehicles in intelligent driving.With the continuous development of deep learning algorithms and sensor hardware levels,environmental perception technology based on cross-modal information fusion shows a wide range of application prospects.Image sensor data has rich color and texture information,and lidar can obtain accurate three-dimensional environmental information,and has stronger all-weather work capabilities.Based on image data and lidar point cloud,this thesis overcomes the shortcomings of single sensor data,fuses different sensor information,and studies the realization of environmental perception technology in outdoor scenes.The main work content is as follows:First,this thesis introduces the theoretical basis of multi-sensor data fusion,researches and analyzes the respective imaging principles of lidar and image sensors,and the mathematical basis of multi-sensor information fusion.Then for resource-constrained devices,the YOLOV4 object detection algorithm is optimized based on the lightweight network Mobile Net V3.The aim is to reduce the size of the model and increase the speed of inference when the accuracy of the model does not decrease significantly.The backbone network uses Mobile Net V3,using deep separable convolution instead of traditional convolution.The experimental results prove that the improved YOLOV4 algorithm model size,reasoning speed,and detection accuracy all achieve the expected goals.Then the image information is fused to improve the sparse point cloud upsampling algorithm.The original point cloud data collected by lidar is often sparse,making point cloud-based feature extraction and semantic perception difficult.This thesis obtains pixel semantics and instance information from the image segmentation task,and uses the mapping relationship between the point cloud and the image pixel to complete the point cloud segmentation,and perform upsampling of the point cloud of interest.Experiments have proved that the point cloud upsampling algorithm that integrates image information more completely retains the original point cloud three-dimensional information,and the operating efficiency of the point cloud upsampling algorithm has also been significantly improved.The upsampling point cloud has a significant promotion effect on the object detection task.Finally,object detection based on late fusion of industrial camera and lidar is completed.The hardware equipment chooses to use HORIZON lidar and Hikvision industrial camera,and uses Zhang Zhengyou’s calibration algorithm to complete the camera’s internal parameter calibration.According to the lidar data transmission protocol,the laser point cloud data acquisition and analysis are realized.Use the calibration board to complete the joint calibration of the camera and the lidar,and realize the multi-sensor data fusion.On the basis of completing the image and point cloud detection,a late fusion is carried out on the image and point cloud detection results.It is verified through experiments that the detection algorithm of fusion image information has obvious advantages over the single-sensor algorithm,and it can effectively avoid the missed detection of the object.