Research On 3D Object Detection Of Self-Driving Vehicles Based On Image And Point Cloud Fusion

3D object detection is an important part of the autonomous driving environment perception system,which can obtain data such as 3D size and directional angle of the object,and the accuracy of detection directly affects the safety of vehicle driving.The two types of sensors,lidar and camera,have their own advantages and disadvantages,and they complement each other,Therefore,the research of 3D object detection based on lidar and camera fusion is of great significance to improve the perception ability of autonomous vehicles.Aiming at the problems of the existing 3D object detection algorithm based on lidar and camera fusion,such as insufficient fusion,rough fusion method,poor detection effect of far and small objects and objects with occlusion,a 3D object detection algorithm based on image and point cloud multi-view multi-stage fusion is proposed.The main research contents are as follows:(1)Based on the working principle of Li DAR and camera,a coordinate transformation model is constructed,and the spatial alignment of Li DAR and camera is realized based on the KITTI dataset,And according to the coordinate transformation relationship,the point cloud is projected to a 2D plane to generate both a bird’s eye view and a front view based on reflection intensity.To address the problem that the generated point cloud reflection intensity map is sparse,a method is proposed to dense the point cloud reflection intensity map by combining Delaunay triangular dissection linear interpolation and guided filtering,and experimentally verify that the proposed method significantly enhances the features of road objects and effectively fills the image voids.(2)To solve the issue of insufficient fusion between image and point cloud,this thesis adopts a multi-stage fusion method of "data-level fusion + feature-level fusion" to fuse the generated dense reflection intensity map and color image to generate RGBDense-Reflectivity(RGB-DR),and constructs a feature extraction network based on deformable convolution.In the feature fusion stage,to address the problem that the existing algorithms fuse in a crude way,an attention mechanism-based candidate region feature fusion module is proposed,which can adaptively adjust the importance of different features.(3)In terms of loss function design,for the problem that the regression loss function commonly used in existing 3D object detection algorithms does not consider the constraint between points and is inconsistent with the object detection evaluation criteria,this thesis extends the 2D DIOU loss function to 3D space.A joint regression loss based on 3D＿DIOU and Smooth L1 is proposed to further optimize the similarity between the detection frame and the real frame.(4)The proposed 3D object detection algorithm is experimentally validated in the KITTI dataset and compared with other methods in both qualitative and quantitative aspects.Experimental results show that the proposed algorithm achieves good detection results while ensuring real-time performance,especially for small and far objects and occluded objects.