Research On 3D Object Detection And Lane Segmentation For Autonomous Driving Based On Multi-Sensor Fusion

To improve the safety and reliability of autonomous driving environment perception,more types and numbers of sensors are being used in autonomous driving perception technology.Due to the differences in the characteristics of different types of sensors,these sensors can complement each other to enhance the sensing performance of the system.In this paper,we start from the key technology of multi-sensor data fusion to improve the accuracy and redundancy of 3D object detection and lane line segmentation by fusing the information acquired by camera and Li DAR sensors in order to enhance the perception capability of autonomous vehicles under different lighting and weather conditions.The main research of this paper is as follows:(1)For Li DAR-based 3D object detection,a line beam-based dynamic downsampling method is proposed to generate pseudo-low-beam point clouds to train a low-beam Li DAR-based point cloud model that does not require additional data annotation.On this basis,an exponentially weighted pooling and sparse backbone network-based point cloud encoder is proposed for better generation of Bird’s Eye View(BEV)features.Finally,a center-based detection head is used to accomplish the 3D object detection task,and the excellence of the proposed algorithm is demonstrated by ablation experiments and real vehicle experiments.(2)For pure visual 3D object detection,we propose to transform multi-camera perspective features into BEV space by discrete depth estimation and context vector prediction of images,and use the same detection head as point cloud branching to accomplish 3D object detection.On this basis,deformable convolution is introduced in the backbone network and multi-axis attention is used in the Neck network to enhance the model feature extraction capability.Finally,the feasibility of the algorithm is verified by test set testing and in real vehicle experiments.(3)For multi-sensor feature-level fusion object detection,BEV features in BEV space fused image and point cloud data are proposed.To improve the quality of BEV feature maps for image branching,pseudo-depth maps are generated by introducing Li DAR data,which are used to optimize BEV feature map generation.On this basis,a dynamic attention structure is used to achieve feature fusion of different sensor channels while suppressing unimportant feature information.Ablation experiments demonstrate the effects of the pseudo-depth map and dynamic attention structure on model performance,and real-vehicle experiments also demonstrate the enhancement effect of multi-sensor fusion on single-sensor detection results.(4)For 3D lane line detection with multi-sensor fusion,a BEV fusion method using the same object detection is proposed.To reduce the computational effort of BEV segmentation,the BEV feature map is decoded by a continuously upsampled segmentation head to generate 3D lane lines under BEV.The comparison experiments demonstrate that the BEV feature fusion approach can effectively improve the segmentation performance of the model,and the real-vehicle experiments also prove the feasibility of the algorithm.