Research On Front Object Detection Algorithm Based On Millimeter Wave Radar And Vison Information Fusion

With the successful application of autonomous driving in multiple scenarios,people’s demand for its safety is also increasing.Object detection is an important link in the perception of an autonomous driving environment,and it has a vital influence on whether the autonomous vehicle can drive safely.However,at this stage,the single-sensor-based object detection algorithm cannot meet the real-time and high-precision requirements.Therefore,the object detection algorithm based on multi-sensor fusion has become of great research significance.Considering the high cost of lidar and the sharp drop in robustness under severe weather,the object detection algorithm based on millimeter wave radar and visual information fusion highlights the broad application prospects.This subject researches the object detection algorithm of millimeter wave radar and visual information fusion,and proposes the SMOKE-Fusion algorithm.The image object detection network without anchor frame is applied to the visual information processing branch of the fusion algorithm,and the frustum method is used to associate the radar point cloud and image features,point cloud and image features are merged in the feature layer to generate more accurate object bounding box information.The specific content is as follows:(1)The visual information processing branch adopts the anchor-free frame image object detection algorithm SMOKE to quickly generate the preliminary 3D bounding box of the object by one-stage manner,which provides the possibility for the realization of autonomous driving in real time.At the same time,the smallest 2D box containing the projection of the3 D bounding box is obtained on the image to prepare for the subsequent generation of the frustum.(2)In order to solve the problem of the lack of radar point cloud information on the Z axis,unlike traditional solutions that directly project the radar point cloud to the image,the pillar shape is applied to the radar point cloud,which specifically sets the width,length and height expanded radar point cloud to a radar point cloud pillar,which provides a spatial shape closer to the real situation for the radar point cloud.(3)Improve the traditional frustum size based on the information of the smallest 2D box and 3D bounding box,generate a new frustum area of interest,correlate the radar point cloud pillars in the area,and take the closest radar point cloud pillar in the area Is the point cloud information that matches the object.At that time,the radar depth information feature and the image feature will be fused and used as the fusion feature input to accurately regress the head network to obtain more accurate 3D bounding box coding information.(4)For the proposed SMOKE-Fusion algorithm for front object detection based on millimeter wave radar and visual information fusion,test it on the novel nu Scenes dataset,and compare with other existing single-image-based algorithms 、lidar-based algorithms and fusion-based algorithms.Combined with the evaluation indicators of the nu Scenes data set,the SMOKE-Fusion algorithm is about 1.4% higher than the second highest algorithm in the nu Scenes detection score.In the error series of indicators,the average velocity error and the average attribute error are 0.536 and 0.130,which are the lowest values among all comparison methods.In summary,the SMOKE-Fusion algorithm has a certain improvement in performance,and the fusion of visual information and radar information reduces the value of some error items of the algorithm.