Research On Object Detection Method Based On Fusion Of Lidar And Millimeter Wave Radar

Autonomous driving technology is advancing rapidly with the strong support of government for the artificial intelligence industry.Accurate and fast detection of objects in road scenes by self-driving car is one of the most important factors to ensure the safe driving of vehicles.At present,the sensors carried by self-driving cars mainly include cameras,lidars and millimeter wave radars.Cameras are mainly used to identify traffic signs,lane lines,etc.,and other objects such as car and pedestrian in road scenes mainly rely on lidar for detection.However,most of the current object detection algorithms based on lidar data has two problems:On the one hand,lidar cannot adapt to complicated urban roads scenes,rains and fogs weather environments;on the other hand,the detection range of road scenes is large,which consumes a lot of computing labors and affects the real-time detection.In view of the above-mentioned problems of lidar in road scene object detection,this article proposes two solutions.First,the attention fusion method of millimeter wave radar and lidar is proposed to improve the detection accuracy in complex environments.This is based on the advantages of millimeter wave radar that has strong penetrating ability and is not affected by rain and fog.Second,based on the characteristics of millimeter wave radar that can instantly detect the approximate positions of most targets,a detection method that directly uses millimeter wave radar to generate sparse candidate boxes is proposed to improve the detection speed.The main work of this paper is as follows:1.Data preprocessing.Spatial alignment: By obtaining the rotation matrix and translation matrix of the lidar and millimeter wave radar with IMU as the reference,the sensor data is transformed into the same coordinate system to realize the spatial alignment of the two sensors data.Time alignment: Through the scanning frequency of the two sensors,the time alignment of the labeled frames of the two sensors is realized.Data aggregation: According to the needs of the experiment,multiple scan frames of lidar and millimeter wave radar are aggregated into labeled frames to obtain more dense point cloud data.2.Feature fusion algorithm design.Focusing on the problems with low penetration ability,weak long-range object detection ability and poor anti-rain jamming ability of lidar,an attention fusion algorithm of lidar and millimeter wave radar has been designed.The lidar data and millimeter wave radar data are converted into pseudo-images of the same size by discrete voxel encoding;then the feature maps of the two pseudo-images are extracted separately using the feature extraction network;finally,the feature maps of the two sensors are merged by using the attention mechanism.The feature attention fusion method is applied to improve the detection ability of the lidar for occluded objects,objects in the rain,and long-distance objects.3.Data fusion algorithm design.Concentrating on the problem that existing object detection algorithms need to traverse a large number of candidate boxes and consume a lot of computing resources,a radar region proposal network(RRPN)is designed that uses millimeter wave radar to generate sparse candidate boxes.The 3D candidate boxes are generated from the information of the millimeter wave radar,which are regressed and classified through the RRPN network.This method effectively decreases the complexity of the network and greatly reduces the huge amount of calculation when screening candidate boxes.4.Experimental design.This paper verifies the effectiveness of the two proposed fusion methods,and visually analyzes the results of our object detection fusion algorithm.