An Object Detection Method For Road Scenes Based On Point Cloud-image Fusion

In the past few years,we have witnessed the rapid development of autonomous driving.However,due to vehicles’ complex and changeable driving environment,achieving fully autonomous vehicles is still daunting.3D object detection is essential to the automatic driving environment perception system.Compared with the traditional 2D image object detection,the 3D object detection task required in the field of automatic driving needs to provide the vehicle with objects such as vehicles and pedestrians in the surrounding environment.Meanwhile,It is also necessary to provide a series of surrounding environment information necessary for automatic driving,such as the spatial position and size of the object,so that the vehicle can perform more accurate decision-making and planning tasks.This thesis mainly focuses on the point cloud data obtained by the Li DAR sensor(Li DAR)mounted on the autonomous vehicle and the image data obtained by the visual sensor to study the 3D object detection method in the autonomous driving road scene.Based on analyzing the current 3D object detection algorithms,this thesis proposes two 3D object detection schemes based on deep learning,and experiments prove the effectiveness of the proposed methods.The main contribution of this thesis is as follows:(1)The main 3D object detection algorithms with different data are analyzed.First,the data employed to extract road scene information,including image data and point cloud data,are analyzed for their characteristics and representation method.Then,the algorithm structure of object detection is studied and analyzed at the network level;the advantages and disadvantages of various fusion methods are analyzed,and a feasible method route is determined for the follow-up research.(2)This thesis proposes a multi-attention 3D object detection method for Li DAR point cloud.In the process of point cloud feature extraction,existing point cloud object detection algorithms treat information of different feature channels indiscriminately,affecting the accuracy of detection results and increasing the amount of computation.To solve this issue,based on the Point Net network,this thesis improves the network in the point cloud feature extraction stage.Combined with the attention mechanism,the channel attention module and the spatial attention are introduced from the two feature dimensions of local features and global features.By the attention module,the neural network can learn the weight information of different feature layers,and complete the combination and application of the attention mechanism in the field of 3D point cloud object detection.(3)This thesis proposes a 3D object detection method based on fusion of point cloud and image data.RGB images contain rich semantic information and texture information and have irreplaceable advantages in object classification tasks,but image data lacks accurate spatial location information of objects.On the contrary,the point cloud data obtained by Li DAR can accurately obtain the spatial information of the object,which can form an excellent complement to the image data.Therefore,this thesis proposes a decision-level fusion object detection network on the basis of fusion between laser point cloud and visual image data.Compared with the object detection method based on a single data modality,the detection scheme using point cloud-image fusion can reduce the insufficient detection accuracy causing from the poor quality of data obtained by a certain sensor in special environments and improve the stability and reliability of the object detection network.