Traffic Vehicle Detection Based On Image And Laser Point Cloud Fusion

In recent years,with the development and maturity of artificial intelligence technology,it's applications have become more and more widespread,and autonomous driving has been an important research in the whole world.Vehicle detection is the first task that autonomous driving needs to complete,and fusing multiple sensors is the mainstream research direction in the future.Because the existing detection solutions based on multi-sensor fusion cannot meet the application requirements,this thesis proposes a new vehicle detection solution.In this thesis,the vehicle detection solution includes four details: the main network structure,feature extraction network,fusion algorithm,and detection output network.In order to ensure the detection accuracy,this thesis designs a main network based on two-stage,and makes two predictions.For the image and point cloud data,this thesis designs the feature extraction network separately.The image data uses an improved VGG16 network structure.The point cloud feature extraction network is designed in two parts,including a voxel feature learning network and a feature extraction network.The voxel feature learning network is designed for the three-dimensional data,and its purpose is to preserve the original spatial information.The point cloud is rasterized on the top view,and counts the coordinate information of each point in each grid.The neural network is used to learn the feature information of each point in the grid and aggregate the features to get the point-to-point features.A feature extraction network based on 3D convolution and 2D convolution is designed to complete the feature extraction.For fusing the image and point cloud feature map,a fusion algorithm based on spatial transformation is designed.The algorithm projects the image features onto the bird's-eye view of the point cloud and merges with the features of the point cloud,and obtains the fused feature map.In this thesis,the candidate region generation network is used on the fused feature map to obtain candidate anchors.Finally,a detection output network is designed to make a second prediction of each candidate anchors,and output the classification and position information.Evaluate the detection method on the public data set,the results shows that in the vehicle detection based on multi-sensor fusion,this thesis has obvious advantages on accuracy,and can meet the application requirements.