Research On Point Cloud-image Calibration And Fusion Method In Autonomous Driving Scene

In recent years,with the the environmental perception methods extended to threedimensional space,autonomous driving technology has been able to develop rapidly.Lidar is one of the most important sensors for autonomous driving.It can obtain target distance information and generate high-precision point clouds.However,a single-frame point cloud has two main drawbacks:(1)The information is single and lacks texture features.Optical cameras can obtain visual images with dense color information.Through calibration technology,point cloud-visual information can be integrated at the pixel level,breaking through the limitations of data;(2)Some targets are incomplete and distant sight clouds are sparse,through point cloud registration technology,multi-frame point clouds can be merged to achieve completeness and densification of point clouds.In this paper,we study point cloud-image calibration and point cloud alignment fusion methods in autonomous driving scenario,using LIDAR and optical cameras as environmental sensing tools,and the high-quality RGB-D data obtained through these two efforts are important for tasks such as target detection and identification.The details of the study are as follows:Due to the large difference between the 3D point cloud and the visual image,this paper proposes the concept of homogeneous features.Firstly,the two sets of data are converted into depth maps,and then the uniform angle neighborhood points are used to calculate the SLBP(sparse LBP)features and SVN(Sparse virtual normal)feature.After experiments,it is proved that SLBP and SVN feature maps have obvious homogenization effect on the original data,and have anti-rotation and anti-interference properties,respectively.To solve the problem that lidar and optical cameras require manual calibration for fusing point clouds and visual images,this paper proposes a convolutional neural network model,which is used to estimate the extrinsic parameters of the point cloud and the visual image,realizing the post-calibration of heterogeneous data.In the data processing stage,the non-weighted layer is used to extract the homogeneous features.In the network structure,a multi-level fusion matching layer is introduced to calculate the correlation of the feature map.Experiments show that the model in this paper can significantly improve the calibration accuracy and perform real-time operations.Aiming at the problems of slow running speed and poor alignment effect of traditional point cloud alignment algorithm in processing autopilot field point clouds,this paper proposes a point cloud alignment fusion algorithm based on the spatial structure similarity of visual key points.Firstly,SIFT+KNN algorithm is used to obtain 2D key point pairs of visual images,then the depth information of 2D key points is estimated and back-projected to 3D space,after that,the 3D key point pairs with the highest confidence are obtained by the spatial structure similarity algorithm to calculate the rigid transformation parameters.Experiments prove that the method in this paper achieves a substantial improvement in both the operation speed and the alignment effect.