Front-view Vehicle Detection Based On LiDAR Point Cloud And Visual Information Fusion

Vehicle detection is used to perceive vehicles around autonomous vehicles,which is significance to avoid vehicle collision.Therefore,vehicle detection is crucial for autonomous driving and assisted driving.However,the driving environment in reality,is complicated due to the diversity of scenes,day-night alternation,illumination changes,occlusion,and other issues that have brought significant challenges to the vehicle detection method.Multiple sensors are used together to make the data complement each other,so as to deal with the intelligent perception of driving scenes in complex environment.Using Li DAR point cloud data and front visual images,and based on deep learning network and related theories,we made an in-depth study of vehicle detection technology.The main research contents are as follows:（1）The advantages and disadvantages of using LiDAR point cloud data and visual images are analyzed.We conduct research on two problems: the sparse and disorderly point cloud data is difficult to use directly,and the large difference between point cloud data and visual images makes it problematic to fuse.We generate the forward-looking mask data and point cloud dual views by projecting the point cloud based on the coordinate transformation mechanism.Meanwhile,for the hole of point cloud view,we propose a densification method based on the weighted median interpolation,and finally,the point cloud data and image data are effectively fused,lays a foundation for the subsequent detection methods.（2）A vehicle detection network based on LiDAR and image fusion is proposed.The network takes the point cloud mask and visual image as input,and guides the network to focus on part of data by combining attention mechanism,the robustness of the method is improved.Aiming at the detection problem of difficult samples,we propose a special loss function（Our Smoth Loss,OSL）to make the network pay attention to the learning of difficult samples,so as to improve the detection rate of difficult samples.（3）A 3D vehicle detection method combining two-dimensional features and threedimensional features is proposed.Fusion attention mechanism is proposed in the method,and design RPN network based on multi-level fusion.Moreover,we propose a 3D feature reconstruction method based on three-view restoration theory,and we detect vehicles based on the 3D feature.The experimental results on the KITTI data set show that the method has a high detection rate in complex scenes.