Research On Fusion And Real-Time Of Image And Point Cloud For Autonomous Driving

Autonomous driving is a significant direction for the intelligent and automatic development of urban traffic.The research on environmental perception algorithm is the key part of the autonomous driving system,the basic guarantee of system safety and reliability,and provides object information for decision-making and path planning.At present,it is difficult for a single sensor to deal with complex and changeable traffic scenes,so multi-sensor fusion has become a research hotspot.It is expected that the characteristics of different sensor data can be fused to complement each other’s advantages,so as to meet the requirements of safety and reliability of autonomous driving.Based on this,this thesis focuses on the fusion of image and lidar point cloud,and carries out method research,technology optimization and experimental verification on ke y technologies such as 2D object detection,3D object detection and 3D object tracking.The main research contents are as follows:(1)Research on 2D object detection algorithm based on model fusion.To address the difficulty of small objects and data imbalance in autonomous driving,a detection model is designed in this thesis based on model fusion and the complementarity of different Retinanet and MS-CNN detection models are utilized to improve the accuracy of small objects.The RetinaNet detector is optimized.K-Means is used to generate the anchor box.Mixup is used to enhance the data.Multithreading and multi-process are introduced to improve the running speed of the single model detector.Experiments on KITTI validation set and test set show that the algorithm can realize accurate identification and positioning of cars,pedestrians and cyclists,and provide a relatively reliabl e candidate region for 3D object detection.(2)Research on 3D object detection algorithm based on image and point cloud fusion.Frustum Pointnet only considers Euclidean distance in local region sampling,which may lead to the misclassification of point cloud,this thesis proposes an improved Frustum Pointnet network model,which introduces the dynamic abstract feature vector and the original static distance vector to constrain the local region sampling,and guides the network to learn the geometric features of similar points.A large number of experiments on the KITTI validation set and the test set show that the introduction of feature constraints can better extract the features of pedestrians and cyclists and achieve more accurate detection.In the KITTI test set,there was a 4.6% and 2.35% improvement in 3D object detection and location for the simple category of cyclists.(3)Research on 3D object tracking technology based on Kalman filter.In order to solve the problem of unstable detection in continuous multiple fra mes only depending on the detector,On the basis of single model 2D object detector RetinaNet and improved F-Pointnet 3D object detector,this thesis designs a 3D object tracking technology based on 3D Kalman filter.Qualitative and quantitative experiments on the KITTI object tracking validation set show that this algorithm can achieve real-time and fast object tracking.