The Research On 3D Vehicle Object Detection Algorithm Based On Stereo Vision

Vehicle 3D detection is an important part of the development of Intelligent Transportation.It plays an important role in automatic driving,assisted driving,and Intelligent transportation systems.In an automatic driving system,vehicle-mounted cameras and lidar collect scene information from the driving view,which enables the vehicle to acquire 3D world information in real-time.Color image data contains a large amount of visual information,but it has the disadvantage of scale uncertainty and cannot obtain spatial information.Lidar can detect spatial information accurately,but it lacks color information and the data is sparse in spatial distribution.To detect the vehicles in the road environment better,this paper proposed a vehicle 3D objects detection method for automatic driving.In theory,the method is based on computer vision and deep learning theory.In technology,the method uses the data of binocular camera and lidar sensor to detect the 3D vehicle objects through multi-information fusion technology.This paper studies on KITTI data set,and the method of this paper mainly includes three parts.In the first part,this method uses binocular vision technology to estimate the disparity and get disparity map of the left view.The disparity map is transformed into depth map and then into binocular point clouds.In the second part,the binocular point cloud data and the lidar point cloud data are fused to get more comprehensive point clouds data.The fused point clouds are used to detect vehicle objects by the 3D detector to get 3D bounding boxes.The main research work of this paper includes the following four aspects :Firstly,this paper uses the AANet+ model to estimate the disparity of binocular images.The process of disparity calculation is divided into four stages: feature extraction,matching cost calculation,adaptive cost aggregation,and disparity refinement.To get depth better through disparity,this paper takes depth error as the loss value for model training.After disparity and depth estimation,each pixel in the left view can be mapped to 3D space to get the point cloud.Secondly,the point clouds created by binocular disparity and the point clouds created by lidar will be fused.The fused point clouds are used for vehicle 3D detection by Voxel R-CNN.In this process,the point clouds data will be transformed into voxel,and the spatial features will be extracted to get the Ro I.After further inference,the 3D Bounding Boxes of objects will be obtained,which including coordinates,size,and global orientation angle.In this paper,the original Voxel R-CNN model is improved by introducing residual connection channels.The performance of the model is enhanced.The experimental results show that the 3D detection method is effective.And it reached 92.88% AP for moderate difficulty objects in KITTI.Thirdly,for 2D target detection,this paper introduces the development of 2D objects detection methods and studies the characteristics and structure of YOLOv5.To make full use of the advantages of the YOLOv5 network,this paper optimizes the deployment of yolov5-x.Experimental results show that after the deployment in this paper,yolov5-x has high performance,and it reached 95.2% AP for car detection.Fourth,to combine visual and spatial information,a joint detection method is proposed to integrate the results of 2D and 3D detectors.The method calculates the overlap of the 2D bounding box and the 3D bounding box.Then the results of 3D detection are adjusted according to the overlap and the confidence score.The experimental results show that the joint detection method can effectively improve the accuracy of 3D detection.After joint detection,the average precision of moderate difficulty objects in KITTI is improved to 93.17%,and other indicators are also improved.