Research On 3D Object Detection And Tracking Algorithm Based On Binocular Vision

Environment awareness system in intelligent driving processes the environmental information acquired by the camera,millimeter-wave radar,lidar and other sensors to obtain the relative position relationship between the surrounding objects and the vehicle,which provides an important basis for vehicle path planning,decision-making and control.In the face of the complex and changeable road traffic environment,the point cloud data acquired by lidar lack stability,and the image data acquired by camera lack 3D information,making it difficult to achieve ideal results in 3D object detection.Accurate identification and detection of object vehicles has become a key technique in the environmental perception system.Therefore,it is of great significance to develop accurate and stable 3d object detection and tracking algorithms.In this thesis,vehicles in road traffic environment are taken as research objects.By improving the 2D object detection network Faster R-CNN and combining binocular vision,a 3D object detection network was established.In order to reduce the occurrence of false detection and missing detection,combined with Kalman filter,a multi-target online tracking algorithm was proposed to realize accurate recognition and tracking of vehicles.The main works are summarized as follows:(1)Build binocular vision system.Based on the camera imaging principle and binocular vision ranging principle,the appropriate hardware and software platform and binocular camera are selected to build the binocular vision system.Zhang's calibration method was used to calibrate binocular cameras,and the internal and external parameter matrices and distortion coefficients were determined.Then the left and right images were distorted and horizontally corrected according to the calibration results.(2)Central Point Networks(CPNet)were established for 3D target detection.According to the space vehicle detection scene,the Faster R-CNN network structure was improved,and a feature extraction network based on multi-scale output was established to solve the problem of long-distance vehicle detection.Aiming at the problem of low positioning accuracy of space vehicles,a projection center point regression branch was established,and a 3d object box correction algorithm based on photometric correction was proposed to optimize the detection results.Finally,CPNet was trained and tested on the KITTI data set,and static and dynamic vehicle detection experiments were set up to verify the effectiveness and robustness in the actual road traffic environment.(3)Propose a multi-object tracking algorithm combining CPNet and Kalman filter.The optimal estimation of Kalman filter is used to solve the problem of misdetection and omission of CPNet.CPNet and Kalman filter are combined,and the efficiency of the tracking algorithm is improved by optimizing the tracking strategy and similarity index.The performance of the tracking algorithm was tested on the KITTI tracking data set and the local image data set,respectively.The experimental results show that the tracking algorithm can effectively reduce the error detection rate and the missing detection rate,and the computational cost of the tracking part takes a small proportion of that of the overall algorithm.