Research On Obstacle Detection Based On Information Fusion Of Vehicle Lidar And Camera

With economic development,people’s living standards continue to improve,the number of cars continues to increase,and with it comes the issue of vehicle driving safety.In recent years,the intellectualization of automobiles has gradually attracted widespread attention,which brings a new turning point for the solution of vehicle driving safety problems.The automobile can realize the perception of the driving environment through the sensors integrated on the body.Based on lidar and camera in intelligent driving,this paper studies the sensor joint calibration method and data fusion method of lidar and camera,as well as the application of fusion data in obstacle detection and obstacle tracking,which can make the vehicle better to sense obstacles in the surrounding driving environment and improve the safety of drivers and traffic participants.In addition,this paper selects the emerging non-repetitive scanning lidar as the research object,and uses its characteristics of high FOV coverage and low cost to obtain relatively dense forward point cloud data.The main research contents of this paper are as follows:（1）The calibration methods of lidar and camera sensors are studied,and the camera model,camera distortion correction method,and the joint calibration method of lidar and camera are studied to realize the spatial calibration and time synchronization of lidar and camera.And the influence of different calibration plate sizes on the joint calibration of sensors is analyzed.（2）Using the sensor calibration parameters,a multi-sensor information fusion algorithm based on coordinate projection transformation is proposed.Firstly,aiming at the redundancy problem of some point clouds caused by the different FOV of the camera and lidar,a matching and cropping algorithm of lidar point cloud data based on the FOV is proposed to filter out unnecessary data points in the fusion process.Then,aiming at the fusion of point cloud data and image data,an algorithm is proposed to map the 3D space point cloud data to the 2D space using the joint calibration parameters of the sensor,so that it can form a coordinate mapping with the 2D camera image data.Finally,point cloud coloring is performed on the point cloud data that can be mapped with the camera image,so as to realize the multi-sensor information fusion of the lidar and the camera,and obtain the fusion data.（3）Carry out obstacle detection work based on fusion data to improve the accuracy of obstacle detection based on traditional methods.Firstly,aiming at the point cloud density problem of lidar,a multi-frame fusion algorithm of point cloud data is proposed to improve the density of point cloud data,and then a ground segmentation algorithm based on the height difference of the lowest point on the grid is used to filter out unnecessary ground point.Then,for the clustering problem of obstacles,the traditional clustering algorithm based on Euclidean distance is improved,and the color information in the fusion data is used to filter the edge noise points in the color space on the basis of Euclidean clustering,so as to obtain more Accurate obstacle size data.Finally,for the obstacle classification problem,the proposed obstacle feature voting classification algorithm,combined with the size feature of the obstacle.（4）The obstacle tracking work is carried out according to the fusion data,and the trajectory tracking of the obstacle is realized by combining the Kalman filter algorithm.Aiming at the problem of association and matching of obstacle data,an obstacle data association algorithm based on fusion data is proposed to complete the matching between Kalman filter and obstacle data.The experimental results show that the multi-sensor information fusion algorithm proposed in this paper can well fuse the lidar point cloud data and the camera image data,so that the two sensors can complement each other’s advantages.The obstacle detection work using the fusion data can obtain more accurate obstacles.It can bring more improvements to the upper-layer applications in intelligent driving.