Lidar And Camera Information Fusion Based Environment Sensing For Unmanned Surface Vehicles

In recent years,with the rapid development of sensor technology and automation technology,the application of autopilot technology in unmanned surface vehicles has received extensive attention and research.Environment sensing technology,as one of the key technologies of unmanned surface vehicles,is the basis on which unmanned surface vehicles can navigate autonomously and safely.Lidar maritime target detection is an active detection method,and lidar is less affected by the environment,can work around the clock,has good robustness,and can provide distance information and three-dimensional information of surrounding targets.Visual maritime target detection is a passive detection method,and the images acquired by the camera can provide rich feature information of the target,such as colour,texture,etc,and can output the category information of the target.This thesis combines the different characteristics of the two sensors and proposes an unmanned ship environment perception method based on the fusion of lidar and camera information,aiming to combine the advantages of different sensors to make up for the shortcomings brought by a single sensor and improve the unmanned surface vehicles ability to describe the surrounding environment.The main research elements of this thesis are as follows:(1)The visual maritime target detection method is investigated: firstly,YOLOv4 is used as the base network,and in order to meet the training deployment of the model as well as the accuracy and real-time requirements for maritime target detection,the Mobilenetv3 network is used to lighten the backbone network,and a CBAM module is added to the end design of the feature fusion network to improve the model target detection accuracy.The improved non-maximum suppression algorithm soft-diou-nms algorithm is used in the post-processing stage to remove the redundant prediction frames and achieve the detection effect on dense occlusion targets.The initial anchor frame suitable for maritime target detection is obtained by using the K-means++ clustering algorithm with a home-made dataset to optimise detection accuracy.(2)A study on the detection method of lidar maritime targets: for the characteristics of large amount of lidar point cloud data and invalid point cloud data,the laser point cloud data is pre-processed,and the reduction of data volume and the elimination of invalid points are completed by using voxel filtering and statistical filtering to improve the operation efficiency of the clustering algorithm.According to the physical properties of lidar,the point cloud data is not uniformly distributed,a dbscan clustering method with the dynamic adjustment of the clustering radius Eps with the scanning distance is used to determine the size of the clustering radius Eps adaptively,to reduce the sensitivity of the clustering algorithm to the selection of the clustering radius and to improve the accuracy of the target detection while network.It also reduces the complexity of computation and improves the real-time performance of the target detection model by constructing a KD-tree.(3)The fusion method of lidar and camera information is studied: the decision-level fusion is chosen to fuse the information of the two sensors.Firstly,the internal reference matrix and distortion coefficient of the camera are solved using the Zhang zhengyou calibration method,and then the joint calibration of the lidar and camera is carried out to obtain the transfer matrix from the lidar to the camera and complete the spatial calibration of the lidar and camera.To ensure that the lidar and the camera match to the same target at the same moment,the nearest neighbour time synchronisation method is used to complete the time synchronisation.After completing the temporal synchronisation,the 3D point cloud is projected onto the image using the transfer matrix obtained from the joint calibration,and the area overlap rate of the target detection frame is used to determine whether it matches as the same target.If the targets are not matched,the detection results are output separately,taking into account the safety of ship navigation.If the match is the same,the maritime target category information identified in the image is fused with the 3D information of the maritime target identified in the 3D point cloud to obtain the complete maritime target information at the data decision level.