Research On Application Of Computer Vision For Unmanned Surface Vehicle

Unmanned surface vehicle is an intelligent multifunctional surface transportation equipment,which occupies an important position in the military and civilian fields.China has a vast area of ocean and inland waters,in order to better develop and utilize the resources of the oceans and inland waters,it is particularly important to study more intelligent unmanned surface vehicle.At the same time,with the increasingly severe environmental problems,the environmental pollution problems in waters have become more serious.There are many floating debris on the water surface in rivers,lakes,and seaside,etc.The traditional method of using manual or large mechanical boats to clean up floating objects on the water surface has a series of problems such as high costs,low efficiency,poorly cleaned scattered garbage,and unsafe personnel safety.Therefore,it is imminent to design an intelligent,low-cost,high-efficiency unmanned surface vehicle for cleaning floating debris on the water surface.In order to solve the above problems,this paper uses unmanned surface vehicle as a research platform,and uses computer vision technology and multi-sensor fusion artificial intelligence methods to study key technologies such as waterfront line extraction,surface target recognition and tracking,and visual positioning of unmanned surface vehicle.The main research work of this paper is as follows:(1)By improving the traditional region growing method and constructing the least squares problem according to the seed point selection rules,the automatic selection of seed point is realized.According to the statistical characteristics of the waterfront image samples in Lab color space,and using the standard deviation of the image to realize the threshold adaptation of the growth rule.Then the waterfront image segmentation and waterfront line extraction were realized by the improved region growing method.(2)Based on the transparent object(water)absorption characteristics of the laser beam emitted from the 3D lidar,a water surface floating object recognition method based on fusion of the camera data and the 3D lidar data was proposed to eliminate the false alarm of target recognition caused by ripples and reflections.Through camera and 3D lidar calibration,the internal parameters of the camera and the external parameters between the camera and the 3D lidar are obtained,and then the 3D lidar data is projected onto the image according to the external parameters and the camera projection model.The CornerNet-Squeeze target detection network is used to detect the floating debris of the segmented water surface image to obtain the candidate target bounding boxes.Count the number of 3D lidar point clouds in the candidate target bounding boxes,and calculate the candidate target confidence based on the percentage of the point clouds in the candidate target bounding boxes and the score output by the CornerNet-Squeeze network.When the confidence level is higher than the set threshold,it is considered that floating debris are detected,otherwise the candidate bounding box is considered to be a false alarm caused by reflection or ripple.(3)In order to improve the response of the unmanned surface vehicle's computer vision system in the target tracking state,a target detection framework combining a target recognizer and a target tracker is proposed.Firstly,search for the floating debris through the target recognizer.When the target is detected,it is used to initialize the KCF(Kernelized Correlation Filters)target tracker.In subsequent frames,only the KCF tracker is used to track the target.Therefore,the KCF tracker's advantage in efficiency is used to realize the real-time feedback of the vision system to the target position.According to the position of the target relative to the unmanned surface vehicle,the control amount is calculated to make the unmanned surface vehicle close to the target and perform the salvage operation.(4)Aiming at the application of visual odometer on unmanned ships,there are problems of positioning failure due to sparse water surface image features and hull sway,etc.Using the complementarity of camera and IMU(Inertial Measurement Unit),a visual inertial odometer fused with IMU data is constructed.And we use the KLT(Kanade-Lucas-Tomasi)feature tracking algorithm instead of feature point matching.Thus,a robust visual positioning of the unmanned surface vehicle is realized.