Vision-based Autonomous Environment Perception Of Air-sea Cooperation For Unmanned Surface Vehicles

The continuous development of sensing technology,artificial intelligence technology and intelligent control technology has promoted the development of unmanned surface vehicle(USV).The intelligentization is an important research hotspot in the development of USVs,and providing accurate surrounding environment information for USVs is a prerequisite for achieving intelligentization.At present,there are a variety of sensors that provide different perception information for USVs.Among them,visual sensors have become an important means of USV’s environment perception system due to its advantage of small size and rich information.However,due to the limitation of the installation height of the USV’s vision sensor and the influence of complex and changeable water surface scenes,how to solve the USV autonomous perception under the restricted field of view has become a hot issue in the research of the USV visual perception system.Regarding the above problems,this paper aims to improve the visual perception and scene understanding capabilities of the USV,and studies the joint perception method based on air-sea coordination.This paper discusses the small sample learning of USV visual semantic segmentation network in the absence of training data,adaptive learning of USV visual semantic segmentation network in complex and changeable water environment,three-dimensional modeling of complex water scenes and perception during USV autonomous navigation.Specific innovations include:First of all,to solve the problem of insufficient data in the training and learning process of the USV visual semantic segmentation network,a small sample learning method with data enhancement is proposed.A multi-feature fusion convolutional neural network model is constructed to perform semantic segmentation on USV images.A data enhancement algorithm based on the generative adversarial networks is proposed to solve the problem that the network model depends on a large amount of manual label data.It is verified through five different navigation scenes.The experimental results show that after a small number of sample training,the network model achieves good results,effectively solving the problem of insufficient training data for water navigation scenes.Then,aiming at the problem of performance degradation of USV visual semantic segmentation network in the complex and changeable water environment,an adaptive transfer learning method for semantic segmentation network is proposed.A region mask generation method based on seed points is proposed to realize the automatic generation of self-training labels for segmentation networks.A research on the uncertainty of automatically generated labels is conducted to solve the degradation problem of semantic segmentation network self-training.In order to verify the effectiveness of the algorithm,the network model based on the East Lake scene was transferred to five different navigation scenes.After continuous self-learning,the network model gradually adapts to the new environment.It shows that the proposed method can effectively improve the adaptive ability of the network and solve the transfer problem of the network model in different scenarios.Secondly,aiming at the problem of USV’s visual susceptibility to environmental influences and limited vision,a three-dimensional modeling method for complex water scenes based on air and sea is proposed.Fusion of visual images from UAVs and USVs from different perspectives,enriching data sources for 3D reconstruction.Semantic segmentation network is used to process visual images to reduce the impact of water surface dynamic texture on 3D reconstruction.A dense 3D model reconstruction method based on a two-level view selection algorithm is proposed to improve the accuracy of image matching and the efficiency of subsequent model reconstruction calculations.Taking an island in East Lake as the experimental object,through visual images from different perspectives,the surrounding scene was effectively reconstructed in three dimensions,which improved the USV system’s stereo perception ability of the environment.Finally,aiming at the perception problem of USVs in different stages of autonomous navigation,a systematic air-sea cooperative perception method is proposed.Systematic analysis of the mission process of the USV system is carried out,and the mission execution process is abstracted into three stages: mission target search,autonomous navigation and target approach.For the task of target search stage,the semantic segmentation network is used to process the UAV vision to assist the USV to quickly identify different targets on the water surface.For the autonomous navigation stage,the air-sea cooperative stereo perception algorithm is used to provide the USV system with global environmental information of the complex environment.Using the visual semantic segmentation algorithm of USVs,it provides real-time dynamic navigation environment information for USVs.Aiming at the target approach stage,an air-sea coordinated water target positioning algorithm is proposed,and a visual observation model of the air-sea coordinated UAV and USV is constructed to realize the USV’s spatial position perception of surrounding objects.The algorithm is systematically verified by the two tasks of USV autonomous harbour entry and water object search.Through the cooperative perception of USVs and UAV,it effectively solves the perception problems of USVs in different stages of navigation and provides safety guarantee for autonomous navigation of USVs.