Path Following And Collision Avoidance For Unmanned Surface Vehicles Via Deep Reinforcement Learning

As one of the important components of unmanned systems,unmanned surface vehicles have a wide range of applications in scientific research,civil and military fields.For the unmanned surface vehicle,ensuring that it can navigate along the planned path and avoid obstacles is the precondition for the unmanned surface vehicle to perform various tasks.Therefore,the research on path following and collision avoidance algorithms for unmanned surface vehicles is vital.Due to the complexity of environmental disturbance and the under-actuated characteristics of the unmanned surface vehicle,the existing control algorithms have problems in path following and collision avoidance,such as complicated parameter tuning,difficulty in accurate modeling,and complex calculation.To achieve robust and accurate motion control,this thesis attempts to utilize deep reinforcement learning algorithms to solve the problems of path following and collision avoidance for the unmanned surface vehicle.This thesis first designs a simulation environment suitable for training the reinforcement learning algorithm of unmanned surface vehicle motion control,and researches the path following algorithm and collision avoidance algorithm of unmanned surface vehicles via deep reinforcement learning based on the simulation environment.Specifically,the research contents of this thesis are as follows:①This thesis constructs a simulation environment of reinforcement learning for unmanned surface vehicle’s motion control.To solve the problem of lack of training environment for motion control of unmanned surface vehicles,this thesis firstly selects an appropriate water simulation environment as the basis for development through comparative analysis and builds the simulation model for the unmanned surface vehicle used in the experiment.Subsequently,this thesis develops a ROS software package to encapsulate the simulation environment and the simulation model and designs the generation method of path following and collision avoidance scenarios,which provides a foundation for the path following and collision avoidance algorithm research based on deep reinforcement learning.②This thesis proposes a path following algorithm for unmanned surface vehicles based on the Soft Actor-Critic.Aiming at the path following problem of unmanned surface vehicles,this thesis first transforms the path following problem into a Markov decision process based on the control requirements of the unmanned surface vehicle and vector field algorithm guidance.The Markov decision process’ state space,action space,and reward function are designed reasonably.The rudder angle of the unmanned surface vehicle is output by the Soft Actor-Critic algorithm,and then the path following control of the unmanned surface vehicle is realized.In this thesis,the algorithm is trained and tested in a simulation environment.The simulation results show that the algorithm has the advantages of small tracking error,stable heading control,fast adjustment speed,and strong anti-interference ability compared with other comparison algorithms.③This thesis further proposes an end-to-end collision avoidance algorithm combining representation learning and the Soft Actor-Critic.Aiming at the problem of obstacle avoidance during the navigation of unmanned surface vehicles,it can directly use the point cloud data obtained by Li DAR to control the unmanned surface vehicle to complete collision avoidance.By adding representation learning to extract hidden layer features from point cloud data,it alleviates the problem of low learning efficiency in the initial training stage of the deep reinforcement learning algorithm.The simulation results show that the algorithm can converge faster in the training stage,and can achieve a high success rate of collision avoidance both in the path following and navigation scenarios.