Path Following Of Intelligent Unmanned Surface Vehicle Based On Deep Reinforcement Learning

The 21st century is the ocean century.The development of ecology,economy,society and other aspects is closely related to the ocean.The development and protection of the ocean is a long-term subject.Marine autonomous vehicles can greatly improve the efficiency of human exploration of the ocean and have important research significance in the utilization of marine resources and the protection of marine ecology.As a marine autonomous vehicle,unmanned surface vehicle(USV)has the characteristics of small size,high speed,low cost and strong autonomy,and has unique advantages with performing dangerous or boring tasks in harsh environments.This paper studies the control problem of USV path following based on deep reinforcement learning.The main research work is as follows:First,for the path following problem of the USV,this paper presents a fully data-driven motion control method for the USV based on deep reinforcement learning,which overcomes the limitation of the motion control method based on the mechanism model,a data-driven endto-end self-learning modeling method for the USV was established.Based on the neural network prediction model,a model predictive controller that can realize the path following control target of the USV is designed.Finally,the controller is simulated and verified.Simulation results show that by using the input and output data of the USV based on random shooting sampling to train the deep neural network,the proposed model predictive controller can make the USV accurately track the parameterized path.Second,for the path following problem of the USV,this paper optimizes the path following control method of the USV,and adopts a more flexible model predictive control.The method is based on the information theory for the model predictive path integral control to study the path following problem of the USV.This method can effectively solve the finite-range nonlinear optimal control problem,which is not limited by the state cost function.This paper proposes a path following control method for the USV based on deep reinforcement learning and model predictive path integral control.Specifically,a neural network is trained using deep reinforcement learning methods to approximate the state transition model of the USV.Then,based on the learned model,a model predictive path integral controller is used to obtain optimal actions.Based on the sampled control actions,the model predictive path integral controller calculates the optimal control input.Finally,the path following task of the USV is realized.The simulation verification is carried out for various parametric path following problems such as straight line and curve.The simulation results show that the proposed data-driven USV path following algorithm has good performance.Third,for the collision avoidance path following problem of the USV,the collision avoidance penalty function is designed,and a model predictive path integral controller integrating collision avoidance mechanism is proposed.Firstly,the neural network model is trained with random input and output data.Secondly,a collision avoidance penalty function is designed for dynamic obstacles,and a model predictive path integral controller integrating collision avoidance mechanism is constructed.Finally,the simulation results of the collision avoidance path following problem of the USV is carried out.The simulation results demonstrate the effectiveness of the model predictive path integral control method with integrated collision avoidance mechanism.