Research On AUV Path Following Method Based On Reinforcement Learning

In recent years,Autonomous Underwater Vehicle(AUV)has become an important equipment for human to explore the ocean,and with increasing knowledge of the ocean,the technology related to AUV has been developed,and higher autonomy and machine intelligence have become the new goals of AUV.In order to improve the autonomy and intelligence of AUV,this paper rely on the project of "Intelligent System Design Technology",studies the path following problem of AUV based on the reinforcement learning method.The main research contents are as followed.Firstly,in order to analyze how reinforcement learning theory can be used to improve control autonomy and intelligence,this paper first introduces reinforcement learning theory,designs Q learning controller based on reinforcement learning theory,and analyses the advantages and disadvantages of the controller used in AUV velocity and heading control by simulation experiments.Aiming at the problem of poor control accuracy of Q-learning controller,the definition of reinforcement learning quadruple is redefined,and the reinforcement learning S-plane controller is designed.On this basis,the neural network reinforcement learning S-plane controller is designed to solve the problem of slow training speed.The control performance and training speed are verified by simulation experiments.Secondly,this paper studies the line-of?sight(LOS)path following method based on the neural network reinforcement learning theory.Since the following performance of LOS depends largely on the performance of the controller,this paper takes neural network reinforcement learning S-plane controller as the underlying controller of LOS path following,and the simulation experiments were carried out to verify the control effect of the system under the interference-free and current conditions.Then,this paper deduces the formula of path following method which based on tracking error dynamics deduction.Based on the neural network reinforcement learning method,design the neural network reinforcement learning path following controller.A simulation experiment is carried out for the linear and curve path following of the controller to verify the effect of the improved path tracking control period on improving the autonomy and intelligence of AUV control.Finally,by comparing the two improved path following methods in this paper,their respective performances are explained from three aspects:control performance,the difficulty of controller design and the sensitivity of reinforcement learning method optimization.In the end,two methods application conditions are given.