Research On Control Algorithms Of Autonomous Underwater Vehicle Based On Reinforcement Learning

Autonomous underwater vehicle(AUV)has become an important tool in the process of underwater unmanned automation.In practical application,it shows more and more importance.With the development of AUV in recent decades,along with the innovation of various control algorithms in the field of industrial control,AUV control algorithms are constantly optimized and updated.From the original classical Proportion-Integral-Differential(PID)control algorithm to fuzzy control,adaptive control,intelligent control and various combination control algorithms,the AUV control has become mature.However,in the face of more complex tasks and working environment,the above algorithms still have some problems,such as: weak adaptive ability,relatively low robustness,poor control accuracy and other issues.In view of the shortcomings of the traditional AUV control algorithm,this paper proposes an AUV control algorithm based on reinforcement learning.We studie the kinematics and dynamics model of AUV,designs an improved AUV control algorithm based on reinforcement learning,builds a small AUV control system and verifies the actual performance of the algorithm.The specific research contents are as follows:1.Aiming at the complex shallow water environment,the six-degree-of-freedom kinematics and dynamics equations of the underwater vehicle are established,and the model of the small autonomous underwater vehicle designed in this paper is simplified.According to the proposed model,the simulations are carried out.2.The commonly used reinforcement learning algorithms are studied,and their characteristics and applications are analyzed.Then,the control model of underwater vehicle is studied,and the control algorithm of underwater vehicle based on reinforcement learning is designed according to the characteristics of reinforcement learning.In order to improve the generalization ability of the algorithm,the DPG(Deterministic Policy Gradient)combined with the neural network method is adopted,and the idea of DQN(Double Q-Network)is used to solve the instability of the algorithm training by using the same structure of the dual network model.Aiming at the problem of short duration of underwater vehicle,the concept of priority is introduced into the empirical playback sample pool of the control algorithm,and the binary tree method is used to realize the priority sampling of the samples,which improves the convergence speed of the algorithm rapidly and increases the endurance ability of the underwater vehicle.3.In view of the shallow water environment such as rivers and lakes,a small autonomous underwater vehicle is designed in this paper.Its unique design not only has the advantages of low resistance of torpedo-like AUV,but also has flexible motion control of open-shelf AUV.In order to understand the motion of underwater vehicle in real time and intuitively on land,this paper also designs a ground station system,which has many functions such as data bidirectional communication,motion control,task sending and so on.Finally,through a large number of simulation experiments,the effectiveness of the proposed control algorithm is verified.At the same time,based on the built control system,the proposed control algorithm is used to control AUV in the actual environment.Relevant experimental results also show that the control algorithm has good control effect.