Research On AUV Behavior Replanning Method Based On Reinforcement Learning

Autonomous Underwater Vehicle(AUV)is one of the key equipment technologies for marine resource development.As the task is more and more complex,the AUV's decision-making ability needs to be higher.Planning technology is one of the important means to realize AUV intelligent decision-making,which determines the interaction ability of AUV with the external environment,and is the premise basis for autonomously completing the task.This paper focuses on global planning and local behavioral re-planning in complex environments.It takes the tunnel detection task as the typical application background,and combines the artificial intelligence technology.This paper applies the layered reinforcement learning method to the AUV global route planning task and applies the deep reinforcement learning to the AUV behavior re-planning method,and organically combines the global planning and the local behavior re-planning.It gives the AUV planning system self-learning ability and improves the environmental adaptability.Firstly,aiming at the dimension disaster problem of complex environment state in global route planning,the theory and application of layered reinforcement learning method are deeply analyzed.The hierarchical structure of global route planning task is established from high to low,including:route planning root task layer Sub-task selection layer,basic action layer.This method is easy to expand,and the problem is decomposed into low-dimensional space to solve.Then,based on the principle of hierarchical reinforcement learning algorithm,the AUV global route planning model is designed.The environmental state model and the action model are established and the evaluation function of the route planning task is designed Finally the simulation test platform is built to test to verify the accuracy and practicability of the algorithm.Aiming at the AUV real-time behavior re-planning problem in complex dynamic environment,a deep reinforcement learning algorithm is designed to design a behavior re-planning architecture based on multi-behavior network calling.Then a behavioral re-planning system model is constructed.The behavior re-planning system extracts the continuous environment features with the deep learning network to strengthen the approximation function of the learning output action,and forms the environment state-action mapping,which solves the perception and decision problems in the continuous environment state.According to the requirements of the tunnel task,three typical behaviors of target point,wall tracking and obstacle avoidance are defined,and the corresponding behavior networks are constructed respectively.For each behavioral goal,the corresponding input environment variables and reward and punishment functions are designed to construct the basic action space.Based on the Python platform,the simulation environment was built and the learning algorithm was written.Aiming at the corner problem,an improved wall tracking method based on virtual target points is proposed.At last,simulations of single behavior in multiple environments and multi-behavior calling experiments were carried out.This paper combines the reinforcement learning algorithm with the AUV planning system to improve the adaptive level and self-determination ability of AUV.The global route planning is the premise of the tunnel detection task.In the actual execution process,the AUV completes the task according to the route point output by the global planning,and then reaches the path point by calling the behavior network.In the behavior re-planning system,different network models are constructed for different behavioral requirements,so that AUV calls the relevant behavior network according to the real-time environment during the operation process to improve the planning level of AUV in complex and unknown environment.