Research On Improved USV Option Based Path Planning Method

Autonomous Navigation of Unmanned Surface Vehicle(USV)in marine environment is not only an important goal of shipbuilding and marine transportation,but also the general trend of artificial intelligence era.The ability of autonomous collision avoidance and reasonable path planning according to the surrounding environment is not only the core of this goal,but also the key technology of USV in practical application.At present,reinforcement learning system has been widely used in path planning of USV in unknown environment,but the traditional reinforcement learning algorithm still has many shortcomings,such as slow convergence of neural network,easy to fall into local minimum and long training time in high dimensional states space.This paper introduces the development idea of hierarchical reinforcement learning to solve the path planning task of USV under various environmental conditions.The main research work is as follows.Firstly,through the deeply analysis of the research status of USV,route planning technology and reinforcement machine learning,the research background and significance of this subject are clarified,the interaction mechanism between USV and water environment is theoretically modeled and calculated,and a path planning algorithm based on double-layer cyclic neural network and actor critical structure is proposed.Secondly,for the path planning problem of USV in relatively complex environment,a hierarchical reinforcement learning algorithm based on the operation characteristics of unmanned ship is provided.Through the algorithm optimization of the motion characteristics of USV,the action space optimization based on the navigation characteristics of unmanned ship,the dynamic reward and punishment function can inspire and guide the path planning,and solve the problem of using reinforcement learning algorithm and exploring the balance between them.On this basis,the path planning strategy of unmanned ship in the environment with complex obstacle distribution is given.The basic theory and application of hierarchical reinforcement learning method are deeply analyzed,the learning strategy architecture of USV path planning task is constructed by using option critic learning algorithm,and the design simulation comparison test is used to verify the reasonable feasibility of the improved algorithm in USV path planning.Finally,the simple use of reinforcement learning to realize the path planning of USV in complex environment has a long training time and the possibility of convergence.Aiming at the disadvantages of unstable training and long training time of neural network in large state space of reinforcement learning,a path planning algorithm combining actor critic algorithm and Dijkstra algorithm is proposed.The algorithm combines reinforcement learning and traditional Dijkstra algorithm,which effectively shortens the training time;At the same time,the dual critic network in the actor critic network structure can effectively eliminate the large deviation in calculating the value function and enhance the stability of training.The algorithm uses the traditional algorithm to generate subtasks,which reduces the state search space of reinforcement learning and improves the training speed.The simulation experiments on maps with different obstacle distribution show that the algorithm has good results.In this paper,the global route planning of USV is closely combined with the local route planning,which improves the efficiency of its operation and navigation,and is conducive to the safe,reliable and independent realization of the tasks of medium and long-distance navigation and geological exploration.It has great theoretical significance and practical application value.