Research On Autonomous Navigation Decision-making Technology Based On Deep Reinforcement Learning

With the vigorous development of the shipping industry,the number and tonnage of ships have increased substantially,the types of ships have become increasingly diversified,the navigation density has been further improved,and the difficulty of maneuvering has been further increased.In recent years,the unmanned technology has become more and more mature,and the intelligent ship which is economical,safe,reliable,environmentally friendly and efficient has been widely studied,and it has also been concerned and favored by the shipping field.As the core of the autonomous navigation decision-making system,its effectiveness provides support for the safe and reliable navigation of ships.In this paper,through the basic technology research,navigation decision-making algorithm research and decision-making simulation system construction,on the basis of the integration of previous research results,a deep reinforcement learning autonomous navigation decision-making algorithm based on improved deep Q network is proposed,and the effectiveness and feasibility of the decision-making algorithm are verified.It expands the depth and breadth of the research on the autonomous navigation decision-making technology of intelligent ships.1.Integrate the basic technology with the decision-making algorithm.In this paper,a number of technologies including encounter situation division,ship domain model and collision risk evaluation are integrated with the navigation decision-making algorithm,so that the system can clearly and accurately reconstruct the state of the two ships,accurately delimit the safe navigation area of the two ships,quantify the collision risk,and reasonably judge whether to avoid collision and when to avoid collision.Many technologies play a role in the corresponding stages of the whole collision avoidance process,providing the basis for the implementation of the decision-making algorithm,so as to build a safe and effective autonomous navigation decision-making system.2.Proposed an autonomous navigation decision algorithm based on improved deep Q network.This paper designs a reasonable state space and action space from a global point of view,and designs different reward functions from two aspects of route following and collision avoidance actions to restrict the algorithm to meet the established rules in different situations.At the same time,the improved greedy strategy search algorithm is used to improve the traditional DQN algorithm,and an autonomous navigation decision algorithm based on improved deep Q network is proposed.3.Build a modularized autonomous navigation decision-making simulation system.In this paper,the autonomous navigation decision simulation system is built by modular programming to verify the relevant algorithms.The whole simulation system connects the navigation information,basic operation,decision algorithm,navigation information simulation and data visualization modules through the information integration and processing module to achieve data exchange.In this paper,experiments are carried out in the decision simulation system based on different scenarios of two ships encounter to further verify the effectiveness and feasibility of the decision algorithm,and the simulation results are compared with those of the traditional DQN algorithm and Q-Learning algorithm.According to the simulation results,the algorithm proposed in this paper is safe,effective and has application value.In this paper,the key technologies of autonomous navigation decision-making for intelligent ships in open waters are deeply studied,and an autonomous navigation decision-making algorithm based on improved deep Q network is proposed,which integrates the technologies of encounter situation division,ship domain model and collision risk evaluation,and provides a strong technical support for the application of autonomous navigation decision-making system on real ships.