Research On Algorithm Of Ship Intelligent Collision Avoidance Based On Deep Reinforcement Learning

With the gradual development of maritime traffic toward high speed,intensification and complexity,traditional ship collision avoidance technology often cannot timely and correctly handle complex ship encounter scenarios,and ship collision accidents still occur frequently,causing major harm and huge economic losses.According to statistics,the failure of the crew to follow the rules of collision avoidance when carrying out the collision avoidance decision is the main cause of the collision accident.The International Regulations for Preventing Collisions at Sea（COLREGS）are based on human-made agreements.The interpretation of the rules and regulations of some collision avoidance rules is open to some degree,and it is difficult to achieve uniform standards.Aiming at the problems of ship collision avoidance technology,the establishment of an intelligent collision avoidance decision-making system with the ability to deal with multiple meeting scenarios has become an important research topic in various countries.In order to reduce ship collision avoidance accidents caused by improper man-made operation and improve ship navigation safety,this paper proposes a ship intelligent collision avoidance algorithm based on Deep Q Network（DQN）reinforcement learning method.The main contents include:（1）Analysis of design criteria for ship collision avoidance model based on reinforcement learning.Firstly,the decision-making process of ship collision avoidance was sorted out,and the ship’s encounter situation division model was given,and the safe encounter distance of the ship was determined according to the collision avoidance rule.Then the basic principle of reinforcement learning algorithm was briefly introduced,and based on the characteristics of ship collision avoidance,a reinforcement-based approach was proposed.Learn the problems of ship collision avoidance.Finally,from the three aspects of real-time collision avoidance decisionmaking,observing international maritime collision avoidance rules,and restoring course,the design criteria for ship intelligent decision-making based on reinforcement learning are proposed.（2）Construct a ship intelligent collision avoidance model based on DQN.By collecting ship navigation status information in real time,combining the factors such as the collision urgency of the ship,the calculation of heading offset,and the basis for judging the situation,the ship’s intelligent collision avoidance DQN status set is designed from a global perspective.Analyze the characteristics of navigation of maritime vessels,and on the basis of fully understanding the international maritime collision avoidance rules,reasonably quantify the requirements of some core collision avoidance rules,and reward DQN for intelligent collision avoidance of ships from the perspectives of "heading maintenance" and "ship avoidance" respectively.Function design.Finally,the implementation process of ship intelligent collision avoidance algorithm based on DQN is combed.（3）Collision avoidance simulation verification based on the ship’s DQN intelligent collision avoidance algorithm.The numerical simulation experiments are designed for the multiple encounter situations and multi-ship encounter scenarios of two ships,and the analysis of the convergence effect of the DQN algorithm and the results of the ship simulation trajectory show that the intelligent collision avoidance method proposed in this paper can better complete complex Avoidance missions.This research is based on deep reinforcement learning theory,and based on a full understanding of international maritime collision avoidance rules,a ship DQN intelligent collision avoidance algorithm is proposed,which expands the research ideas of ship collision avoidance intelligence,which can provide intelligent ship collision avoidance and a reference for technical research.