Research On Intelligent Ship Collision Avoidance Algorithm In Complex Waters Based On Improved Deep Q Network

Ship collisions often result in huge losses of life,cargo,and ships,as well as serious pollution of the water environment.Meanwhile,it is estimated that between 75% and 86% of maritime accidents are related to human factors.Thus,it is necessary to enhance the intelligence of ships to replace the traditional piloting mode partially or fully,and eventually achieve autonomous collision avoidance to dilute the impact of seaman factors.The main research of this thesis is the intelligent ship autonomous collision avoidance decision-making algorithm.According to the framework of reinforcement learning,a decisionmaking algorithm for intelligent ship autonomous collision avoidance is designed,and the following research work is completed:In this research,a multi-ship automatic collision avoidance method based on a double deep Q network(DDQN)with prioritized experience replay is proposed.Firstly,we vectorize the predicted hazardous areas as the observation states of the agent so that similar ship encounter scenarios can be clustered,and the input dimension of the neural network can be fixed.The reward function is designed based on the International Regulations for Preventing Collision at Sea(COLREGs)and human experience.Unlike the application of previous collision avoidance methods based on deep reinforcement learning,in this thesis,the interaction between the agent and the environment occurs only in the collision avoidance decision-making phase,which greatly reduces the number of state transitions in the Markov decision process(MDP).The prioritized experience replay method is also used to make the model converge more quickly.Finally,19 single-vessel collision avoidance scenarios were constructed based on the encounter situations classified by the COLREGs,which were arranged and combined as the training set for the agent.The effectiveness of the proposed method in close-quarters situation was verified using the Imazu problem.The simulation results illustrate that proposed algorithm achieve multi-ship collision avoidance in crowded waters,and the decisions generated by this method conform to the COLREGs and are close to the human level handling.In addition,considering that the responsibility for collision avoidance is shared in actual maritime multi vessel encounter situation,this article extends the proposed collision avoidance method based on single agent reinforcement learning to the field of multi-agent distributed collaborative collision avoidance.Combined with imitation learning,it provides faster and more stable training for decision-making models.On the other hand,agents adopt a centralized learning and distributed execution architecture,and exhibit implicit coordination through shared decision-making networks.Simulation experiments have verified that the model trained by this method can be replicated to any number of agents,and this method surpasses the simulated experts in reducing sailing distance.