Research On Ship Intelligent Collision Avoidance Decision Based On COLREGs And Reinforcement Learning

The international rules for preventing collisions at sea(COLREGs)is of great significance to avoid collision and improve the safety of navigation.Currently,the COLREGs is generally applicable to manned ships.However,with the continuous development of ship intelligent technology,people have higher and higher requirements for unmanned ship operation.How to use the technology of artificial intelligence to make the ship sail automatically and safely according to the COLREGs when the ship sails autonomously is of great significance to promote the development of ship intelligence and improve the safety of the ship.Therefore,based on the COLREGs and reinforcement learning algorithm,this paper studies the autonomous collision avoidance strategy of large cargo ships on the sea.The main contents are summarized as follows:(1)Aiming at the problem of ship collision avoidance,according to the COLREGs,the process and constraints of ship collision avoidance are clarified,and the collision risk model based on the DCPA and the TCPA is established.Combined with the collision risk model,the "simplest collision avoidance operation process" is proposed to establish the model of ship maneuverability and collision avoidance process.(2)A navigation encounter situation classifier based on the COLREGs and machine learning is proposed.Firstly,the scene simulation model of ship collision avoidance is established to collect scene samples of ship encounter under different situations,and then the sample of standard collision avoidance decision is obtained by combining the collision avoidance decision-making made by navigation experts in accordance with the COLREGs.Two machine learning methods,neural network and support vector machine,are used to train and verify the decision samples of collision avoidance.An effective ship navigation encounter situation classifier is obtained,which can determine the encounter situation of collision avoidance according to the navigation parameters between the two ships and provide auxiliary collision avoidance strategies.(3)Based on the finite state Markov decision-making process,the mathematical model of ship collision avoidance is established,and the inverse reinforcement learning is introduced to search for the return function of the operation strategy demonstrated by human experts.Finally,using reinforcement learning to search for collision avoidance strategies to obtain collision avoidance strategies that are close to human experts’ demonstration operations.This study obtains human expert decision-making strategies with good collision avoidance effect.At the same time,inverse reinforcement learning can reliably invert human expert decision-making strategies with different operation habits.Compared with conventional reinforcement learning,it has the characteristics of fast convergence,good collision avoidance effect and low requirement for online processing ability.Therefore,the inverse reinforcement learning can be used in the automatic collision avoidance study of ships in combination with the COLREGs.