Research On Ship Collision Avoidance Path Planning Based On DDPG Algorithm

With the rapid development of the inland-water transportation,the number of ships is increasing day by day.Therefore,the safety of ship's navigation has been receiving more and more attention.Ship's collision-avoidance,as the core issue of ship navigation,has gradually become a hotspot research for scholars.Deep reinforcement learning is an algorithm that can obtain excellent strategies through interacting with the environment.At present,deep reinforcement learning has made breakthrough progress in the research of vehicle's automatic driving.The process of ship collision-avoidance is very similar to the automatic driving of vehicles.Some research results in the field of vehicle driving can be applied to the process of ship collision-avoidance.As a result,this paper turns to research the ship's intelligent collision-avoidance algorithm based on Deep Deterministic Policy Gradient(DDPG)reinforcement learning and ship navigation dataIn this paper," Tianditu " map system,Automatic Identification System,and ship domain model are used to construct a highly realistic ship's collision-avoidance simulation environment.Then we start the research of ship collision-avoidance regulation which based on DDPG algorithm.In order to improve the accuracy of the ship domain model,this paper adjust ship domain model with AIS data and human observation data of ship in the Zhouzhuang areaIn order to build a ship's collision-avoidance framework,we designed the state,action,reward and neural network model of the DDPG algorithm.This paper simulates various encounter scenarios of the ship during training and realizes avoiding collision in those scenarios.To solve the problem of insufficient of exploration in DDPG algorithm and utilization efficiency of samples,this paper proposes an improvement strategy about strengthening learning in failure areas.This strategy can increases the diversity of data in failure area.By exploration-learning in the failed area which can improves the learning efficiency and learning speed of the algorithm.Simulation experiments show that the improved DDPG algorithm is superior to the original DDPG algorithm in both learning rate and learning effect.This paper applies the ship collision-avoidance algorithm to the scheduling manage of Zhouzhuang touring boats,realizes the function of intelligent planning of touring boat's routes,further improves the safety of the touring boat's sailing.