Path Planning Of Unmanned Surface Vehicles Based On Deep Reinforcement Learning

The unmanned surface vehicle(USV)has a broad application prospect.Autonomous path planning,as its key technology,has become a hot research direction in the research field of USV.Combined with the theory and method of artificial intelligence,this paper studies the path planning of USV based on deep reinforcement learning,and solves the problem of autonomous path planning in unknown and complex navigation environment.First of all,in open water,by designing the fitting grid of the safety range of USV and the similar line-of-sign' pruning technology,combined with Theta*theory,the sparsity global optimal waypoints are generated.Furthermore,in the complicated narrow waters,an elite-duplication genetic algorithm(EGA)for global optimization of sparse waypoints is designed by integrating elite and diversification strategies' into the genetic algorithm(GA).B-spline technique is further deployed to make flexibly smooth interpolation that contributes to path smoothing supported by optimal sparse waypoints.Then,under unforeseen circumstances,a dynamics-constrained global-local(DGL)hybrid path planning scheme incorporating global path planning and local hierarchical architecture is created for a USV with constrained dynamics.To deal with dynamically unforeseen environments,a local hierarchy is established by fuzzy decisionmaking(FDM)and fine dynamic window(FDW)layers,which are responsible for large-and close-range collision avoidance,respectively,by governing surge and yaw velocity guidance signals.With the aid of the FDW,constrained dynamics pertaining to the USV,are elaboratively embedded into local path planning,which in turn governs trackable collision-avoidance local path.At the same time,a virtual waypoint is set up to make the local path and the global path connect seamlessly,and thereby contributing to the entire DGL hybrid path planning scheme.Simulation studies and comprehensive comparisons in real-world geographies have been conducted to demonstrate the effectiveness and superiority of the proposed DGL hybrid path planning scheme.Additionally,cohering with the smooth path,a deep deterministic policy gradient(DDPG)algorithm with a task-oriented reward function is developed to deterministically extract in-depth pilotage policies,i.e.,mappings from path tracking errors to force-level control actions,from data reservoir stored in a sliding-window replay buffer.Consequently,the genetic-assisted deep reinforcement learning(GDRL)algorithm is obtained,which is an autonomous pilot framework of a USV in constrained waters that have been firstly established by integrating waypoints generation,path smoothing,and policy guidance.This paper compares the proposed algorithm with other algorithms in different geographical areas under the same conditions,so as to verify the stability and effectiveness in response to the sudden situation and ocean disturbance.In addition,the deep navigation model obtained by using other environment training in different geographical areas,from the analysis of the data results obtained,the model has good generalization performance.