Research On Unmanned Ship Path Planning Based On Improved DQN Algorithm

This paper aims to study the path planning problem of unmanned ship based on improved Deep Q network algorithm(DQN).In this paper,PER Dueling DQN network algorithm(PER Dueling DQN)is proposed to solve the problems of slow convergence in network training and model oscillation during path planning.The action value function of the traditional DQN algorithm is decomposed into the advantage function and the action value function,which are used to evaluate the importance of each action in the action space in a certain state and evaluate the quality of a certain state respectively,effectively avoiding the problem of network model oscillation in the training process.By introducing a preferential experience replay mechanism,the absolute value of TD-error is used as an indicator of the priority of experience samples to improve the sampling rate of key experience data,and thus significantly accelerate the convergence speed of network training.Finally,this paper uses PER Dueling DQN algorithm to carry out simulation experiments in three environmental models,namely random grid,narrow sea area and complex sea area,and makes comparison and analysis with traditional DQN algorithm.The experimental results show that compared with the traditional DQN algorithm,the convergence speed of PER Dueling DQN algorithm in the random grid environment is increased by 50%,the total length of the planned final path is shortened by 16.5%,and the number of path turning points is reduced by31.6%.In the narrow sea environment,the convergence speed is increased by 42.85%,the total length of the final path is shortened by 1.73%,and the number of inflection points is reduced by20%.PER Dueling DQN algorithm can effectively plan a safe and relatively short path in a complex sea environment,while the final network model of the traditional DQN algorithm is always in a state of oscillation,and no effective path can be planned.In summary,PER Dueling DQN path planning algorithm not only significantly improves the convergence speed and the stability of network models,but also effectively plans a safe and relatively optimal path in a variety of map models,especially in complex environments.Therefore,the algorithm proposed in this paper provides a useful reference for the path planning of unmanned ships in complex sea environment.