Dynamic Route Planning Of Inland Water Network Based On Deep Reinforcement Learning

The route planning of inland water network is of great significance to realize the intelligent navigation of ships.However,most of the current methods are still using static planning algorithms,and there are few dynamic route planning algorithms that take real-time ship traffic flow as an influencing factor.Using the deep reinforcement learning and graph neural network methods that have achieved great success in recent years,this thesis studies the real-time dynamic route planning of the inland water network in view of the complex and changeable navigation environment of the inland water network.The main work completed by the thesis is as follows:(1)Dynamic route planning algorithm of inland water network based on DQN(Deep Q＿Network).On the basis of previous research on the DQN state space setting,this thesis proposes a dynamic state space generation method for inland water networks.Firstly,the inland water network is mapped into matrix data according to the longitude and latitude of the inland water network nodes;secondly,since the matrix data formed based on the longitude and latitude mapping of the path nodes has redundant interval nodes for storing the connection relationship,the method of interval deduplication is used,greatly reducing the state space and improving the running speed of the algorithm in practical application;finally,the position of the ship in the inland water network,the destination of the ship and the channel traffic flow density information in the inland water network are reflected in the matrix data,forming a dynamic state space as the input to the algorithm model.In addition,according to the characteristics of the water network,this thesis conducts research on the targeted action space setting,channel connectivity and direction detection and sparse reward problems,which solves the problem that DQN is difficult to train,so that the dynamic route planning model based on DQN can quickly find the optimal route in the changing environment.The experimental results compared with traditional route planning methods such as A* show that the proposed method can consider the real-time waterway traffic flow,find a route that can make the ship reach the destination faster,and realize the real-time dynamics of the inland water network route planning.(2)Dynamic route planning algorithm of inland water network based on the combination of DQN and graph neural network.Aiming at the problem that the DQN-based dynamic route planning algorithm has complicated steps in practical application,the method of combining it with graph neural network is further studied.The inland water network is a typical graph structure data.Since it does not meet the translation invariance in the convolutional neural network,it cannot be directly sent to the network as the input of DQN for convolution calculation.In order to solve this problem,this thesis proposes to replace the convolutional layer of DQN with the backbone network in STGCN graph neural network,and let the agent make decisions according to the dynamic spatial features extracted by STGCN,so as to realize dynamic route planning.The experimental results show that the dynamic route planning algorithm based on the combination of STGCN and DQN in this thesis has a large action space and the model is more complex than the DQN model,so the time cost of the algorithm is increased compared to the dynamic route planning algorithm based on DQN,but it achieves end-to-end training and inference,and also has more room for improvement.