Research On Empty Container Repositioning Problem Of Marine Containers Based On Reinforcement Learning

The rapid development of global trade is inseparable from maritime logistics.Container transportation is an important part of maritime logistics.However,empty container repositioning has become a key issue in maritime logistics due to the uneven development of global trade.Empty container repositioning is a costly activity that does not directly generate economic benefits.The realization of efficient empty container repositioning can reduce the cost of maritime logistics and improve the operational efficiency,thus further promoting the development of global trade.In this paper,the empty container repositioning problem is analyzed in depth.Aiming at the characteristics of empty container repositioning problem and the shortcomings of existing research methods,the problem is modeled from the perspective of reinforcement learning,and a multi-agent reinforcement learning empty container repositioning decision model is proposed.The model uses the port as the decision-making agent to trigger the repositioning decision by the cargo ship arrival event,thus achieving more flexible and efficient empty container repositioning.At the specific design and implementation level,this paper first proposes a multi-agent DQN decision-making model,and optimizes the problem that the model has limited action space.A multi-agent DDPG decision-making model is proposed.For the complex maritime scenario,this paper further proposes a parameter-sharing model implementation method to save the computation and memory resources while making the model learn a better strategy.In addition,this paper designs and implements a container empty container repositioning simulation system as an experimental and evaluation platform.The simulation system has flexible architecture and strong scalability,and can be used as a general platform for researching empty container repositioning problems.In this paper,the proposed model is simulated and compared with several repositioning strategies.Experiments show that the multi-agent reinforcement learning decision model can spontaneously learn an effective empty container repositioning strategy in the process of interacting with the environment,which can realize efficient empty container repositioning and reduce repositioning costs while ensuring the demand for empty containers in each port.