Research On Sequential Decision-making Of Intelligent Ship Control In Container Terminal

Container terminals are key nodes of international logistics.As the development of international shipping,shipping capacity demand increasing through time,which makes the operating bottleneck of container terminals a more prominent factor in shipping operations.Vessel control is one of the most important handling decision of a container terminal.The efficiency of vessel control directly determines the completion time of container vessels and directly affects the competitiveness of a container terminal.Therefore,how to rationally optimize the vessel control decision is the core research topic of container terminals to further improve the efficiency of shipping operations.Current vessel control researches mainly focuses on local vessel-scale or sigle quay crane work lane handling scale desisions,which may cause handling conflicts and stagnation during continues handling of entire terminal quay.Because of the complexity of constraints and compositions of global vessel control decision problem,manual handling schedule is nearly imposible.Vessel control decision problem is a dynamic decision process as vessel handling proceeds and it performs no aftereffect.Accordingly,vessel control decision problem is a Sequencial Decision Problem(SDP).To solve the global vessel control decision problem,a SDP framework is proposed to formulate intelligent vessel control SDP,artificial intelligence related technologies and methods are introduced to solve intelligent vessel control Sequencial Decision Problem(SDP),to realize optimization of the container terminal's global loading instructions,and to ensure that the loading operation is carried out in a balanced,orderly and efficient manner.In this paper,intelligent vessel control SDP framework is proposed,in which the basic intelligent vessel control SDP is defined as two sub-issues named container loading point SDP and loading order SDP.In this framework,container loading point SDP mainly focuses on optimizing loading point scheduling to secure handling efficiency and balance,loading order SDP focuses on optimizing handling orders to ensure handling safety and continuity.A simulation platform is established to test and verify proposed framework and method.The main research content and innovation results of this paper include:1.Proposed a general solution to the problem of intelligent vessel control SDP in container terminals.The basic problem of sequential decision-making of intelligent vessel control in container terminals is proposed and constructed as two SDP sub-issues.These two key issues are defined as container loading point SDP and loading order SDP;here Based on the sequential decision theory,the intelligent vessel control sequential decision-making framework is proposed,which integrates container loading point SDP and loading order SDP,and simulates the loading operation of the container terminal as decision evaluation.Simulation cases and application cases proves that the intelligent vessel control sequential decisionmaking framework can solve the vessel control SDP effectivelly.2.To solve the problem of container loading point SDP,the intelligent vessel control loading point SDP model is established;S-MCTS algorithm is improved and designed to solve the intelligent vessel control loading point SDP problem.Realistic cases are solved and simulated by proposed method and simulation platform,result proves the proposed method can effectively reduce the conflict of loading point operation,and can realize the effective balance and overall optimization of the handling process.3.To solve the problem of intelligent vessel control loading order SDP,the intelligent vessel control loading order SDP model is established;SDP deep reinforcement learning pp DDQN algorithm is proposed to solve the SDP problem.In this algorithm,key decision features of loading order SDP including reshuffling and shifting is considered,PER is included to improve training and solving efficiency.Through the solution and simulation experiment of problem case,the effectiveness of the intelligent vessel control loading order SDP is verified.It proves the proposed method can effectively reduce the repeated shifting of the yard crane and containers during the loading operation,and realize the effective and continuous optimization of the loading sequence of intelligent vessel control commands.4.Intergration application of the proposed SDP method is simulated and applied to container terminal production environment.Application cases proves that the proposed SDP method can achieve effective,comprehensive,overall and timely optimization vessel loading process at the container terminal.The intelligent vessel control SDP method is verified through vessel-scale and terminal-scale case simulations.The integrated and implemented intelligent vessel control system has been applied to the production decision-making of seaport container terminal.Through practical application of this method,unmanned intelligent command transmission and handling during loading operation has been realized.According to the current production application result,proposed method managed to reach average quay crane handling efficiency of 30.84loops/h in more than 40 voyages,which shows this method has reached or surpassed manual decision in reducing handling conflict yard crane shifting and container shifting.The SDP method of container terminal vessel control proposed in this paper is of reference significance for solving other dispatching control problems of port operations.