Solving Dynamic Slab Relocation Problem Based On Reinforcement Learning

Slab yard is the intermediate link between continuous casting and hot rolling in iron and steel production process.Its logistics operations mainly include slab storage,slab shuffling,discharging and stacking.These logistics operations are carried out alternately,and the incoming slabs arrives dynamically.Efficient logistics scheduling in slab yard can reduce logistics cost in steel production process.Considering various logistics operations and dynamic arrival of incoming slabs,the dynamic slab relocation problem is proposed,and the moving order and allocation position of slabs should be determined.Dynamic arrival of incoming slabs refers to the arrival of incoming slabs in the process of slab extraction.Before the arrival of incoming slab,the information about the stage and type of incoming slab is not known.Considering the strong time-space coupling of the problem,a time-space network model is established.Aiming at the large-scale,multistage and dynamic characteristics of the problem,a model-based reinforcement learning algorithm is proposed.The main work of this thesis is as follows:1)A time-space network model for dynamic slab relocation problem is established.The dynamic variable is added to represent the arrival of incoming slabs.The objective is to minimize the total number of slab movements.The practical constraints such as stacking constraints,shuffling constraints and time window constraints are taken into account.Because there is dynamic variable that can not be solved,assuming that the arrival stage of the incoming slab is known,the dynamic problem is transformed into a static problem.The model is tested by CPLEX,and the validity of the model is verified.2)A model-based reinforcement learning algorithm is proposed.Aiming at the actual scale problem,a model-based reinforcement learning algorithm is proposed to solve the dynamic slab relocation problem,which belongs to sequential decision problem and has dynamic characteristics.The reinforcement learning algorithm decomposes the problem into a series of single-stage problems by reducing dimension in stages.For single-stage problems,it becomes known whether the incoming slab arrives at this stage or not,and the dynamic problem becomes a static problem.In order to solve the dimension disaster problem,the approximate value function is used instead of the optimal value function in Markov decision-making process.According to the influence of different logistics operations,the value function based on logistics operation is designed,and the sequential scheduling between the same logistics operations is further considered,the value function based on stack selection is designed,and the parameters and updating methods of the value function are set.The effectiveness of the proposed algorithm is verified by numerical experiments.3)Based on the model and algorithm mentioned above,an intelligent decision system for dynamic slab relocation is designed and developed.The system can allocate a reasonable position for slab during the process of slab storage and extraction.It also provides the functions of data analysis and data maintenance,which can carry out statistical analysis and management of different dimensions of slab yard information.