The Research On Dynamic Parallel Machine Scheduling Based On Reinforcement Learning

Manufacturing industry is the cornerstone of modern industry.With the innovation and breakthroughs in important areas such as information technology,manufacturing industry has begun transition to intelligent and green.Proper scheduling of the production process can enhance corporate competitiveness.But in the actual production process,some dynamic factors of the system or man-made,such as change of the processing time of jobs,the failure of machines,etc.,the original scheduling scheme become suboptimal or infeasible,result in lower product quality,longer production cycles,and increase energy consumption.Therefore,reasonable dynamic scheduling of the production process is conducive to improving the level of enterprise intelligence and green,reducing production costs,improving product quality,and reducing energy consumption.In this paper,we are motivated by the background of the blast furnace-converter section in steel enterprises,we analyze the transportation characteristics and dynamic factors in the production process,refine the dynamic parallel machine production scheduling problem and the coordinated scheduling problem of transportation and parallel machine production.The objective function is to minimize the expected stay time of jobs,an optimization model for production scheduling and coordinated transportation and production scheduling are established.Reinforcement learning is a type of machine learning.Compared with other methods,reinforcement learning does not need a definite problem model,which is suitable for solving dynamic scheduling problems.But the state space of scheduling problems is too large,and it is easy to fall into a "dimensional disaster" when using reinforcement learning algorithms.Therefore,based on the Q-learning algorithm and function approximation,we design a solution algorithm that can not only solve the data memory problem of large-scale scheduling problems,but also has certain advantages in solution accuracy and has stability.The main research contents are as follows:(1)Taking the converter steelmaking production process as the research background,a dynamic parallel machine production scheduling problem is refined.The release time of jobs,the processing time of jobs and the failure of machine in the production environment are random variables.The objective function is to minimize the expected stay time of jobs,a hybrid integer programming model is established.The production scheduling problem is transformed into a multi-stage decision problem.Based on the impact of random variables such as the processing time on the production environment,the state and action in the reinforcement learning algorithm are divided,the linear function generalizer is used to solve the data storage problem,and the Q-learning algorithm is used to solve the problem.The experimental results show that the Q-learning algorithm based on the linear function generalizer has better effectiveness and stability.(2)Taking the continuous production process of blast furnace-converter as the research background,the problem of coordination scheduling of transportation and parallel machine production is refined.In the transportation stages,there are multiple transport vehicles,the capacity of the transport vehicle is 1,and the transport time is limited.In the production stages,the release time of jobs and the processing time of jobs are random variables.The objective function is to minimize the expected stay time of jobs.A coordination scheduling of transportation and production model is established.The two-stage problem of transportation and production is transformed into a multi-stage decision problem,the state space and action space of the transportation and production stages are respectively set according to the transportation characteristics and dynamic factors in production.The Q-learning algorithm based on the linear function generalizer is applied to solve the problem.The experimental simulation results verify the effectiveness of the Q-learning algorithm in solving the coordination scheduling problem of transportation and parallel machine production.