Job-shop Scheduling Using Deep Reinforcement Learning Methods

Advancing with the Industry 4.0,traditional manufactories are faced with the challenge of transforming to the new Smart Manufacturing systems and methodologies.Job-Shop Scheduling refers to the process of optimizing the order of the production operations,which plays a ubiquitous role at improving the manufacturing productivity.Job-Shop Scheduling Problems(JSP)focuses on how to schedule the resources so that the overall job completion time is minimized.While lots of scheduling methods are doing the job,most of them suffer from long solving time and narrow generalization across different scheduling problems.This article attacks these two limitations by the following work:1.This article uses Deep Learning(DL)Methods to learn a neural network based feature extractor which both generalizes on different job states and reaches better computational efficiency.Since JSP comes with no available training data and is by complexity NP-hard,we complement a Reinforcement Learning(RL)approach to generate the data and propose an unsupervised static JSP solving model.2.As RL agent needs to learn from the interaction data with the environment,we designed a JSP simulation environment and solved data problem by providing the compatible interfaces to major RL frameworks.3.As the static RL JSP solving model is unable to tackle the dynamic job arrivals,we designed a JSP environment feature representing method utilizing graph data structure.And by piping it with the Graph Neural Networks we show our proposed RL method on dynamic job scheduling problems performs overall better than traditional heuristic methods.This article proposed two RL models to cope with the static and dynamic job scheduling problems separately.The benchmarking results on multiple testing problems show that our model beats traditional methods in terms of computation efficiency and generalization.Our method is also competitive with the state-of-the-art RL method on minimizing the scheduling makespan.