Research On Flexible Job Shop Scheduling Based On Reinforcement Learning Algorithm

Manufacturing industry is the core part of national economy.Optimizing production scheduling is very important to improve workshop production efficiency and reduce production cost.Flexible job shop scheduling is a representative problem in modern manufacturing industry.In this paper,the mathematical models of flexible job-shop scheduling problems under single objective,multi-objective and dynamic environment are set respectively,and the corresponding Markov decision process is constructed,and the improved reinforcement learning algorithm is used to solve the problems.Specific work is as follows:(1)For single-objective flexible job shop scheduling problem.A mathematical model is constructed to minimize the maximum completion time and the problem is transformed into Markov decision process.The general feature description state set was extracted from the scheduling environment,and the action set was constructed by processing time,workpiece completion rate and machine utilization rate,and the direct reward and indirect reward were set to describe the reward function together.Based on DQN algorithm,D5QN algorithm is proposed.It is verified on Mk standard data set,and the results are better than the heuristic algorithms and other DQN optimization algorithms proposed in recent years.(2)Flexible job shop scheduling problem for multiple objectives.The goal is to minimize the maximum completion time and reduce the total machine wastage.Two groups of coefficients of total machine loss were added to the state set,the total number of actions in the moving space was increased,and the expected value of completion time and total machine loss was set to describe the reward function.A2c-Gauss algorithm is proposed by adding two parts of Gaussian noise on the basis of A2C algorithm.Compared with other methods,the efficiency of the method is verified.(3)Aiming at dynamic flexible job shop scheduling problem.Set machine random fault constraints to minimize the total delay.By adding the actual delay,estimated delay and other feature description state sets,three groups of complex action space rules are designed,and the reward function is set according to the actual delay and other indicators.GC-PPO algorithm is proposed by adding gradient clipping technique into PPO algorithm.Compared with PPO algorithm,the proposed algorithm can obtain a scheduling scheme with smaller total delay in a shorter period of time on the custom data set.In conclusion,this paper constructs mathematical models and corresponding Markov decision processes respectively for the three sub-problems of flexible job shop scheduling,proposes an improved reinforcement learning algorithm,and verifies the effectiveness of the algorithm on the simulation data set,which provides a new idea and inspiration for reinforcement learning to solve flexible job shop scheduling problems.