Research On Real-time Scheduling Method Of Multi-agent Discrete Manufacturing Workshop Driven By Digital Twins

Workshop scheduling optimization is one of the core technologies of intelligent manufacturing and modern management,and is an important way for discrete manufacturing enterprises to achieve efficient production organization,meet personalized user needs,and quickly respond to the market.With the improvement of the level of intelligence and flexibility within the workshop,coupled with the frequent occurrence of dynamic interference events,the complexity,dynamism,and integration of scheduling problems have become increasingly prominent,posing higher requirements for the realtime response ability of the workshop.How to use technologies such as digital twin and distributed artificial intelligence to portray the inner evolution law of production process in discrete manufacturing workshops,analyze complex characteristics of workshop scheduling system operation,and realize the resource allocation in discrete manufacturing workshops under the action of dynamic factors through the intelligent decision-making is an urgent problem to be solved.Therefore,this paper is oriented to methods related to the real-time scheduling of discrete manufacturing workshops and provides the theoretical basis and technical support for workshop operation state analysis and control decisions.The research takes two typical scheduling problems of the hybrid flow-shop scheduling(HFSP)and the flexible job-shop scheduling(FJSP)in discrete manufacturing plants as research objects and explores key technologies around the multi-agent real-time scheduling framework,the digital twin modeling and the interaction of shop scheduling,and multi-agent reinforcement learning scheduling decision of HFSP and FJSP.The main research contents of the paper are as follows:(1)Aiming at real-time scheduling problems and demands of discrete manufacturing plants,a two-layer logic model of the real-time scheduling is established from the perspective of system integration based on the distributed artificial intelligence multiagent and the digital twin technology,and the value of digital twin for multi-agent systems to sense and act on the environment is emphasized.The multi-agent composition and its interaction behavior are defined,and a super heuristic real-time scheduling optimization strategy is proposed to lay the foundation for realizing dynamic iterative optimization of agents.Accordingly,a digital twin-driven multi-agent discrete manufacturing workshop scheduling framework and operation mechanism are constructed,and key technologies such as the digital twin modeling,the interaction,and the multi-agent real-time scheduling decision are proposed.The real-time scheduling framework supports the selfawareness,the self-decision,and the self-execution of the scheduling process at the workshop operation level,and the self-organization,the self-learning,and the selfoptimization of the agent at the decision optimization level,and provides a method system for the real-time scheduling of the discrete manufacturing workshop.(2)For problems of the digital twin modeling and the virtual-real mapping of the workshop scheduling,the composition and the operation mechanism of the twin model are defined,the modeling method is studied and the modeling process is designed.A production factor domain ontology model with a virtual-real interaction interface and a hierarchical MACTPN production process behavior model with scheduling knowledge rules are constructed to describe the complex operation logic of a discrete manufacturing workshop.The digital twin multidimensional virtual-real interaction mechanism is proposed,and the fusion between elements,behaviors,and rules are carried out through production factor virtual-real interaction interfaces and data mapping rules to realize the simulation operation of the digital twin model,which provides an interactive environment for the agent real-time scheduling decision and reduces the dependence of the agent scheduling decision on the physical workshop data.By driving the operation of the digital twin model through data mapping and the operation of the physical workshop through the instruction mapping,it realizes the reflection of reality and control of reality with reality and supports the dynamic iterative optimization of the agent for scheduling decisions.(3)A super heuristic real-time scheduling method based on the centralized training and the decentralized decision with the deep reinforcement learning of the heterogeneous multi-agent is proposed for the hybrid flow shop scheduling problem with eligibility constraints.The high-level selection policy is generated by the workpiece sorting agent,the machine selection agent,and the machine failure agent;the low-level scheduling rules are designed with eligibility constraints,which constitutes the action space of the agent;the observation space of the agent is formed with the workshop operation state and environment information;the completion time and assembly set rate are the scheduling objective,and the reward function of the agent is designed,and the agent is based on The heterogeneous multi-agent training algorithm is designed based on MAPPO.Under the HFSP digital twin simulation environment,the interaction and training of the agent and environment are realized.The simulated cases and actual production data show that the proposed method is better than heuristic scheduling rules and can effectively solve the real-time scheduling decision of HFSP with eligibility constraints.(4)For the dual-resource constrained FJSP real-time scheduling problem containing robot multi-processing cells,a homogeneous multi-agent multi-task real-time scheduling framework is proposed,where each agent completes the workpiece sorting,the machine selection,and the process planning decision subtask within the cell.The action space of the agent is designed based on the dual resource constraint,the global reward of the agent and the local reward function of the each subtask are designed with the scheduling objectives such as minimizing the drag time and the resource utilization,and the homogeneous multi-agent multi-task reinforcement learning algorithm is designed based on the multi-gate expert hybrid structure and the DDQN.In centralized training,the value network evaluation is used to optimize the policy network,and the attention mechanism is introduced to achieve the multi-agent collaboration and the information sharing among multiple tasks;in decentralized decision-making,each agent uses local observation and policy network to achieve multi-task decision making.In the built FJSP digital twin simulation environment,the simulated arithmetic cases and actual production data show that the proposed method can adapt to the dynamic changes of the FJSP workshop and realize the dual-resource FJSP real-time scheduling.(5)For the real-time scheduling of a multi-agent discrete manufacturing workshop driven by a digital twin,the architecture and the functional module of the prototype system are designed,and the human-computer interaction interface is developed,and the hardware deployment and the implementation method of the digital twin system for the discrete shop scheduling is proposed.Through the application validation of the prototype system,the usability and the effectiveness of the module of the data collection,the data management,the workshop production state visualization,the virtual-real mapping,and the intelligent decision-making of the production process in the built digital twin processing cell are verified.Real-time scheduling driven by the digital twin is achieved,which demonstrates the effectiveness and the feasibility of the proposed method.