Optimal Control Of Multiple CSPS System With Two-type Product Mixed Flow

In current manufacturing enterprises, Conveyor-Serviced Production Station (CSPS) is a typical automated manufacturing system, in which parts are delivered to the production station with a finite-capacity buffer to be further processed. As an abstract model for automated production process, CSPS has been widely used in real industries. Therefore, the optimal control of this type of model has practical significance.This thesis is mainly concerned with the optimal control of multiple CSPS system with two type products, in which varieties of parts arrive at the station in an independent Poisson process. Different parts are stored independently in the buffers to be further processed, and the processing time is also stochastic. We proposed an operating mode based on equilibrium of products so that a balance of processing rate could be achieved between the two type products. Each station makes its decision independently, and local information of neighbor stations should be interacted. By using look-ahead range as control variable, the vacancies of all buffers in each station as state, a model-free reinforcement learning algorithm with the reaction-diffusion mechanism of multi-agent systems was applied to solve the coordinated optimization control problem. The optimization objective is to maximize processing rate in an infinite horizon and to improve the balance of each station. The simulation results suggest that the proposed operating mode could effectively balance the production of different varieties. We also discussed the impact of different parameters on system performance. The rationality of our proposed model and the effectiveness of our method are illustrated in our results,In the manufacturing enterprises, demand of customer tends to be the driving source of the production. Therefore, a demand-driven production mode is incorporated into the multiple CSPS system in this thesis. System with stochastic customer demand and finite bank capacities are investigated. By using look-ahead range as control variables of each station, the vacancies of all buffers in each station and vacancies of all banks as state, we further proposed a demand-driven multi-agent Q-learning algorithm to solve the optimal control problem of the system. Performance value such as customer attrition rate, total processing rate and load balance are evaluated. The simulation results validate the rationality of demand-driven production mode and the effectiveness of our method.