Research On Reinforcement Learning Based Order Acceptance Model In Make-to-Order Enterprises

In the increasingly competitive manufacturing environment, more and more maunufacturing enterprises increasingly adopt make-to-order (MTO) mode in the modern production in order to quickly response to the distinct requirements of customers, aiming at gaining profit and competitive advantage in the market. For MTO enterprises, there is a trade-off between the revenue brought in by a particular order, and its associated costs of processing during the order acceptance process. Hence, there are significant theoretical and practical values to selectively accept orders, to maximize the long-term profit, which has already caused extensive concerns of the business circles and academia. Judging from the existing situation, aiming at the characteristics of different orders and MTO enterprises, many mathmatical models and calculating methods are proposed which provide workable strateges to solve the problem of order acceptance, but most of existing studies assume that market demand is static. Less research is developed for the stochastic and dynamic environment, and for the situation that jointly considers the factors like the general customer level, delay penalty cost and capacity constrains for each order. In addition, for the existing models and methods, it is difficult to construct accurate model when the environment is complex and uncertain, and it is shown NP-hard when calculating the model. There is less research involving the implementing of decision support system for order acceptance problem. It is thus clear that further research is still required for the order acceptance problem and decision support system.Reinforcement learning (RL) avoids computing transition probabilities and reward matrics, and it is suitable for solving the sequential decision problems under stochastic and dynamic environment. Besides, the combination of multi-agent technology and web service technology can provide effective mechanisms and solutions to the implementation of decision support system. Thus, it is notable and realistic to study the reinforcement learning based order acceptance model and decision suppot system in MTO enterprises.The contributions of this study lie in fourfold:Firstly, this study refines the order acceptance problem that considering the factors like the characteristics of finite orders and capacity constrains under stochastic and dynamic environment. With regard to the shortcomings of the existing research, this study refines a more realistic order acceptance problem under stochasitic and dynamic environment, not only taking the production cost, delay penalty cost and reject cost into account, but also considering the factor of customer level.Secondly, this study proposes the reinforcement learning based order acceptance model and solving method, which combined with RL’s model-free thought. A semi-markov decision process based order acceptance model is built, along with a SMART algorithm based solving method.Thirdly, this study presents the simulation experiment, and the reliability and validity of reinforcement learning based order acceptance model and solving method are verified. The simulation results indicate that the RL based order acceptance strategies performs better than the first-come-first-serve (FCFS) based strategies. Moreover, the necessity and importance of considering the customer level are verified by comparing the two scenarios that whether or not to consider the customer level.Fourthly, the RL based order acceptance decision algorithm is embedded into the multi-agent system, and the order acceptance decision support system (OADSS) is designed and implemented based on multi-agent technology and Web service technology. The prototype of OA_DSS is developed using the development software like Eclipse.The proposed order accept strategy based on reinforcement learning and the order acceptance decision support system in MTO enterprises, can better assist decision makers quickly and accurately response to the requirements of their customers. In addition, this study enriches and expands the research contents and methods of order acceptance problem, and it can be a reference for further research work.