Reinforcement Learning Based Maintenance Policy For A Deteriorating Machine With Multiple Yields

In a manufacturing system, machine deterioration commonly leads to degradationof product quality. Taking into the account the interaction between machine maintenanceand product quality issue, this paper proposes a method to identify maintenance policyfor a machine with deteriorating quality states represented by multiple yield levels. Whena machine is operating on a deteriorated quality state, it is still operational but with ahigher probability to produce defective parts. However the underlying yield level cannotbe directly obtained. Instead product inspection information with inspection errors isconsidered as system observation state, based on which maintenance decisions are made.The system is influenced by both yield deterioration process and random shocks. Ifwithout purposefully triggered minor maintenance, the machine will eventually fail to beoperational, and a major repair will be compulsively initiated. Modeled as a partiallyobservable Markov decision process (POMDP), this problem is solved using areinforcement learning algorithm, Q-learning, to obtain the maintenance policyassociated with each possibly observed system state so that the average system rewardrate for an infinitely long time horizon is maximized. The results of a set of simulationexperiments indicate that the reinforcement learning based maintenance policyoutperforms the conventional periodic maintenance policy in terms of average rewardrate and total maintenance time. The impacts of system parameters on the proposedpolicy are investigated in the simulation study as well.