Civil Aero-Engine Life Cycle Maintenance Decision Method Based On Reinforcement Learning And Its Application

Civil aero-engine is a complex equipment,with high reliability requirements,long life,high maintenance costs,maintenance complex and other characteristics.Reasonable arrangement of the maintenance occasion and the mainenance strategy of civil aero-engine in life cycle,not only can effectively improve the reliability of the aero-engine,but also can greatly reduce the maintenance costs of the aero-engine.Therefore,the research on the maintenance decision method of civil aero-engine in its life cycle has important theoretical significance and economic value.At present,most of the research on the maintenance decision of civil aero-engine is to separate the maintenace occasion,the maintenance level of aero-engine module and the life limit parts replacement,while it influence each other in the actual engineering.In view of the shortcomings of previous research,this paper comprehensively considers the maintenance occasion,the maintenance level of aero-engine module and the life limit parts replacement strategy,and establishes three different civil aero-engine maintenance decision models,then uses different methods to solve.The civil aero-engine life cycle maintenance decision includes maintenance occasion decision,the life limit parts replacement strategy and the maintenance strategy of aero-engine modules.Aiming at the problem that the solution space of aero-engine maintenance decision model which directly established based on maintenace occasion,aero-engine module maintenance level and life limit parts replacement strategy is too large to get a good solution.This paper decoupling civil aero-engine maintenance decision problems.A civil aero-engine maintenance decision model is established with the maintenance occasion as a decision variable and with the minimum total maintenance cost in life cycle as the optimization goal.Under the condition of determining the maintenance occasion,a life limit parts replacement rule is proposed to solve the optimal life limit parts replacement strategy and cost,a dynamic programming algorithm is used to solve the optimal maintenance strategy and cost of the aero-engine module maintenance.Finally,the particle swarm optimization algorithm is used to solve the civil a ero-engine maintenance decision model.The simulated data and real-world data is used to evaluate and validate the proposed method.The results show that the proposed method can effectively solve the civil aero-engine maintance time decision problem.For a large number of aero-engine maintenance decision problems,the particle swarm optimization algorithm has the disadvantages of slow speed,low efficiency,and results insufficient.Reinforcement learning can storage the the knowledge,and it has the characteristics of fast solution speed.Therefore,the Q learning algorithm in reinforcement learning is introduced into the civil aero-engine maintenance decision problem.Firstly,the aero-engine maintenance process is regarded as a Markov decision process with each flight cycle to decision,the civil aero-engine life cycle maintenance decision model is re-established,and the state,action,reward,Q table in the model are designed respectively.Then,the Q learning algorithm is trained to solve the model.Finally,the numerical experiment method is used to evaluate and validate the proposed method.The results show that the maintenance decision method of civil aero-engine based on Q learning is suitable for the maintenance decision of civil aero-engine with a small number of life limit parts and engine modules,and the feasibility of reinforcement learning application to the maintenance decision problem of civil aero-engine is verified theoretically.In view of the fact that when the number of life limit parts and aero-engine modules is large,the state space increases,and the Q learning algorithm has the curse of dimensionality.The deep Q learning algorithm is applied to the civil aero-engine maintenance decision problem.Combining the rules of civil aero-engine maintanence occasion,the life limit parts replacement and the engine modules maintanence strategy to reduce the state space and action space.Then establish the maintanence decision model of civil aero-engine based on deep Q learning.In order to accelerate the convergence of the algorithm,a deep Q learning pre-training method is proposed,which generates a certain amount of better data based on rule s to pre-train the network.Finally,the simulated data and the real-world data of airlines are utilized to verify the proposed method.The results show that in the face of large-scale civil aero-engine maintenance decision problems,the proposed method can quickly get a good solution.Finally,based on the aero-engine maintenance decision method studied in this paper,combined with the needs of civil company,a prototype system for aero-engine maintenance decision was designed and developed,and it can pro vide technical support for civil aero-engine life cycle maintenance decision.