Research On Resequencing Scheduling Based On The Deep Reinforcement Learning On Automotive Paint Shop Manufacturing System

In recent years,automobile manufacturers have produced cars with multiple car-models and colors in the same hybrid production line,which increase production flexibillity and reduces operating costs.A complete automotive product needs to experience four manufacturing processes,which runs through the Press shop,Body shop and Paint shop,and finally delivers it to the Assembly on time and in sequence.In theory,the products left from the body shop enter the Paint shop in the order of assembly demand sequence.However,resequencing scheduling problems exist in the Paint shop due to the production constraints and requirements.Meanwhile,the production sequence of completed Paint shops' jobs strives for high sequence adherence to the planning demand sequence of the assembly to achieve lean production.Therefore,it is significant to investigate the resequencing scheduling problem of the paint shop-assembly production system under mixed-model production system.On the one hand,scholars considered the resequencing scheduling in the paint shop and the sequence adherence scheduling when the painted cars enter the assembly as two individual scheduling optimization problems but ignore the linkage between the paint shop and assembly.On the other hand,the sequencing scheduling of mixed-model production system is NP-complete.Previous researches have proposed algorithms and verified for small-scale cases,but it is difficult to solve the realworld large-scale problems,which involve the scheduling of 10+colors,10+car-models and 1000+cars,the previous exact algorithms,heuristics and meta-heuristics cannot achieve rapid optimization scheduling scheme.Based on practical applications,this research analyzes parameters that need to be considered in the actual painting production process.Combined with the complex production constraints,buffer structure and production line layout,the management and optimization method of production scheduling problem in the paint shop is constructed.The hybrid algorithm of deep reinforcement learning and heuristic algorithm is used to solve the problem.The proposed approach not only has performed better than benchmarks but also used less computational time.We observed that the proposed approach provides production and operation management with stronger sequence flexibility,cost reduction,and environmental friendliness in daily operation.The main contents of this study are as follows:(1)Research on sequencing scheduling problem in Paint shopsBased on the real-world production process,KPIs and the hybrid flow shop scheduling structure,this study analyzes and discusses the factors that need to be considered in the production process of the paint shop and defines the objectives,decisions and scheduling framework.Two sequencing scheduling metrics are proposed from the perspective of costs and scheduling quality.A three-stage Hybrid Flowshop Scheduling with limited Buffers under Mixmodel production system is designed in this paper.A rule-based heuristic algorithm is proposed to solve an actual scheduling scenario to verify the effectiveness of the proposed model and algorithm.(2)Research on sequencing scheduling problem based on deep reinforcement learningBased on(1),the scheduling problem of parallel painting production line on stage two is studied.From the perspective of operational costs,a single-machine flowshop scheduling model with finite mix-bank buffer is established to minimize the cost of color change.A colorhistogram model is proposed to reduce the decision dimension.Then,a Deep Q-network algorithm based on deep reinforcement learning is proposed to solve the color-batching resequencing scheduling problem.The proposed model and algorithm outperformed the conventional heuristic algorithm,and readily apply in the production control of automotive paint shops to resolve order-resequencing problems.(3)Research on multi-objective sequencing scheduling problem based on multi-objective reinforcement learningBased on(2),the multi-objective optimization scheduling problem with the same condition is studied.From the perspective of operational costs and scheduling quality,the colorbatching and sequence adherence resequencing scheduling problems on stage two are considered.A single-machine flowshop multi-objective scheduling model with finite mix-bank buffer is established to minimize the cost of color change and tardiness.Multi-objective deep Q-network algorithm based on the multi-objective reinforcement learning is proposed to solve this multi-objective scheduling problem.Compared with conventional heuristic algorithm and envelope Q-learning algorithm,the proposed model and algorithm can effectively and quickly solve the optimization problem.The algorithm has a strong generalization ability for objective preferences and can be used in real-time in practical applications.(4)Research and application of the multi-objective sequencing scheduling problem for Paint shop-Assembly production systemBased on the multi-objective optimization algorithm proposed in(3),the framework of scheduling problem proposed in(1)is deepened,and the scheduling problem of paint shop is extended to the paint shop-assembly production system.From the perspective of operational costs and scheduling quality,the two optimization objectives are to minimize the total cost of color change and total non-sequence adherence of completed cars.The three-stage hybrid flowshop sequencing scheduling model with limited buffer under mixed-model production system is reanalyzed,and a hybrid algorithm combining heuristic algorithm with multiobjective deep Q-network algorithm based on multi-objective reinforcement learning is proposed.The multi-objective scheduling scheme is designed,which integrated with discreteevent-simulation,manufacturing execution system and programming logic controller and artificial intelligence algorithm,to achieve the goal of reducing operational costs and efficient production.It can provide scientific decision-making basis for modern automotive manufacturers.