Reinforcement Learning-Based Differential Evolutionary Algorithm For Constrained Optimization Problems

Constrained optimization problem is an important research content in optimization field,which is widely used in many practical applications such as engineering manufacturing,job scheduling,antenna optimization design and resource allocation.In recent years,many excellent works have been proposed to solve constrained optimization problems based on differential evolution algorithm.However,these related works mainly design algorithms from the perspective of constraint processing mechanism,and rarely consider the correlation between constraints and objective functions.As a result,the algorithm will miss the feasible domain which may contain the global optimal solution in the search process.Therefore,this paper carries out algorithm research from the perspective of such correlation,proposes to use reinforcement learning technology to coordinate the weight between constraints and objective functions,abstracts the weight setting in the algorithm iteration process to Markov decision process in discrete time,and proposes constraint processing technology based on reinforcement learning from the aspects of single agent and multiagent respectively.And the corresponding differential evolution algorithm is designed.The main work and contributions of this paper are as follows:(1)A differential evolution algorithm based on single agent reinforcement learning(RLCODE)is proposed.In order to make full use of the correlation between the constraint and the objective function,RLCODE uses the correlation to control a weight parameter to coordinate the balance between the constraint and the objective function,so as to guide population optimization.In order to mine the correlation accurately,single-agent reinforcement learning is introduced,and the process of finding the correlation between objective function and constraint is abstracted into a decisionmaking process.Each decision represents an action on the weight,and then reinforcement learning is used to make decisions,and the correlation is mined through the "trial and error" mechanism.The comprehensive experiments were carried out on two sets of widely used benchmark test functions,and compared with seven wellknown constraint processing algorithms including CEC2017 constraint optimization competition champions.The experimental results show that RLCODE has good performance.(2)A differential evolution algorithm based on multi-agent reinforcement learning(MARLCODE)is proposed.To further improve the performance of the algorithm,multi-agent reinforcement learning technology is introduced on the basis of RLCODE.In order to alleviate the problem of motion space explosion caused by the increase of the number of agents in multi-agent reinforcement learning,the average field theory is introduced to convert the interaction behavior between multi-agents into the average interaction behavior of each agent and its neighbor agents,and transform the multiagent system into a double-agent system,effectively eliminating the limit of the number of agents.A comprehensive experiment was carried out on two sets of widely used benchmark functions,and compared with RLCODE and seven well-known algorithms including CEC2017 constraint optimization competition champions.The experimental results show that MARLCODE has good performance.