Research On Fast Multi-point Expected Improvement-based EGO Algorithm

Those optimization problems that require a lot of time to calculate their objective function are often called expensive optimization problems in the field of optimization design.It is usually impossible to directly use meta-heuristic algorithms to solve expensive optimization problems because meta-heuristic algorithms such as genetic algorithm and particle swarm optimization algorithm require thousands of evaluations on the objective function to obtain relatively reliable results.EGO(Efficient Global Optimization)algorithm is one of the most widely studied and applied algorithms for solving expensive optimization problems.The standard EGO algorithm uses the Kriging surrogate model to approximate the objective function,and continuously selects new points for calculation by maximizing the EI(Expected Improvement)criterion to search for the optimal solution of the problem.As computing resources become more abundant,the standard EGO only selecting one infill point at one time to perform expensive calculations wastes a lot of computing resources.Parallel EGO algorithm can make full use of computing resources and greatly improve the efficiency of the algorithm.As one of the most classic criteria for parallel optimization,qEI has good mathematical properties and optimization efficiency.But it also has two major problems.The first problem is that it takes a lot of time to accurately calculate qEI when the parallel number q is large.The second problem is that the dimensionality of the optimization problem of the qEI is very high,and the problem of maximizing qEI is difficult to solve.In order to solve these two problems,this work makes further research and improvement.The major contributions of this work are:1)This work proposes a new fast multi-point expectation improvement criterion,which takes into account the local search and global search capabilities and greatly reduces the computational cost.Experiments and analysis prove that the proposed algorithm has lower computational cost and better optimization efficiency than qEI.2)The curse of dimensionality makes optimizing the qEI function very difficult.This work proposes to decompose the FqEI maximization problem into q sub-problems to make each sub-problem correspond to one infill sample.Based on this idea,this work proposes to use cooperative co-evolutionary algorithms to solve high-dimensional internal optimization problems to alleviate the curse of dimensionality.Experiments show that the optimization based on cooperative co-evolutionary algorithms can greatly improve the optimization results of the algorithm,thereby alleviating the dimensional disaster problem of qEI in a certain degree.3)In order to verify the optimization efficiency of the algorithm proposed in this work in practical applications,this work selects the popular Neural Architecture Search application as our experiment.The experimental results show that,compared with the traditional qEI criterion,the criterion proposed in this paper can reduce the computational cost and improve the optimization efficiency of the Neural Architecture Search problems.