Research On Multimodal Multi-Objective Ensemble Optimization Algorithm For Homogeneous Learners And Its Application

Ensemble Learning is a top research topic in the machine learning area.Previous studies have demonstrated that combining multiple weak learners may compensate error and improve the prediction performance of learning system.However,how to balance accuracy and diversity is still a challenge.In this thesis,we regarded the generation process of base learners as a multimodal multi-objective optimization(MMO)problem.Then,the generalization ability and forecast accuracy can be improved if there exists an excellent MMO algorithm.This thesis studied MMO algorithms and ensemble strategies for tackling problems from the aspects of optimization ability,runtime,and high-dimensional situation in ensemble learning.Moreover,the designed MMO algorithms were applied to solve feature selection and power load forecasting problems.The main studies can be summarized below:1.In order to improve the optimization ability of MMO algorithm,a selforganizing map based MMO algorithm(MMOPIO)was designed.This algorithm used self-organizing map to locate multimodal classifiers around the best particles.Based on a consolidation parameter,MMOPIO simplified the structure of pigeon-inspired optimization.Special crowding distance and elite learning strategy were also applied to improve diversity and convergence performance both in decision space and objective space.According to the multimodal characteristics of feature selection,an ensemble optimization algorithm based on homogeneous learners(MMOPIO-ENS)was designed to balance accuracy and diversity.Moreover,multimodal solution based ensemble strategy(MS-EnS)was designed to solve problems with incomplete Pareto front or constraints in objective space.The experiment results validated that the proposed MMOPIO-ENS can maintain better generalization ability and forecast accuracy.2.In order to improve efficiency,an efficient hybrid MMO algorithm(MMOHEA)was designed based on the two-archive model.There were two diversity maintenance strategies.The first strategy was competitive swarm optimizer with niching local search and the other one was differential evolution algorithm with reverse vector mutation.Moreover,both strategies were free of niching parameters.MMOHEA obtained good performance even with a small population size.The efficient non-dominated sorting method was applied to improve algorithm’s calculation efficiency.Based on MMOHEA,an ensemble optimization algorithm based on homogeneous learners(MMOHEA-ENS)was designed to balance accuracy and diversity.Based on the characteristics of feature selection,MMOHEA-ENS combined classifiers with higher forecast accuracy to build the forecast model.The proposed accuracy preference based ensemble strategy(AP-EnS)can be used to solve problems with complete Pareto front and accuracy preference.The experiment results validated the effectiveness and efficiency of MMOHEA-ENS.3.In order to improve the optimization ability and efficiency of MMO algorithm under the high-dimensional situation,a novel algorithm named MMOWVA was designed.MMO WVA took advantage of comprehensive weighted vector angle to estimate the distribution state of solutions in the decision space and objective space.Moreover,MMOWVA applied shift-based density estimation to increase the selection pressure in the objective space.In order to obtain more multimodal solutions,MMOWVA used niching local search and reverse vector mutation.The two-archive model and treebased non-dominated sorting were also used to improve efficiency.The efficacy of MMOWVA was validated on some large scale MMO test problems.Aiming at solving high-dimensional feature selection problems,an ensemble optimization algorithm named MMOWVA-ENS was built.The experiment results showed that MMOWVAENS can balance accuracy and diversity,and obtain a higher forecast accuracy.Aiming at solving power load forecasting,an ensemble method that combining predictors around the knee point in the objective space(KPA-EnS)was proposed to meet the modeling requirements of keeping multiple objective values optimal at the same time.The effectiveness of the proposed method was validated by the experiments on some practical power load datasets.