Multivariate Correlation And Selection Strategy Applied To High Dimensional Bayesian Optimization

Artificial intelligence is a new technical science that studies the theoretical methods and application systems used to simulate,extend and expand human intelligence.It has a profound impact on the future economic and social development.The development of big data has opened up a new era for artificial intelligence.However,many applications are usually accompanied by complex characteristics such as large-scale heterogeneous computing and distributed storage ar-chitecture.Therefore,it is of positive significance to solve the problem of artificial intelligence under the background of big data based on mathematical theory.Bayesian optimization is one of the most advanced and promising technologies in the field of artificial intelligence and probabilistic machine learning.It skillfully designs the probabilistic surrogate model and acquisition function.The ideal solution can be obtained only after a few times of objective function evaluation.It is very suitable for solving complex research problems such as non-convex,multimodal,black box and high evaluation cost.However,the research of extending it to high-dimensional space still faces some challenges.This paper designs variable selection strategy based on the study of multivariate correla-tion.Some applied researches on high dimensional Bayesian optimization are carried out.The details are as follows:Firstly,the analysis method of multivariate correlation is proposed.A JMIC(Joint Max-imum Information Coefficient)measure for judging the correlation between multiple random variables and response variable is presented.Meanwhile,the MMIE(Multivariate Maximum Information Entropy)method for analyzing the redundancy correlation between these random variables is given.Furthermore,the definition of correlation matrix is improved for the large number of random variables and data samples.Another MMIE_Ymeasure that identifies the mag-nitude of the correlation between multiple random variables and response variable is then pre-sented.The properties of the three definitions are analyzed in detail theoretically.The variable selection strategy MSS(Multivariate Selection Strategy)based on the correlation of multivari-ate variables is formulated.Finally,the effectiveness of the proposed multivariate correlation theory and its selection strategy is verified by tests on data sets.Then the multivariate correlation theory and its selection strategy are applied to high di-mensional Bayesian optimization.Two high dimensional Bayesian optimization algorithms are proposed.The first high-dimensional Bayesian optimization MCS-BO(Bayesian Optimization Based on Multivariate Correlation Selection)algorithm based on variable correlation selects some important dimensions of the optimization target by variable selection strategy.Bayesian optimization model is constructed in subspace to alleviate the difficulty of acquisition function optimization in high dimensional space.For the other unselected dimensions,a filling strategy is defined to evaluate the objective function and update the probabilistic surrogate model.The second high-dimensional Bayesian optimization MCC-BO(Bayesian Optimization Based on Multivariate Correlation Clustering)algorithm based on variable correlation formulates clus-tering criteria according to multivariate correlation.A new clustering algorithm is designed to group high dimensional variables.Thus,the original high dimensional optimization problem is transformed into some middle or low dimensional problems which are easy to deal with.In addition,the regret bound problem of MCS-BO algorithm and MCC-BO algorithm are ana-lyzed theoretically.Simulation experiments on some test functions verify the effectiveness of the high-dimensional Bayesian optimization algorithm based on variable correlation.