Sampling And Design Of Complex Response Surfaces

Many scientific experiments involve exploration of unknown complex systems.Generally,people are interested in the relationship between response and variables,which is called response surface.The objectives of exploring response surface include identifying the important variables,prediction of the response surface,optimizing the complex system and so on.These objectives need to sample and design of response surface.In recent years,the commonly used sampling and designs of response surfaces are space filling designs,which spread the design points evenly in the region.However,in some applications,such as finding the optimum,the space filling designs will fail to capture the characteristics,which are not useful.This paper presents a new sampling method for exploration complex response surface.This sampling method can be used not only in system optimization,but also in Bayesian computation and high dimensional numerical integration.In addition,some large sample properties of a class of response surface designs with compound variables are discussed in this paper.The main work is shown as follows:1.We propose a new kind of deterministic sampling based on the KullbackLeibler divergence and the kernel function estimator,which is called KullbackLeibler points.When we obtain some prior information,for example the complex surfaces contain many local optima or can be expressed explicitly,and the purpose is to find the maximum response surface,due to the traditional space filling designs are well spread in the space,so design points may placed in the zero-yield regions which are not useful.Therefore,in this paper,we regard the complex response surface as the unnormalized density function,and then,by minimizing the Kullback-Leibler divergence between this density and the kernel density estimator of a points sets,we propose a new sampling method to obtain the representative points of complex response surface,which is called Kullback Leibler(KL)points.KL points can capture the characteristics of the surfaces and make the points are separated as far as possible.In this paper,we generate n KL points by two-step approximation.Firstly,in order to avoid the high dimensional integral,we generate KL points by minimizing Monte Carlo approximation of the Kullback-Leibler divergence.Secondly,as a result of generating n KL points at the same time is very difficult,so we adopt a one point at a time greedy algorithm for the generation of n KL points.Figure 2.2-2.4 and Table 2.1-2.2show that KL points have apparent advantage in space filling property and numerical integration compared with support point(Mak and Joseph,Ann Stat,2018,[2])and minimum energy design(Joseph et al.,Technometrics,2019,[1]).2.When the surfaces can't be expressed explicitly or time-consuming,we propose an adaptive Kullback-Leibler(AKL)points set based on the KullbackLeibler divergence and the kernel function estimator.When the surfaces have complex structure,for example it can't be expressed explicitly or time-consuming.Then,we fit a stationary Gaussian process model based a initial space filling design to replace this density,and then generate a class of generalized KL points by minimizing the Kullback Leibler divergence between the Gaussian process model and kernel density estimator.We adopt an adaptive strategy to generate KL points,at every KL point is generated,the Gaussian process model is updated sequentially to generate the next KL point.The resulting points are called adaptive KL(AKL)points.AKL points can be used in sampling of complex density functions,such as Bayesian posterior,and optimizing complex black-box functions(see Figure3.1-3.3).3.When the dimension of response surface is large,we propose KullbackLeibler-nn points based on the Kullback-Leibler divergence and the nearest neighbor estimator of function.When the dimension of complex response surface is large,we can still regard it as as the unnormalized density function.In this paper,the KL points can be obtained by minimizing the Kullback-Leibler divergence between the density function and the nearest neighbor estimator of density among a points set,which are called Kullback Leibler-nn(KL-nn)points.We use a form of local search algorithm based on MCMC to generate KL-nn points.Numerical simulations show that KL-nn points have better space filling property(see Figure 3.6),and have smaller integration error than support point and minimum energy design in the case of high dimensional numerical integration(see Figure 3.7).4.We derive some sampling properties of marginally coupled designs which are used to explore the complex surfaces involving both qualitative and quantitative variables.The traditional design method can't be used for complex surfaces involving both qualitative and quantitative variables.For constructing designs for these complex surfaces,Qian(JASA,2012,[3])suggested using sliced Latin hypercube designs.Such a design for each level of any qualitative factor,the corresponding design points of quantitative are space filling designs.He and Qian(Stat Sin,2016,[4])derived some sampling properties of sliced Latin hypercube designs to show that the space-filling properties.However,the run sizes of these designs can be very large,even for a moderate number of qualitative factors.To solve this issue,Deng et al.(Stat Sin,2015,[5])proposed a new type of designs called marginally coupled designs.These designs can greatly reduce the run sizes,and thus save the cost.Due to the structure of marginal coupled designs are different from sliced Latin hypercube designs,hence the sampling properties obtained by He and Qian(Stat Sin,2016,[4])can't be directly applied marginal coupled designs.Motivated by these,we derive some sampling properties of marginally coupled designs and develop the central limit theorems.