Generative Model Driven By Winslow Functional

Probabilistic generative models,also known as generative models,are a class of models with extremely high practical application value in statistics and machine learning.It can be used to model different types of data,such as images,sounds,and text data.It can also be incorporated into reinforcement learning in several ways.so it has been widely used in data prediction,image processing,and text generation.However,how to design an effective generative model is also very challenging.The core of generative model is to estimate the target distribution parametrically.To simplify the discussion somewhat,we will focus on generative models that work via the principle of minimizing Kullback-Leibler divergence(KL).There are many kinds of generating models,but they can be divided into two categories.One is to construct an explicit density distribution.In these explicit density models,the density can be computationally tractable,so the update of the model is relatively direct.Such as Variational auto-encoder(VAE).The other generative models do not explicitly show the probability distribution of the data.Instead,it provides a way to reduce the direct interaction with the probability distribution.It is usually the ability to extract samples directly,such as using Markov chain to transform the existing samples.The model defines a way to stochastically transform an existing sample in order to obtain another sample from the same distribution.In particular,there is a special generative model with explicit density function,which is based on defining continuous,nonlinear transformations between two different spaces,called flow model.In other words,this kind of model starts from a simple distribution,combines it with a transformation.This transformation warps space in complicated ways and can yield a complicated distribution.If this mapping is carefully designed,the density is tractable too.This kind of models,such as NICE and Real Nvp,define a clear and manageable probability density distribution by designing a reversible encoder.However,it also has some disadvantages,such as the complex network structure,resulting in large amount of calculation and long training time.This paper also considers such a special generation model,by finding the mapping between the initial distribution and the target distribution to estimate the target probability distribution.We find that there are many similarities between finding this mapping and the moving mesh method.The moving mesh method,also called adaptive grid method,is an iterative grid redistribution method based on the variational method.It can change the grid distribution near the large change area of the solution of PDEs,and is especially effective in the process of solving PDEs with singular solutions.Such a grid movement is controlled by the Winslow functional.However,we can find that mapping the samples to the region with higher probability density is similar to the process of moving the grid point to the region with larger gradient of solution.Therefore,we can use Winslow functional to establish the relationship between these two problems,and then use this energy functional to build a new generative model.This article consists of three parts.The detailed work of this paper is listed as follows:(1)We introduce some basic generative models,including flow maps method and Stein variational gradient descent(SVGD)method.Some simple examples are presented to validate their numerical properties.(2)We introduce the source of our idea,an iterative moving mesh method,which can be used in solving partial differential equation.The definition of Winslow functional is briefly introduced,and its effect and principle are introduced through simple examples.This part provides good theoretical basis of the following discussion.Its effect and principle are introduced through simple examples.This part provides good theoretical basis of the following discussion.(3)After having a certain understanding of the generative model and the adaptive grid method,we briefly explain the connection between the two methods,and analyze the feasibility of applying the Winslow functional to construct the generative model.Then,we propose a generation model driven by Winslow functional,and briefly introduce its principle.We briefly introduce the principle and prove the relevant details.We show the its correctness by numerical experiments in the examples of one-dimensional and twodimensional,and make a simple comparison with the generation model mentioned before.(4)It is very complex to solve the partial differential equation in high dimensional cases.In order to apply our method to the problem of high dimension,we discuss how to realize our algorithm in the framework of neural network.Therefore,we briefly introduce the neural network algorithm for solving PDE,the Ritz algorithm and a more classic generative model,generative adversarial network(GAN).We discussed the design of the neural network structure of our models in detail,including the design of loss function,the treatment of boundary conditions and iterative process.Finally,some numerical examples are used to verify the effectiveness of this method,and a brief analysis is made.The results of numerical experiments in this paper show that such a generation model based on Winslow functional is effective.Whether using numerical solution or neural network to solve the problem,the method has achieved good results,and can reach a lower cross entropy in a lower number of iterations.The contribution of this paper is mainly reflected in the following aspects:(1)Our work provides a new direction for the research of generation model.We have successfully applied the Winslow functional used in the moving mesh algorithm to the generation model,and transformed the problem of solving the target mapping into the problem of solving the partial differential equation.We have successfully verified the effectiveness of the algorithm in some numerical experiments.In addition,we have carried out some numerical experiments on the related generative models,and give some preliminary analysis and exploration on the results.(2)We have successfully implemented our model in the framework of neural network.We introduce how to use neural network to solve partial differential equations,and then discuss in detail the difficulties and feasible treatment in the design of network structure for the generative model based on Winslow functional.Finally,the training effect of our model is verified in the case of low dimension.(3)There still remains some problems worthy of study.In the study of generative models,the numerical examples of the work in this paper are all carried out in low dimensions,and the target probability is explicit.This paper does not discuss the discrete data sets and high-dimensional problems.However,we believe that this model can also be applied to these problems.There are some shortcomings in our network framework design.For example,the process of calculating Jacobian matrix is not optimized.With the increase of dimensions,the calculation of Jacobian matrix will become very complex.These problems are worthy of further study.