A Study On Efficient Bayesian Inference Methods Using Manifold Structures

Bayesian models have drawn notable attention in artificial intelligence and machine learning,for their flexible modeling ability and remarkable robustness in learning.Its learning task,i.e.Bayesian inference,is faced with new challenges and requirements in this Big Data era.The variety of data formats poses the demand for Bayesian inference methods to efficiently tackle variables with manifold structures;powerful but complicated models require a high approximation flexibility of inference methods;intense downstream tasks need particle efficiency;and the immense amount of data asks inference methods for efficiency in training time and computation resource.Moreover,designing and developing new inference methods also require a clear knowledge on the fundamental principles of and relations among various existing methods.On the other hand,the mathematical concept of manifold could provide a fundamental perspective and powerful tools for these problems,thanks to its inclusive definition,rich structures and reflection on the intrinsic geometry of a space.In addressing all these challenges and requirements on the efficiency of inference methods from various aspects,we present in this thesis a study on enhancing the efficiency of Bayesian inference methods for both theoretical and practical concerns,by utilizing the explicit or implicit manifold structures of data and models.The study focuses on the two vital fields in Bayesian inference of Markov chain Monte Carlo(MCMC)and particle-based variational inference(ParVI).In this thesis,the main contributions are highlighted in the following.1.We propose stochastic gradient geodesic MCMC methods,so that we substantially improve the time efficiency of manifold-variable-targeted MCMC methods for processing large scale data sets.2.We develop Riemannian Stein variational gradient descent methods,which on one hand enhance the iteration efficiency of existing ParVI methods,and on the other hand introduce the first ParVI method for manifold variable inference tasks,with both approximation flexibility and particle efficiency.3.We make a theoretical analysis on the assumptions that ParVI methods are based on,which also reveals the relation between existing ParVI methods and inspires two new ParVI methods.For all ParVI methods in practice,we propose an acceleration framework and a bandwidth selection method based on the theory,so that the computation efficiency and particle efficiency of ParVI methods are further improved.4.We propose a unified theoretical framework for describing general MCMC methods as flows on the Wasserstein space,which systematically explains the behavior of existing MCMC methods,and bridging general MCMC methods with ParVI methods.Based on the theory,we develop two novel ParVI methods that improve the computation efficiency of ParVI methods,and enhance the particle efficiency of MCMC methods.