Bayesian Inference for High Dimensional Models: Convergence Properties and Computational Issues

This dissertation focuses on Bayesian inference for high-dimensional models, including estimation of the mean in different regression models, and estimation of precision matrices for high dimensional random variables. Along with studying theoretical properties of posterior distributions, we also develop computational methods for efficient and fast model assessment.;In Chapter 2, we consider a fast Bayesian variable selection method for generalized additive partial linear models. The functions in the non-parametric additive part of the model are expanded in a B-spline basis and multivariate Laplace prior put on the coefficients with point mass at zero. The coefficients corresponding to the strictly linear components are assigned a univariate Laplace prior with point mass at zero. The prior times the likelihood is mathematically intractable but we find an approximation by expansion around the posterior mode, which is the group lasso solution in generalized linear model setting for the choice of prior. We thus completely avoid Markov Chain Monte Carlo (MCMC) or any other time consuming sampling based methods, hence leading to quick assessment of various posterior model probabilities. This technique is applied to the high-dimensional situation where the number of parameters may exceed the number of observations. We evaluate the performance of the Bayesian method by conducting simulation studies and real data analyses.;In Chapter 3, we consider Bayesian estimation of a p x p precision matrix, when p can be much larger than the available sample size n. It is well known that consistent estimation in such ultra-high dimensional situations requires regularization such as banding, tapering or thresholding. We consider a banding structure in the model and induce a prior distribution on a banded precision matrix through a Gaussian graphical model, where an edge is present only when two vertices are within a given distance. For a proper choice of the order of graph, we obtain the convergence rate of the posterior distribution and Bayes estimators based on the graphical model in the L infinity-operator norm uniformly over a class of precision matrices, even if the true precision matrix may not have a banded structure. Along the way to the proof, we also compute the convergence rate of the maximum likelihood estimator (MLE) under the same set of conditions, which is of independent interest. The graphical model based MLE and Bayes estimators are automatically positive definite, which is a desirable property not possessed by some other estimators in the literature. We also conduct a simulation study to compare finite sample performance of the Bayes estimators and the MLE based on the graphical model with that obtained by using a Cholesky decomposition of the precision matrix. Finally, we discuss a practical method of choosing the order of the graphical model using the marginal likelihood function.;In Chapter 4, we consider a similar problem of estimating a sparse precision matrix of a multivariate Gaussian distribution, including the case where the dimension p exceeds the sample size n , but now without the assumption of a banding structure in the model. A popular non-Bayesian method of estimating a graphical structure is given by the graphical lasso. In this chapter, we consider a Bayesian approach to the problem. We use priors which put a mixture of a point mass at zero and certain absolutely continuous distribution on off-diagonal elements of the precision matrix. Hence the resulting posterior distribution can be used for graphical structure learning. The posterior convergence rate of the precision matrix is obtained. The posterior distribution of different graphical models is extremely cumbersome to compute. We propose a fast computational method for approximating the posterior probabilities of various graphs using the Laplace approximation method by expanding the posterior density around the posterior mode, which is the graphical lasso by our choice of the prior distribution. We also provide estimates of the accuracy in the approximation.