Bayesian Model Averaging and multivariate conditional independence structures

We develop a framework for the modeling of high-dimensional data that is robust to a variety of data types and modeling paradigms. Central to our considerations are the issues of structural learning and posterior parameter estimation. In particular, we focus on several classes of models that each employ conditional independence assumptions to derive estimators. We pay particular attention to Gaussian graphical models (GGMs) and Instrumental Variable (IV) models. We begin by considering the problem of model search in the space of GGMs. By specifying a suitable prior distribution, we are able to separate the problems of determining model probabilities from posterior parameter estimation, which proves computationally appealing. In this context, fast stochastic search algorithms are critical in finding the highest probability models and we propose a new algorithm, the Mode Oriented Stochastic Search (MOSS) that employs concepts related to Bayes factors and Occam's Window sets. We then consider the problem of describing multivariate interactions between data of mixed type. By developing a semiparametric Gaussian copula model, we are able to extend the GGM framework non-trivially and propose a class of Copula GGMs (CGGMs) that introduces significant flexibility into the specification of conditional independence restrictions between multivariate data. We conclude by considering the problem of model averaging in IV models, which rely on conditional independence assumptions between subsets of variables to form estimators possessing desirable properties for causal inference. We extend the Bayesian Model Averaging (BMA) approach for regression variable selection and develop an Instrumental Variable BMA (IVBMA) approach that incorporates both instrument and variable uncertainty into the IV framework. We show that IVBMA estimators possess a number of desirable features over standard IV estimators. Furthermore, we propose a class of tests of model assumptions based on the model averaging of predictive p-values which have dramatically improved power over classical tests of IV assumptions proposed in the econometrics literature.