Statistical inferences for correlated observations: Prediction and estimation

This dissertation has three major parts. Chapter 2 compares Bayesian predictive densities based on different priors and frequentist plug-in type predictive densities when the predicted variables are dependent on the observations. The performance of different inference procedures is measured, by averaging Kullback-Leibler divergence with respect to the true predictive density. The notion of second-order KL dominance is introduced, and an explicit condition is given for a prior to be second-order KL REML-dominant using asymptotic expansions. As an example, it is shown theoretically that for mixed effects models, the Bayesian predictive density with any prior from a particular improper prior family dominates the performance of REML plug-in density, while the Jeffreys prior is not always superior to the REML approach. Simulation studies are included which show good agreement with the asymptotic results for moderate sample size. Chapter 3 considers the asymptotic comparison result for both temporal and spatial AR(1) models, as an important special case of correlated data, using some theoretical results from the previous chapter. We show that all the three candidate priors, the Jeffreys prior, the reference prior and the inverse reference prior, are dominating the performance of the REML estimation for variance parameters, in the sense of the expected KL divergence between the true density and the predictive densities. Simulation results are included, which almost agree with the asymptotic result when sample size is moderately large. Chapter 4 considers estimation and prediction problems in modeling with errors in covariances. Many popular topics and data-driven methods on the shrinkage estimation for covariance matrices are discussed. We also consider a model with semi-parametric covariance matrix, which includes both a parametric and an unstructured part. For this model, we derive a plug-in method for parameter estimation, and also consider how the mean and variance of a kriging predictor are affected if the true matrix V is replaced by an approximation Vˆ. We consider a preliminary estimator for the unstructured error, a linear combination of the sample covariance and the diagonal estimator, and we find that in the case of exponential covariance structure (for both simulation and asymptotic results), this estimator performs better than the sample covariance matrix by comparing the mean square errors of their resulting regression coefficient estimators. In the future, we plan to derive some theoretical proofs and more simulation studies.;Keywords: Mixed effect models; Kullback-Leibler divergence; Jeffreys prior; Predictive density; Prediction fit; Autoregressive Models; Time Series; Spatial Models; Shrinkage Estimation; Covariance Matrix; Asymptotic Approximation.