Assessment and robust analysis of survey errors

Random measurement error and errors due to complex sampling designs may have deleterious effects on the quality of parameter estimates. This dissertation is comprised of three research papers that provide (1) an assessment of random measurement error through estimation of reliability using longitudinal, latent variable models, (2) an evaluation of the various probability weighting methods as corrections to unequal selection probabilities in multilevel models, and (3) an evaluation of several probability weighting and modeling approaches to unequal inclusion of observations in growth curve models.;A popular structural equation model used to estimate reliability for a single measure observed over time is the quasi-simplex model. The quasi-simplex model (QSM) requires assumptions about the constancy of variance components over time, which may not be valid for a given sample and population. These assumptions are tested using models that extend the QSM by using multiple indicator factors. The extended models include item specific error variance and additional factor variance estimates. Reliability estimates and their standard errors for the models with and without the QSM assumptions are compared in light of model fit and test results for several scales using survey data. Reliability estimates for a general model without the QSM assumptions are generally similar to the models with the assumptions indicating that the particular QSM assumption may not be that critical to the reliability estimates obtained from these models. However, variance components due to additional factor and item specific error have the potential of affecting reliability estimates markedly when they are estimated by the model.;Probability weights have traditionally been designed for single level analysis and not for use in multilevel models. A method for applying probability weights in multilevel models has been developed and has good performance with large sample sizes at each level of the model (Pfeffermann, Skinner, Holmes, Goldstein, and Rasbach, 1998). But, the multilevel weighting method in Pfeffermann, et al. can result in relatively poor estimation due to large amount of variation in the multilevel weights. This chapter includes a simulation analysis to evaluate several alternative methods for analyzing two-level models in the presence of unequal selection probabilities. The primary method of interest is to specify the level two part of the model such that it is robust to unequal selection bias in combination with weighting for unequal selection at level one. This "mixed" method does result in less bias, lower variance, and lower mean squared error for some models. A limitation is that the mixed method requires that the model is correctly specified at level two and the appropriate level one weight is used. This "mixed" method is a new approach in that it combines the Pfeffermann et al. (1998) weighting methodology at level one with the use of sample design variables at level two, rather than use the full Pfeffermann et al. approach of weighting at both levels.;Panel studies often suffer from attrition and intermittent nonresponse. Panel data is also commonly selected using complex sampling techniques that include unequal selection of observations. Unequal inclusion of individuals and of repeated measures will result in biased estimates when the missing mechanism is nonignorable, that is, when missing values are related to outcomes. Probability weighting may be used to correct estimates for nonignorable unequal inclusion due to selection and intermittent nonresponse. However the growth curve models frequently used in analysis of change have not traditionally been estimated using sampling weights. These models are usually estimated using a mixed model where the repeated measures are modeled as a function of both fixed and random parameters. Whereas sampling or probability weights have traditionally been applied to marginal models, which do not include random effects parameters. In this chapter, several weighting approaches are applied to the mixed and marginal modeling frameworks using simulated and empirical data in linear growth models with continuous outcomes. Probability weighting performs the best in a marginal model when missing data are nonignorable. However, in most real situations including the empirical example provided in this chapter, probability weights may need to be combined with estimation that also utilizes variance weighting such as the GLS estimator with a correctly specified repeated measures correlation matrix as the variance weight matrix. This estimation methods can improve efficiency and decrease bias in estimates when data are missing at random (MAR).