Analysis of paired comparison data using Monte Carlo EM algorithms

Many paired comparison data reported in the literature are obtained in a multiple judgment setting where each judge compares all possible item pairs one at a time. Paired comparison data obtained under such a task not only allow for the identification of systematically inconsistent judges, but also provide a rich source of information about individual differences in the preference judgments. For the analysis of paired comparison data, Thurstonian models provide a flexible framework for the analysis of multiple paired comparison judgments because they allow testing a wide range of hypotheses about the judgments' mean and covariance structures. However, applications have been limited to a large extent by the computational intractability and the parameter identification problems involved in fitting and interpreting this class of models.; When the number of items to be compared gets large, the high-dimensional numerical integrations required for evaluating the response probabilities make the estimation of Thurstonian models computationally intractable. In this thesis, a Monte Carlo Expectation Maximization (MCEM) algorithm is proposed for the maximum likelihood estimation of Thurstonian paired comparison models. MCEM is shown to provide a straightforward solution to the numerical intractabilities that plagued previously the estimation of Thurstonian paired comparison models. A number of simulation studies are conducted to demonstrate the efficacy of the MCEM approach in comparison to the Gauss-Hermite quadrature method. In addition, detailed analyses of two paired comparison datasets are performed to illustrate the usefulness of the MCEM approach for the interpretation of similarity and individual difference effects in preference data.; Representations of Thurstonian models cannot be uniquely identified because of the discrete nature and the difference structure of the paired comparison data. In this thesis, the identifiability of model parameters are investigated by studying empirically equivalent Thurstonian models. Equivalence relations and their implications on interpretation are presented in detail for a number of covariance structures. Wherever possible, specific conditions are given to allow researchers to examine the identifiability of the model parameters.