Application of the Rasch measurement model to examine the utility of medical diagnostic tests

Medical diagnostic tests are evaluated in terms of reliability and validity (i.e., utility) based on measures of sensitivity, specificity, likelihood ratios, and logistic regression generated odds. While these procedures yield meaningful information, they suffer from problems of biased estimates when the gold standard is biased. In addition, the predictive values estimated from sensitivity and specificity are sample dependent. All of these procedures are limited in the event of missing data. Finally, the interpretations of sensitivity and specificity often lead to confusion as to the correct application of a diagnostic test result. The Rasch Measurement Model (RMM) was examined as a method to provide evidence of diagnostic test utility in order to overcome the limitations of these standard procedures. Using a sample of 825 patient files with a diagnosis of a tear to the anterior cruciate ligament of the knee or no tear to this ligament, four diagnostic tests were examined using the RMM and the standard procedures. These tests included the Lachman's test, the pivot shift, a magnetic resonance image, and a mechanical knee examination. The RMM was compared to standard procedures to estimate sensitivity, specificity, and likelihood ratios with the data set of 825 observations. Observations with no missing data (n = 360) were used to compare the RMM with logistic regression. The RMM provided estimates of diagnostic test utility that were comparable to the standard procedures. In addition, it was demonstrated that the RMM was able to overcome the problem of a biased gold standard and missing data. The RMM provided probability estimates that a person had a tear or was healthy, depending on the outcome of any of the four diagnostic tests. These probability estimates correlated with the estimates of sensitivity, specificity, likelihood ratios, and odds ratios, generated by the standard procedures and logistic regression. The RMM estimates were stable across all testing situations, including using data with a biased (missing) gold standard and in the presence of an ordinal outcome variable. Finally, the RMM provided statistical validity evidence of the function of the diagnostic tests, in the form of a separation index, unique to the RMM.