Joint testing of sensitivity, specificity, and kappa in diagnostic studies

Diagnostic tests are becoming increasingly important in patient management in the health care setting and/or as screening tools for the benefit of a population. Sensitivity and specificity are common and well-understood measures of the effectiveness of a diagnostic test. When the result of a diagnostic test requires subjective interpretation, the level of agreement among observers is often quantified with the kappa index of agreement. Statistical methodology for joint evaluation of these measures is needed. We propose to estimate and jointly test sensitivity, specificity, kappa in separate samples of diseased and nondiseased subjects using multiple logistic regression models. The information sandwich is used to develop a consistent estimate of the overall covariance matrix for the logistic regression parameters. The delta method then provides the covariance matrix associated with the estimates of the sensitivity, specificity, kappa in the diseased and nondiseased subjects as these measures are functions of the logistic regression model parameters. The logistic regression modeling may be conducted separately for diseased and nondiseased subjects or jointly. Both approaches are described herein. Alternatively, generalized estimating equations (GEE) with the delta method are proposed as a way to jointly estimate sensitivity and specificity (but not kappa) and provide a robust estimate of the covariance matrix. Again, this modeling may be conducted separately for diseased and nondiseased or jointly and both approaches are described herein. The logistic regression modeling and GEE modeling approaches are desirable over and above the simple binomial proportions since they can accommodate multiple observations per subject and adjust for covariates. Muller et al. (1989) performed experiments investigating various levels of certain technical parameters to determine which would lead to an optimal image for bone scintigram photography and we illustrate the proposed statistical methods with this data.