A comparison model for measuring individual agreement

The interchangeability between raters are often of interest in method comparison studies and in reliability studies. We propose a general comparison model and create a flexible setting such that any subset of raters can be selected as test or reference raters and the individual agreement between them can be assessed.Two comparative agreement indices, the individual difference ratio (IDR) and within difference ratio (WDR), are proposed. IDR is a non-inferiority assessment such that the individual reading from different raters can not be inferior to the replicated readings within the same rater. For an IDR example, when there is one test rater and one reference rater, our approach degenerates to FDA's method for evaluating individual bioequivalence under relative scale. WDR is a classical assessment such that the precision of selected test raters can be better or worse than that of selected reference raters. GEE method is used for estimation and inference. Simulation study is conducted to assess the performance of our approach.