| In language testing, much attention has recently been directed to the social consequences of test use. Because it is standards that ultimately produce successes and failures in tests in various social contexts, standard-setting, the process by which operational performance standards in criterion-referenced testing or cut-off scores are generated, should play a more important role than ever in the field.; Over the past two decades, the focus of standard-setting has been shifted from the quest for single true cut-off scores to the acceptance of the error in decisions that is inherent in human judgment. What underlies this shift in focus has been the increasing awareness among researchers of the tenet of standard-setting: Standard-setting is a social activity. The diversity of judgments is seen as a reflection of judges' varying expectations about the consequences of the misclassification of examinees, an issue involving cost-benefit tradeoffs between the guarantee of competence and the protection of examinees.; In this study, a new Rasch-based standard-setting method for performance assessments suitable for second/foreign language testing was applied to a certification examination in medical translation. The method was based on the notions that the cognitive legitimacy of the judging task is the basis for generating standards that accurately represent judges' intentions and that standard-setting is a social activity in which judges' differing values play a major role. This study was an attempt to provide convincing evidence to support the plausibility of the generated standards and the defensibility of the accepted misclassification through determining whether these notions can be upheld by the empirical outcome.; The panel consisted of eight judges who had sufficient professional experience in writing, translating, rewriting, or reviewing medical articles. They assessed a total of 15 examinees' Japanese to English translations using the analytic and holistic approaches. The Multifaceted Rasch Model (Linacre, 1994) was used for the estimation of operational standards.; The results showed that the above-mentioned notions were supported by the data, which in itself constitutes the basis for (a) asserting the plausibility of the generated standards and (b) defending the accounts for accepting the error in the decisions. |
