A Comparison Of Inter-rater Reliability Between Scale-based Rating And Proposition-based Rating In English-Chinese Consecutive Interpreting

Central to interpreting performance assessment is the choice of the instrument(s)and criteria to apply(Sang-Bing Lee,2015).Raters of a performance test vary and usually interpreting accreditation examinations are of high stakes,thus it is important to ensure the inter-rater reliability of the raters involved.From the perspective of the assessment agency,the use of effective and credible assessment tools can help ensure the objectivity of the assessment results,which is of great significance to the standardization of interpreting performance test and the selection of interpreting talents.This paper explores the difference of the inter-rater reliability when using two kinds of ways to assess the information accuracy of the test-takers' interpretation and the possible reasons behind the differences observed.In the experiment,the author selected samples of different levels from the English Interpreting Certificate test of Xiamen University and 20 raters were invited to score the interpretation samples in terms of information accuracy by two rating methods—scale-based rating and proposition-based rating respectively.Two groups of raters—experienced professional interpreters and novice interpreters(third-year graduate students majoring in English interpreting)—rated six consecutive interpreting performances with two rating methods and provided their feedback.The SPSS tool was used to analyze the inter-rater reliability of the whole group and the two sub-groups.The results showed that for the whole group,the inter-rater reliability of proposition-based rating was a little higher than that of scale-based rating.For the experienced professional raters,there is little difference in raters' consistency of scoring between the two methods.But for student raters,it is obvious that proposition-based rating could better help them maintain the scoring consistency when dealing with samples of different levels—the inter-rater reliability of proposition-based rating was much higher than that of scale-based rating.Based on the analysis of the experiment results and the raters' feedback,the author found that both proposition-based rating and scale-based rating had their own advantages and may apply to raters of different backgrounds.Choosing the proper assessing method according to the purpose,raters,and nature of the interpreting performance test would help ensure the validity and reliability of the assessment.