An Empirical Study Of The Rating Scales In The Oral Retelling Task

This empirical study aims at exploring how raters of oral retelling tasks apply a holistic rating scale to make their scoring decisions, and what problems they encounter when using the criteria and the solutions they use. Four trained and experienced raters took part in the study. They are all female university teachers, whose ages are between30and40, and have been teaching and rating EFL speaking for at least3years. Two of them have a Ph.D. degree. They awarded scores for two sets of20students’speeches. The first set was scored as in an operational rating session. Raters then scored another set using immediate retrospective verbalization to describe the rating process. After this, participants were interviewed about their rating strategies and their perceptions of speaking proficiency. A coding scheme was developed to allow the analysis of the raters’thinking process, their interpretations and applications of the rating scale, and the difficulties they faced in rating.Findings show that (1) raters are all concerned with the content category most, but give different weight to the other two categories of the rating scale;(2) raters also referred to other features not appearing in the scale and had some struggle when the descriptors of the rating scale are too general or the students’performances are at the verge of the descriptors of two grades. And raters would develop various strategies to help them cope with those problems, like comparing other students’performances or referring to previous rating and teaching experience. The results have some implications for the development of rating scales for oral retelling tasks and rater training.