A Study Of Rater Reliability In Achievement Test

Rater Reliability is part of Reliability; it reflects the consistency of raters and plays a key role in the process of judgment. Intra-rater reliability and inter-rater reliability are two subbranches of rater reliability. They describe rater reliability from different perspectives. Reliability is a necessary but not sufficient condition for validity. Therefore rater reliability contributes a lot to validity. Tests with validity could give a positive backwash to education. The study of rater reliability is of high value in teaching and learning. Scorer reliability or rater reliability influences the reliability on the whole. Rater reliability in achievement test could not only ensure validity of a test, but also has a bearing on teaching and learning directly. If rater reliability is high, it could reflect teaching circumstances properly, evaluate students fairly, help master teaching and learning pace perfectly, and provide a beneficial backwash to teaching.This paper is developed around my experimental research on rater reliability in achievement test in Jilin University, with an aim to evaluate raters with mark-remark method. Data was collected from nine teachers at Jilin University who attended grading the test paper answered by non-English major students of 2007.The study is implemented based on the hypothesis that rater reliability might be low enough in achievement test. After remarking the chosen test items, data were collected in order to evaluate the rater reliability. The data is evaluated from the following two perspectives: intra-rater and inter-rater. Correlation coefficient and Cronbach's alpha were employed in order to measure the reliability. The higher the correlation coefficient and Cronbach's alpha index, the higher the rater reliability.The research results demonstrate that the rater reliability is not satisfying. Most of the coefficients are less than 0.50.There were courses behind, but we really need more detailed rating scales to regulate rater's behaviors in achievement test.