A Many-facet Rasch Model Analysis Of Rater Effects In CET-SET

Language Performance Assessment has been gaining increasing interest all around the world in recent years due to its concentration on communicative ability and language use behaviours. As a typical language performance assessment, spoken test has relatively high validity, but some factors including candidates proficiency, item difficulty and rater effects are threatening the quality of scoring procedure. It might lead to scoring bias, which will exert negative impact on testing reliability. Among these factors, rater effects are impossible to be neglected. Owing to the diversity and instability, rater effects might affect testing results. Therefore a lot of researches intend to explore the law and nature of rater effects, in order to control them and to reduce the negative influence on reliability. Many-Facet Rasch Model originates from Item Response Theory in Psychometric field. The model extends the original Rasch Model, introduces more facets that affect testing results. Many-Facet Rasch Model can estimate measurement of every facet independently, examine the bias interaction between different facets, and provide systematic and detailed analysis of quality of subjective ratings.This research analyzed the data of CET-SET in Nov.2010. Through Many-Facet Rasch Analysis, the research indicates that though raters'severity shows significant difference, the consistency is good in general; there is no central tendency in these ratings, the qualified ratings can differentiate candidates'proficency efficiently; there is no significant randomness in general; halo effects only occur on some particular raters such as rater No.26 and rater No.12. Some raters seem to be affected by the time progress, which cause rating bias on the accuracy of scores, it can be inferred initially that raters are affected by fatigue, attitude or mood, but the underlying cause needs further investigation.