The Impact Of Candidate's Oral Performance On Rater's Scoring

Speech is the most manifest of language abilities (Oiler, 1979). Testing speaking ability offers plenty of scope for meeting the criteria for communicative testing ...perhaps more than in any other skill there is the possibility of building into a test a number of the dynamic characteristics of actual communication (Weir, 1988). However, owing to the characteristics of oral activities and the subjectivity of scoring, speaking test faces many challenges and difficulties. It is the least developed and the least practiced in the language testing field (Lado, 1960). In recent years, many testing researchers have focused their work on the scoring variation.There are a lot of factors that influence scores rewarded to candidate and its interpretation. But the most important ones are the following three: the difficulty of the task, severity of the rater and the candidate performance. Up to now, there have been a lot of researches into rater conducted from the raters' perspective, such as their sex, their competence, their questioning techniques, etc, and disregarded the interaction between candidate and rater, especially the impact of candidate performance on rater's scoring. This article mainly deals with the question using p×r×i design of G-theory to investigate the magnitude of the interactions between raters and candidate performance. The data in this study were taken from an experiment. Ten candidates were askedto answer two tasks, which were rated by six raters with analytic method on spot. All the participants were randomly sampled. The data was finally analyzed by SPSS 11.Generalizability Theory (G-theory) is one of the three classical theories in language testing field. G-theory is a useful technique for examining the psychometric quality of the measurement system because it permits the specification and estimation of multiple sources of measurement error.The results indicate that by and large the scores obtained from this speaking test procedure are reliable at separating test-takers in terms of their language ability. We also find that in speaking test the rater has little variation in fluency, pronunciation and flexibility and more variation in grammar, vocabulary and adequacy of content.