Estimating Error Of Writing Scores In A Generalizability Theory Study

Essay assessment as a subjective test is considered to be a valid form of writing assessment. However, it seems that the complexity and multi-facetedness of this type of assessment introduce sources of error. Currently, generalizability theory is ideal approach for estimating and analysing sources of error in the subjective test. In this thesis, besides the writer's writing proficiency, the effect of task (e.g., genre of essay) ,the features or traits scored (e.g., content and organization or language use) and the way in which these traits are scored(e.g., holistically or analytically) may contribute to the writer's score. First, we look back the reference concerned with these sources of error in the essay assessment .Secondly, the generalizability theory is introduced on the basis of comparison between it and classical true score theory in their fundamental conception. Then those effects of sources of error in the essay assessment and their complex relations are estimated under the generalizability theory model. Analyses of these ratings showed that the effects of tasks and raters are very much dependent on the way the essays are scored and the trait that is scored. The differences between scoring procedures, raters, text genre will affect the generalizability of the scores in this study.