E-rater VS. Human Rater In EFL Writing Assessment

In foreign language writing assessment field, human rater has been predominantly used to evaluate writing samples both in large-scale test (such as College Entrance Exam) and classroom test. However, human rater confronts unavoidable problems such as fatigue and inconsistency in score reliability, English writing researchers and assessors consider using e-rater to replace it on the basis of high correlation findings and successful empirical studies in both first and foreign language writing researches.On the foundation of previous research findings and theories, this thesis summarizes the experiments on the correlation between e-rater and human rater carried out in the English Education Department of Capital Normal University (CNU). It amis at a systematic analysis of the correlation in an attempt to provide a consistent and objective writing analysis tool for EFL writing assessment researchers in China. The thesis is divided into six parts:Chapter One presents a brief introduction to the background and deficiencies of human rater writing assessment means in Chinese EFL writing classroom, which necessitates the appearance of e-rater in foreign language assessment. To confirm the possibility of e-rater's replacement of human rater, the author designed a study in CNU to testify the research results of previous assessors based on correlation studies of e-rater and human rater.Chapter Two provides a review of the existing published literature concerning the development of human rater from 1970s to current situation, details the alteration of writing criteria within the similar period, and summarizes the inferiorities of human rater in foreign language writing assessment, including expensive cost in money, time and labor force; unsatisfactory and doubtful results of inter-rater reliability; fatigue, bias and inconsistency encountering during rating process and effects of rater's scoring experience on score result.Chapter Three compares various definitions of e-rater and confirms the definitionof Educational Testing Service (ETS) as the most accurate one; details assessing content of e-rater in aspect of syntactic structure, rhetorical structure, and topical analysis; discusses e-rater's development in chronological order from 1960s to the current; debates possibilities of e-rater's replacement of human rater in respect of its significant correlation with human rater and its supplement and superiority to human rater; introduces four e-rater software used for current writing assessment - Project Essay Grader, Intellimetric, Intelligent Essay Assessor, and CriterionSM.Chapter Four details descriptions of research study in overview, subjects, method, data collection and procedures of the field study upon which this research is based.Chapter Five focuses on the findings from the field study, presenting qualitative text analysis between e-rater and human rater of 30 stochastic writing samples taking into account of grammar, usage, mechanics, style, and organization & development. Quantitative data analysis results and findings show that e-rater is correlated with human rater.Chapter Six concludes practical significance of the research study; analyzes differences of e-rater and human rater hi feedback provision and error detection of preposition, conjunction, article, wordy phrases or sentences, sentence structure, etc; discusses limitation and further validation of this research study, and positive foreground of e-rater's replacement of human rater in writing assessment field.