Automated Essay Scoring In The Formative Assessment Of Chinese Efl College Students' Writing

Online and blended learning have become common educational strategy in higher education(Baleni,2015).Assessment has a strong impact on learning.It is at the core of formal higher education(Angus & Watson,2009)and is a crucial element for effective learning(Bransford,2000).At the same time,China's Medium and Long Term education Reform and Development Outline(2010-2020)explicitly stated that Chinese education should reform on assessment system Formative assessment,as an important component of language programs,benefiting students' comprehensive language development and incorporating summative and formative assessment,and formative assessment should dominate in daily teaching.Kibble(2011)mentioned “in an ideal world,students would receive continuous formative assessment that paved the way for current learning to build upon prior learning with tailored teaching and learning that accelerated the achievement of competency”.Advances in assessment technologies are affording teachers and students new ways to effectively assess and track achievement while also improving learning.The ideal world in formative assessment is approaching! Automated Essay Scoring(AES)is one such technology,a web-based tool that integrates practice and assessment in writing.It provides students with instant,computer-generated evaluations of writing.Its real-time and long-term reports of student activity and progress provide teachers and students rich information for guiding classroom instruction and curricular decisions.This dissertation presents an innovative approach to the development and empirical evaluation of Automated Essay Scoring(AES)technology used in writing class for teaching,learning and formative assessment of writing.It adopted AES engine in pigai.org,the most widely used web-based program in China as the major research instrument(Shi,2012).The major purpose of the dissertation was to implement pigai AES as a formative assessment tool complementing Chinese EFL college-level English writing instruction and to investigate the effectiveness and appropriateness of its automated evaluation and feedback.To achieve this goal,this study sought evidence of students' writing in terms of vocabulary,syntax,organization and content outlined in ETS'(1998)evaluation conceptual framework.Besides,it explored as well students' attitude towards formative assessment and AES-based formative assessment(FA).One year(two semesters)of experiment was conducted in Shanghai Jiaotong University.890 writings of highly proficient 89 students,10 progressively written essays of each student,were collected to examine the impact of AES-based FA on students' writing performance.88 participant students finished questionnaire questions and 12 students were chosen to be interviewed 15-20 minutes by the researcher.A mixed methods approach was employed.Quantitative data consisted of Likert-scale survey responses,automated and human scores of all students' writings in the whole year;text analysis of 13 variables of all students' writing in the whole year.Qualitative data contained semi-structured interviews of students.The empirical study shows that students made significant decrease in their general performance.A more significant decrease was found in the second semester.Students presented the same pattern of change in two semesters that their performance improved gradually right after a sharp decrease on the first writing,and finally an obvious decrease occurred in their last writing.In the terms of vocabulary,syntax,organization and content,a general but not significant decrease was found in vocabulary in the whole year;most indicators exhibited decrease in students' syntax;no significant change was found in organization and content.To explain the decline in students' performance,questionnaire survey and interview were conducted.The result showed that majority of students bore positive attitudes towards FA.However,most students had a rather contradictory attitude to AES-based FA.Based on students' feedback to questionnaire questions and interview feedback,it can be concluded that the inadequacy of AES,students' distrust of online FA and learning environment,students' cognitive and affective development contributed most to the decline of their performance.Moreover,students' contradictory attitudes can be explained in four reasons: 1)social constructivism;2)different feedback strategies;3)computer anxiety and 4)the impact of L1 culture and learning environment.Similar conclusions have been arrived at in other studies.The results of this study provide valuable knowledge to the areas of L2 writing,formative assessment,AES.They have important practical,methodological and theoretical implications and are informative for future research as well as for the design and application of new learning technologies.It is suggested that AES be used as a way of scaffolding learning rather than an assessment method,a complementary assessment approach rather than a substitute approach of summative assessment and better be used by low or intermediate English proficient learners.