Design And Application Of A Computer-assisted Marking System On SAQ

This thesis"Design and application of a computer-assisted marking system on short answer questions"aims to reduce the influence of human raters and save human resources while keeping scoring reliability in marking SAQ.SAQ is defined as the test format of a question or an incomplete statement which expects students (test-takers) to response in one word or a few words. This test format exerts good washback on English teaching like the other subjective tests, whereas its scoring process needs plenty of human resources and the scoring reliability cannot be guaranteed for the involvement of subjectivity. Therefore, its wide adoptability to large-scale English tests is limited. The focus of the study is to design a computer-assisted marking system (CAMS) by making use of two indices"single similar degree"and"valid word ratio". The study is based on theories of three disciplines: applied linguistics, computer technology and mathematics. Such a research method has seldom been employed in present linguistics study. The research result is of great practical value, which not only helps to promote the wide use of SAQ in more English tests, but also facilitates the research on computer-automated marking of English subjective tests.The system is used to mark the same 220 pieces of SAQ test papers in four applications. After each application, the marking results are analyzed and the possible causes leading to the marking errors are explored. Based on that, the system is improved gradually. Therefore, the process of four applications is also the revising process of the system.The major findings of the present study are briefly summarized as follows: First, the two indices"single similar degree"and"valid word ratio"play a very important role in measuring the similarity between the student's response and reference answers. The system can reach 85.40% correctness ratio in marking.Second,"single similar degree"and"valid word ratio"function not quite very well in judging the semantic meaning of student's response;"key word ratio"is an important factor affecting the semantic meaning of student's response. For since"key word ratio"is added to the system, the marking correctness ratio of the system reaches 91.07%, which is 5.67% higher than the former.Third,although the correctness ratio of the system achieves 91.07% which has not surpass that of human raters, the study explores some factors affecting the marking accuracy of the computer-assisted marking system and it is helpful to offer some references for further studies in the future.