Using keywords and computers to assess student writing

This dissertation explored assessing text using only the occurrence of keywords. The study contained two articles that used computers and identifying keywords as a promising method of assessment of student essays. Traditional reader grading systems, such as the holistic or the two-tier method of scoring essays, are time consuming, costly, and somewhat impractical in large student classes. With a computer-based system that uses keywords to mimic the grading of humans, the study showed that writing could be included in curriculum and assessment regardless of the class size.; In the first article 100 college papers were graded by the keyword computer-based method and then re-scored by Washington State University's Writing Assessment program using a two-tiered holistic type grading system. The resulting keyword computer-grading system correlated with human grading at the 0.676 level (i.e. 45% of the variance was accounted for between the two essay grading systems). Human grading, however, did not correlate well with the following readily available automated indicators of writing quality: number of spelling errors, Flesch-Kincaid reading difficulty rating, essay length, average word length, or the average length of sentences in an essay. The keyword computer-based system showed statistical promise as a method of grading content specific essays in college courses. Further, the keyword computer-based system was less time consuming and costly than the traditional method.; Article two analyzed the utility of keywords used by a computer for scoring student essays in a college teaching and learning class. Richard Haswell (1998) developed a two-tiered system of grading essays for college placement. 127 students were graded using the computer-based keyword essay grading system and re-graded using the two-tier essay grading system. The computer-based keyword system generated inter-rater reliability of 0.611 between the computer-based keyword system and the two-tiered human grading system. A new system of keyword computer-based essay grading was proposed using the keyword computer-based essay grading method.