AN INVESTIGATION OF THE RELATIONSHIPS AMONG DIRECT WRITING SCORES, INDIRECT LANGUAGE ARTS SCORES, AND A COMPUTER TEXT ANALYSIS PROGRAM (WRITER'S WORKBENCH, IOWA TESTS OF BASIC SKILLS)

The objective of this study was to investigate the relationships among the results of a computer text analysis program, Writer's Workbench (WWB), the indirect results obtained from the language subtests of the Iowa Tests of Basic Skills (ITBS), and the scores assigned to writing samples by trained raters using a ten-item analytic rating scale.;First, the reliability of the readers was assessed to determine if they were consistent enough in their judgements, both with themselves and with each other, to use their scores as a stable base for further comparisons. It was also important to examine what constraints might influence the final analysis as a result of the readers' scoring behavior (i.e., the tendency to use only a portion of the scale). The next step was to correlate the indirect ITBS language subtest scores with the direct ones from the writing samples. The final step was to enter 291 of the student essays into the WWB program and correlate the text analysis results with the ITBS subtests results and the direct score results.;Moderate relationships were found among the indirect, direct, and WWB measures. Stepwise regression analyses showed that indirect measures (ITBS punctuation, spelling, and usage) and WWB variables (percent of words flagged for spelling check and average word length) in combination were related to direct writing sample scores (R = .78). These results were verified in a cross-validation study on ninety-six of the student essays reserved for this purpose.;The ITBS subtests scores were obtained through the routine administration of the test to 1,028 ninth grade students in a suburban district in the greater Seattle, Washington area. Those same students were required to write an essay of not less than twenty-five lines in response to a prompt under test conditions. Two trained raters read each paper and assigned scores in ten categories which were then weighted and summed for a total score.