Independent and integrated academic writing tasks: A study in generalizability and test method

This study investigates the generalizability of reading-to-write and independent writing tasks. More specifically, the current study addresses the effect of both tasks and raters on score variability, and investigates the relative impact of these two factors. For this purpose, 115 Egyptian university students were randomly selected and assigned to one group. These participants were instructed to write on two independent writing tasks and two reading-to-write tasks. After data collection, three raters who have L2 writing experience were selected to rate these writing samples. The study followed a P• x I∘ x R• design, which is a multivariate generalizability analysis where persons and raters are crossed and tasks are considered as a fixed facet. To analyze the data, the multivariate generalizability analysis program mGENOVA (Brennan, 1999) was employed.; Results of the study showed that the reading-to-write tasks yielded scores that were as reliable as the scores derived from the independent writing tasks. In addition, the multivariate analysis suggested that a composite score of both the independent and integrated tasks is as reliable as the univariate scores of either the integrated or the independent writing tasks. Furthermore, the analysis indicated that having different raters score each task type would produce as reliable scores as having the same raters score both task types. Additionally, the disattenuated correlation between the reading-to-write and the independent writing tasks was a perfect one. A final result indicated that score generalizability is very low when using one task due to the large (pt) variance component. Implications and limitations of the study as well as suggestions for further research are provided.