| The purpose of this study is to develop criteria of evaluation for genre-based narrative writing, for possible inclusion in secondary school language and literature programmes. The genre-based approach is founded on a science pioneered and promoted by M. A. K. Halliday (1985a) and called Systemic Linguistics. This last describes analytically how language is structured and employed at the word, phrase, clause and sentence levels in order to fulfil its primary purpose of conveying meaning. A genre-based approach to writing differs from traditional approaches in as much as it recognizes that grammar is principally a functional concept to be used as a means for conveying meaning.; This study materialized as much from the felt need to create a reliable and valid assessment scale for this new approach to writing as from the fact that, in my understanding, genre-based writing derives logically from the traditional and process approaches underlying the Ontario Ministry Guidelines (1977 & 1987) for written English composition and the Ontario Assessment Instrument Pool (OAIP, 1990). These latter approaches have been presented in this study both as a way of highlighting what were seen to be the genre-based principles they implicitly included, and as a means of showing that no sharp break divides genre-based writing from the approaches that have preceded it in my classroom practice.; The pioneering work of Diederich et al (1974) and of Cooper and Odell (1977) in creating ways of improving the reliability, and therefore to some degree the validity, of grades on essays (ibid.) was adopted as a model for the study. A 10-factor assessment scale, based on the essentials of genre-based narrative writing and modelled after the guidelines of Macken et al. (1989c, pp. 84--100), was developed. The Cooper & Odell format was re-arranged to show its factor-to-factor correspondence with the new assessment scale. One hundred written narrative texts were assessed through the application of both scales by 7 teachers who were judged to be conversant with genre-based narrative writing pedagogy, a suitable time-lapse being allowed to intervene between the two assessments so as to minimize reciprocal influence of the assessment results on each other. A reliability co-efficient for the new assessment scale was calculated for all possible paired combinations (21) of the 7 assessors, by using the Tetrachorics method of Diederich (1974, pp. 32ff) and the Spearman-Brown Prophecy Formula. The mean co-efficient of reliability for all 21 pairs was .8461, which figure exceeds the measure of reliability of .80 stipulated by Diederich as adequate for the purpose of assessing school essays.; The validity of the new assessment scale was also a concern of the study. Messick's (1988b) definition of validity as "an integrated evaluative judgement of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores" was used as the guide. This definition, as expounded in Messick's 4-celled matrix, points to the "progressive" long-term nature of the validation process that requires varied sources as evidence of support. Hence the process merely begun here could lead to further research. The empirical evidence supporting the adequacy of the inferences made about the new writing assessment based on the scores obtained, is an initial pointer to the probable value of the criteria to teachers in their classroom work of assessment. |
