A Content Validity Study Of TEM4Writing Test (2005-2011)

Test for English Majors Band4(TEM4) is a criterion-referenced proficiency testfor English major undergraduates in colleges and universities in China, administeredby Instructional Committee of Higher Education Foreign Language Major of Ministryof Education since1991. The objective of this test is to check the sophomore ofEnglish majors when they have finished all the TEM4courses in order to see if theymeet the requirements of teaching syllabus. It not only measures their comprehensiveability of using English, but also the ability of using basic skills in every aspect;meanwhile, it is to evaluate English language teaching and to bring about beneficialbackwash effect on English teaching and learning.Content validity is an essential issue of validity research in language test. It iscontent validity that determines the extent to which the language ability is tested bythe test method. Writing test, as one of the important parts in TEM4, the contentvalidity of writing has great influence on content validity of the whole test. Zou Shen(1999) introduced the scoring criteria of TEM8writing test part, Xiu Xudong (2010)discussed the “Validating English Language Academic Writing Tests at Tertiary Levelin China”, and Zhang Keke (2010) conducted a content validity study on TEM8writing. Since then, few longitudinal content validity study of writing in TEM isconducted; especially no study is conducted on content validity of TEM4writing.Therefore, this thesis will focus on the content validity study on TEM4writing afterinnovation from2005to2011in the two aspects: whether TEM4writing correspondsto the requirements of teaching syllabus (2000version) and testing syllabus(2004version), or to what extent does the writing part of TEM4meet the requirementsin the teaching and testing syllabuses; whether the content is representative or relevantto the test goal and suitable for the test takers.Based on Bachman&Palmer’s framework of task characteristics, combining thecharacteristics of TEM4writing and with the help of TEM4writing requirements ofteaching and testing syllabuses, the author proposes a revised framework of taskcharacteristics for writing assessment with reference to the actuality of TEM4inChina. The modified framework includes the test rubrics, the input, the expectedresponse, and the relationship between input and expected response. This thesis has examined seven TEM4papers, including seven composition writing and sevennote-writing tasks in terms of the four characteristics in the modified framework; andalso administered a simulated TEM4writing test among60sophomore of Englishmajors, and then conducted a questionnaire and interviews concerning four aspects ofcontent validity, namely test rubrics, input, expected response, and relationshipbetween input and expected response. Meanwhile, the collected data were analyzedby SPSS, and the writing results were rated.The research results shows that the content validity of TEM4writing from2005to2011is high and the writing tasks in TEM4highly meet the requirements ofteaching and testing syllabuses mainly reflected in the characteristics of TEM4writing tasks, such as the genre, topic, length and writing speed, etc; the difficulty oftest tasks is appropriate for the test takers.In addition, some other findings are found:(1) Through the whole analysis of the result and scores of the composition andnote-writing, it showed that the students’ performance was favorable in currentsimulated examination.(2) Through the frequency analysis of data from questionnaire and interview, thecontent validity of the writing part of TEM4is acceptable on the whole. However,some efforts should be made to improve the aspects of the testing rubrics includingthe score allotment and criteria of correctness, the input including the contextualinformation and subject matter (topic), and the expected response including the genreof writing types. Many more than half students were satisfied with the test format of“situation/opinion+title”.At last, some suggestions are made for TEM4writing design, English teachingand testing for English majors.