| Testing, especially those large-scale high-stakes tests, is never a neutral process and always brings about consequences. It is believed that a set of relationships, intended and unintended, positive and negative, co-involve and co-interact between testing, teaching and learning (Stobart,2003; Cheng,2008). For the past25years, the impact of language tests on L2teaching and learning, i.e. washback, has drawn much attention of administrative personnel, policy makers, researchers and English language teachers in the field of language testing.Although washback has received much attention in educational circles, yet insufficient research has been conducted on language tests to explore its nature, extent (Bachman&Palmer,1996) or intensity (Cheng,2005) and its direction (beneficial or " damaging (Alderson&Wall,1993; Hughes,1989), especially on large-scale, high-stakes language tests. The current study, adopting the approach of use-oriented language testing, investigates the washback effect of the National Matriculation English Test (henceforth, NMET) in Beijing, aiming to find out the positive and negative impacts of the NMET on English language teaching and learning in senior middle schools in Beijing. To fulfill the above stated objective, this study seeks to answer the following four research questions:(1) What are the measures taken in reforming the NMET and its’ intended outcomes since the inception of autonomous paper-setting of the NMET in Beijing?(2) What are the characteristic washback effects of the NMET in Beijing?(3) What conclusions can be drawn from the study of washback effects of the NMET in Beijing?Introduced in1985, the NMET, originally known as the Matriculation English Test, is one of the three compulsory tests in the college entrance examination battery in the People’s Republic of China. Scores from the NMET are mainly employed in university admission decisions together with those from other subject tests. English, together set with mathematics and Chinese, is one of the three compulsory subjects for all candidates regardless of their choice of the university.As a norm-referenced, standardized proficiency test with a major function of selecting senior middle school graduates for institutions of higher learning, the NMET is a high-stakes English test in P. R. China, exerting significant impacts on school rankings, teacher reputations and students’ future professional careers as well. Apart from its selecting function and the indicator of L2proficiency, the NMET also aims to bring about positive washback to English language teaching and learning in senior middle schools (Qi,2004a; Qi,2004b; Qi,2005; Cheng,2008).In fact, washback, or impact of language testing, is relatively a new notion. Though substantial research has been conducted on washback or impact of language testing in the past25years, the distinction between washback and impact has been in controversy. Wall (1997) defines impact as "any of the effects that a test may have on individuals, policies or practices within the classroom, the school, the educational system or society as a whole," but she also regards washback as a synonym of impact, being frequently used to refer to effects of testing on teaching and learning done in preparation for it (Cheng,2008). Messick (1996) posits washback (or backwash) as "the extent to which the test affects language teachers and learners to do things they would not otherwise necessarily do". Other researchers (Bachman&Palmer,1996; Hamp-Lyons,1997) consider washback as one dimension of impact. Hamp-Lyons (1997) even claims that test influence would fall between the narrow one of washback and the all-encompassing one of impact (Cheng,2008). In this study, I intend to explore the washback effect, or impact of the NMET on English teaching and learning in senior middle schools in Beijing.Specifically speaking, washback studies of testing are mainly conducted in two fields:language education and language testing. Since the late1980s, studies of language testing have been on the increase. Among the early research, three studies, i.e., Alderson and Wall’s (1993) washback hypotheses, Hughes’(1993) basic model of washback, and Green’s (2007) model of washback direction have been considered the main frameworks in conducting studies of language testing. In language education, most researchers discuss and explore washback mainly in terms of its scope, mechanisms and those aspects of the education system affected by language tests. Bachman and Palmer (1996:30, see Figure1), consider that the impact functions at both the macro level, i.e. society and the education system, and the micro level, individuals as well. The individuals involve all the people directly and indirectly affected by tests, such as test-takers, decision makers, and test takers’ future co-workers or employers. Washback, though not shown in Figure1, can be taken to fall within the scope of impact (Bachman&Palmer,1996:30, see Qi,2004a).Hughes (1993), based on Bachman and Palmer’s conceptualization, proposes a washback model of trichotomy, which consists of participants, processes and products. In his trichotomy, participants specifically refer to students, classroom teachers, administrators, materials developers, publishers, etc., while process involves materials development, syllabus design, changes in teaching methodology, the use of learning and/or test-taking strategies, etc.; products are concerned with what is learned and acquired such as facts and skills (Qi,2004a). Hence, a basic model of washback is constructed on the trichotomy as she notes that:"The trichotomy into participants, processes, and products allows us to construct a basic model of backwash. The nature of a test may first affect the perceptions and attitudes of the participants towards their teaching and learning tasks. These perceptions and attitudes in turn may affect what the participants do in carrying out their words (processes), including practicing the kind of items that are to be found in the test, which affect the learning outcomes, the products of that work"(Hughes,1993:2, quoted in Bailey,1996:262)(See Figure2.2).In the current study, Hughes’basic model of washback is employed in a simplified way. In this simplified model, participants refer to test developers (testing experts), teaching experts, English inspectors, secondary school English language teachers and students as well. Perceptions of these participants of the NMET and its washback on the secondary school English teaching and learning are investigated. Processes refer to regular classroom teaching and learning as well as the test-oriented practice of the NMET. Products refer to learning outcomes, and some ancillary products contributing to students’learning, such as new learning materials and curricula, improved teaching, valuable research findings, etc.(Bailey,1996). In this study, a systematic analysis has been made on the observed classroom teaching practice, teaching materials used (including textbooks), and students’ overall classroom performance.Wall and Alderson (1993), by questioning the claims of the existence of washback, provide the most thorough and comprehensive treatment of the term "washback" up to1993(Gu,2007). They point out that there is no one-to-one relationship between tests and have formulated15washback hypotheses to delimit the different aspects of teaching and learning that might be affected as follows:(1) A test will influence teaching;(2) A test will influence learning;(3) A test will influence what teachers teach; and(4) A test will influence how teachers teach; and therefore by extension from (2) above,(5) A test will influence what learners learn; and(6) A test will influence how learners learn;(7) A test will influence the rate and sequence of teaching; and(8) A test will influence the rate and sequence of learning;(9) A test will influence the degree and depth of teaching; and(10) A test will influence the degree and depth of learning;(11) A test will influence attitudes to the content, method, etc. of teaching and learning;(12) Tests that have important consequences will have washback; and conversely,(13) Tests that do not have important consequences will have no washback;(14) Tests will have washback on all learners and teachers;(15) Tests will have washback effects for some learners and some teachers, but not for others.(Alderson and Wall,1993:120)Later, Alderson and Hamp-Lyons (1996:295) further expand the15washback hypotheses by adding one more to them, i.e. the16th one:(16) Tests will have different amounts and types of washback on some teachers and learners than on other teachers and learners.The16hypotheses formulated, according to the different aspects of teaching and learning that might be affected, can be classified into three groups. Group One focuses on teaching, including facets of contents, methods and attitudes. Group Two involves the learning facets, such as attitudes, contents, strategies, the extent and depth. Group Three deals with the strength of washback in terms of the importance of the test to the characteristics of the individuals.As can be seen, the intention of Alderson and Wall (1993) is not to argue for or against any of the above hypotheses, but to delimit the scope so that the questions that research of "washback" need to address can be more clearly seen. As they point out,"the failure of a test to produce beneficial washback may be due not to problems in the test but to other forces, which exists within the society, education and schools"(Alderson and Wall,1993:116). They are aware that the existence and importance of "washback" has been widely asserted, but little empirical research has been carried out to document the mechanisms by which it works.In the current study, the washback hypotheses of Alderson and Wall (1993) are employed to design questions to investigate participants’ perceptions of the NMET and its impacts on English language teaching and learning in senior middle schools in Beijing. The distinctions made in their hypotheses have been adapted as criteria for observing classroom teaching and learning and for identifying influence on different teaching and learning periods, i.e. regular classroom teaching and learning as well as preparations for or class reviews prior to the NMET. Although Alderson and Wall (1993:68) perceived that testing is one of the many factors affecting educational innovations in their research, other factors are also identified, categorized and analyzed in the current study.Green’s model of washback direction, incorporating test stakes, is mainly composed of two components:test characteristics and participants characteristics; on test method and test preparations (Green,2007).According to Green (2007), Washback is perceived to vary along at least two dimensions, one is direction (positive or negative)(Alderson and Wall,1993; Brown&Hudson,2002; Hughes,1989), and the other extent (Bachman and Palmer,1996), strength (Gates,1995) or intensity (Cheng,2005). The design of test, or test characteristics, reflected mainly by test format and content, including test use and test stakes, or the consequences of test scores, is responsible for the direction of washback. In addition, participants’ characteristics involve all the stakeholders of a test, including test designers, administrators, policy makers, teachers, candidates as well as their parents.In the current study, test characteristics are mainly discussed through the interviews of two testing experts/professors, an English inspector, a teaching expert and five English language teachers from secondary schools in Beijing. A brief review of the test format and measures taken in improving test validity and reliability for the past25years has been identified through the interviews of the2testing experts, the inspector and the teaching expert. Participants’ characteristics have been scrutinized through the analysis of class observations of6English language teachers (two for each grade of the secondary high) and the questionnaires surveys conducted.To the researcher’s knowledge, empirical studies on washback effect of language testing mainly concern large-scale, high-stakes tests. According to its category, the studies carried out so far can be divided into two groups:international studies on washback of language testing and domestic studies, including Hong Kong.The study of Wall and Alderson (1993) was the first empirical research published in the field of language testing, inquiring into the nature of washback of A New O-level English Examination, together with a new textbook series on teaching and learning in Sri Lanka (Cheng,2008). The data of the study, including interviews, class observations as well as the baseline and the follow-up study, were collected from49schools throughout the country. Based on the comparison of teaching before and after the introduction of the new test, they concluded that the exam was only one of the factors affecting how innovations succeed or fail (Wall&Alderson,1993).Wall’s (2005) longitudinal study examined one of the widely asserted ways of creating change in an educational system by introducing and redesigning high-stakes examinations. Her study was meant to serve as a lever for change by analyzing the effects of a national EFL examination. By providing guidelines for the consideration of educators who continue to believe in the potential of examinations to affect curriculum change, her study revealed how the intended outcome was altered by factors in the exam itself, as well as the characteristics of the educational setting, the teacher and learners (Cheng,2008).Shohamy (1993), through questionnaire, interview and class observation in collecting data, examined the washback effect of three language tests——A Reading Comprehension Test, An Arabic-as-a-Second-Language Test and An EFL Oral Test. The major findings of her study reveal that all the three tests have impacts on teaching and learning, especially in "driving attention to areas that had not been explicitly taught previously"(Shohamy,1993; citied in Qi,2004a).In her follow-up study, Shohamy et al (1996) re-examined the long-term impacts of the ASL and the EFL, pinpointing the different patterns of washback and the impact of the two tests. They concluded that washback may change over time and that the impact of a test is likely to be complex and varies depending on the nature, purpose, and other characteristics of the test (Shohamy et al.,1996:318).Alderson and Hamp-Lyons (1996), by using class observation and interview as research instruments, looked into the washback effect of TOEFL on preparatory and non-preparatory (normal) classes taught by the same two teachers. They found that "the TOEFL affects what and how teachers teach, but the effect is not the same in degree or in kind from teacher to teacher", and that "the TOEFL alone does not cause washback, but it is the administrators, materials writers and teachers themselves who cause the washback"(Alderson&Hamp-Lyons,1996:295). The findings of the study indicate that washback effect stems from a complex set of factors, including individual characteristics of the teachers and hence, simple forms of washback hypotheses seem too naive.Watanabe (1996), by using class observation and interview as instruments, inquired into the washback effect of the College Entrance Exam in Japan, claiming that "teacher factors, such as educational background, personal beliefs, and teaching experience may outweigh the possible effect of the entrance examinations"(1996:319).As is presented above, though teacher factors have been investigated in the above two washback studies, the participants are only confined to two teachers, making it difficult to generalize their findings to other settings. To delve deeper into the influence of teacher factors on the washback effect, a larger sample of teachers is required to validate research findings.Hayes and Read (2004), by studying the International English Language Testing System (IELTS) on two IELTS courses, examined the washback effect of high-stakes, large-scale English tests. By using interview, class observation, questionnaire, and pre-and post-tests in collecting data, they made a comparison between two preparatory courses, i.e. a one-month IELTS preparatory course (Course A) and an eight-month General English course (Course B). The findings suggested that the two courses, though both are under the influence of the same test and are run in the same context where the communicative approach seems to be the accepted practice, differ substantially, and it is believed that the chief cause resulting in the differences is time and class objectives (Qi,2004a).Compared with the studies carried out overseas, washback studies conducted in China, including Hong Kong, pioneered by Li (1990), were initiated as early as the late1980s. By conducting a questionnaire survey of229teachers, Li reported the achievement of some intended washback effect of the Matriculation English Test in China (cited in Qi,2004a). The findings of the study indicate that the test has achieved some intended effect on teaching and learning, initiating changes in teaching content from traditional formal linguistic knowledge to actual practice of the four skills in English language teaching in senior middle schools.Qi (2004a,2004b,2005), following Li (1990), examined the intended washback effects of the NMET through interviews, class observations and questionnaires from8NMET constructors,6English inspectors,388teachers and986secondary high school English language teachers in attempting to examine the reasons why the NMET failed to bring about the intended washback effects. The findings of her study show that the paramount factor resulting in the failure to achieve the intended washback is that the two major functions of the NMET, i.e. the selecting function and that of the promoting change, are in many ways in conflict with each other, making it a powerful trigger for teaching to the test but an ineffective agent for changing teaching and learning in the way intended by its constructors and the policymakers偶of the NMET (Qi,2005).In addition to looking into facets and effects of the NMET, washback studies in mainland China mainly focus on large-scale, high-stakes tests such as College English Test (henceforth, CET) and Test for English Majors (TEM) in the past.It is worth noting that Jin (2000) investigated the washback effect of the Oral English Test of the CET in a questionnaire survey of358candidates and28examiners and found that the administration of the Oral English Test of the CET generated beneficial effects on college English teaching, promoting college students’ improvement of their oral English competence.In comparison, Han et al.(2004) conducted a washback study of the CET through a questionnaire survey of1194college English language teachers in40universities. The findings of the study indicate that the CET has brought much negative effect on college English teaching and candidates who have passed the CET do not have the expected language competence.Gu et al (2013) carried out another questionnaire study of the washback effect of the CET on strategy use. By collecting data from284non-English majors in a questionnaire survey, they found that college students pay more attention to the test instead of English language learning and its strategy use. Hence, the CET has differing washback effects on students’testing strategy use of cognitive strategy, management strategy and testing skills.Through questionnaire survey, the above three studies have respectively investigated the washback effect of the CET on English language teaching and learning and students’ strategy use. However, the findings of the three studies differ completely.Tang (2005), by using questionnaire and interview as research instruments, attempted to identify the washback effect of the CET among more than1,000non-English-major college students (sophomore508, junior511) on their English language teaching and learning. The results show that the CET has impact on their attitudes and contents of English learning, their ways and pace of English study, as well as the scope and their in-depth English learning. However, the impact brought about differs among college students.Shi (2010) investigated115college students on the washback effect of the listening comprehension test of the CET through questionnaire survey and interview. The findings of the study indicate that listening comprehension test and its increase in weighting have impact on college students’ attitudes, time and choice of listening materials as well as effects of their listening practice. Still, the impact differs among college students.Through questionnaire survey and interview, the above two studies both examined the washback effect of the CET on college students’ English learning and listening practice. Their findings reveal that the washback effect of the CET does exist in their English language learning and listening practice in terms of attitudes, time, scope and in-depth learning, etc., but the impact differs among college students.Gu (2003,2004,2007), by employing questionnaire, interview, class observation and document analysis, examined the washback effect of the CET on college English teaching among4500stakeholders, including administrators, college teachers and students. The findings of the study indicate that the CET has both positive and negative impact on college English teaching and learning. The impact of the CET differs among universities, grades and teachers. Generally speaking, the positive impact of the CET outweighs its negative one.Tang et al.(2004) examined the washback effect of the Oral English Test of the CET among more than300college English learners. They have found that the washback effect of the Oral English Test has more beneficial influence on college English students. The findings of the study also indicate that the washback effects differ among the potential candidates and general English language learners in their daily English learning behavior and attitudes towards college English learning.Gu et al.(2013) made a longitudinal study on the washback effect of the CET on college English teaching through interview, analysis of teacher talk and re-observation on three college English teachers’ class. The findings of the study show that the innovation of the CET has no impact on college English teaching mode, but has brought great changes on teaching plan, teaching content and teaching approaches, etc.To sum up, the preceding three studies, by using multi-instruments and focusing on different aspects of the CET, examined the washback effect of the CET. Their findings indicate that the washback effect of the CET is complex and it has more beneficial impacts on college English teaching than negative ones. Compared with the washback studies conducted on the CET, fewer studies have been carried out on TEM. Zou (2003) analyzed the interactions between the TEM8and the English Teaching Curriculum for English majors by reviewing the design and administration of the test. Xu (2012) examined the washback effect of the TEM8on English teaching among5English teaching experts and724English language teachers. By using questionnaire, interview and class observation, the result of the study reveals that the positive impacts on English language teaching outweigh its negative influence. The factors resulting in the negative influence are of various kinds such as English language teacher’s competence, students’ motivation in English learning as well as the less transparent rating scales and the inadequate report on students’ performance in TEM8.Washback studies conducted in Hong Kong, also started in1990s, shared similarity to those carried out in mainland China. Cheng (1998,2005), by using interview, questionnaire and class observation in collecting data from policy makers, material writers, teachers and students, examined the washback effect of the New Hong Kong Certificate of Education Examination in English (HKCEE). The introduction of the HKCEE was intended to be used as a change agent. The main findings of the study, similar to what has been found by Wall and Alderson’s study, indicate that the test has more impact on teaching content than teaching method, i.e."the change of the examination has informed what teachers teach, but not how"(Cheng,2008).Andrews (1994), using two parallel questionnaires, investigated10test developers and95school teachers to inquire about the relationship between washback and curriculum innovation by studying the washback effect of the Use of English (UE) Oral Test in Hong Kong. The findings of the study show that the test has an effect on curriculum innovation, but the effect is more quantitative, with more time allocated to the aspects and content of teaching tested, little qualitative effect has been found in the teaching and learning process (Qi,2004a).It was also noted that Andrews et al.(2002), by applying a neutral testing instrument and using three matched cohorts as subjects, carried out a study into the UE Oral Test given to students of Secondary7in Hong Kong. They found that the test has some effects on student performance in spoken English, and the washback varies from student to student. The test has led to improved performance in some students, but only superficial learning outcomes in others.To sum up, the brief literature review shows that empirical studies of washback effect over the past25years mainly fall into two types:one is concerned with the traditional, multiple-choice, large-scale standardized tests perceived to have had mainly negative influences on the quality of teaching and learning, the other being those where a specific test or examination has been modified and improved upon to induce a positive effect on teaching and learning (Cheng,2008). However, most of the washback and impact studies have only established an exploratory relationship between testing, teaching and learning within a specific educational context. The strength of these studies is that they have delved into factors affecting the intensity of washback, i.e. test factors (test methods, test contents, skills tested, purposes of the test), prestige factors (stakes and status of the test), personal factors (teachers’ educational background and their beliefs), the micro-and macro-context factors (the school setting and the society as well)(Cheng,2008). However, the washback effect may change over time and may not last indefinitely within a specific system and hence, it is necessary to conduct further empirical studies in substantiating the current research findings and exploring the complex relationships between testing, teaching and learning. The current study, based on the previous studies of the NMET (Li,1990; Qi,2004a,2004b,2005), is to further explore the washback effect of the NMET on English language teaching and learning in senior schools in Beijing.As can be seen, the above brief literature review also indicates the frequently used research instrument(s) in previous studies, namely questionnaire, and/or interview/class observation. However, only a few studies have used multi-methods, i.e. questionnaire, interview, class observation as well as document analysis. It is believed that "the most complete designs for researching washback include both observing classes and asking the participants about their views and experiences (either through interviews or written questionnaires) to determine whether teaching and learning are ’evidently linked to the introduction and use of the test’(Messick,1996,242; in Bailey,1996) and therefore, the multi-method design is highly recommended in future studies.In this study, the washback effects of the NMET are scrutinized by employing a multi-method design-questionnaires, interviews and class observation-in triangulating the data to achieve a more reliable analysis and thus convincing results.The questionnaires of both students and teachers are respectively used to investigate factors contributing to the success or failure of the English test. In addition to the background information of the respective respondents, both the students’ and teachers’ questionnaires are mainly composed of two sections:one focuses on the situation of English language teaching and learning in senior middle schools, the other concerns their attitudes towards, or perceptions of the English language teaching and learning in senior middle schools and the NMET. Before the questionnaires were designed, the researcher of this study had observed8English language teachers’ classroom teaching from5senior middle schools, and interviewed6teachers to have a better understanding on English language teaching and learning in senior middle schools. In the process of the questionnaires design, in addition to studying the previous influential questionnaires of washback studies employed in investigating the washback effects of the NMET (Qi,2004a,2004b,2005), CET (Gu,2007) and TEM8(Xu,2012), some of the items employed by Qi (2004a,2004b,2005) and Gu (2007) were also adopted in the questionnaires of this study. Some of the items of the two questionnaires were revised based on suggestions of3testing experts and1experienced English language teacher. Finally, the researcher made a pilot study among6English language teachers and24students from4senior middle schools, with one item used both in the student and teacher questionnaire being further revised.The interviews are made among three groups of respondents:4experts,11English language teachers and15students. The4experts (2testing experts,1English language inspector and1teaching expert) interviewed have deep insights into the NMET, and3of them have experience in setting papers for the NMET. The interviews made with the two testing experts are mainly concerned with the general reliability and validity of the testing items used, as well as their views on the innovations of the testing structure and testing content made in the NMET for the past decade in Beijing. The interviews with the teaching expert and inspector focus mainly on the change of the teaching practice brought about by the introduction of new testing items since2002, the inception of autonomous paper-setting of the NMET in Beijing. The interviews with the English language teachers are mainly concerned with their feelings and experience in their daily English language teaching as well as their views on the NMET reform. The findings of the interviews with the teachers can both verify two testing experts’ theoretical insights on the one hand, and recheck the perceptions held by the teaching expert and inspector on the other. The student interviews, in three groups (each grade as one group), serve to testify both their feelings of daily English language learning and their perceptions of the NMET on items surveyed in the student questionnaire.Class observation, another important research instrument considered essential to washback study, has also been employed in6teachers’ classes:Listening, and Listening combined with Writing for grade10; two extensive reading classes for Grade11, and preparatory/review classes of Cloze and Writing for grade12. The objective of observing various modules is to find out factors affecting the intensity and sensonality of washback as well as typical features of English teaching and learning in each grade or teaching period of senior middle schools in Beijing.In addition to the three main instruments, document analysis has also been made in collecting data to evaluate English language teaching and learning in the current context. The respondents participating in questionnaire surveys involve281English teachers from38senior middle schools and1325students from7senior middle schools in addition to2testing experts/professors,1English inspector,1teaching expert, and7students from the researcher’s college who have received delayed interview as a compensation for the missed one of grade12following the questionnaire survey of this study.The findings of the study demonstrate that the intended effect of the NMET in Beijing, apart from its primary function of selecting candidates for institutions of higher education, is designed specifically to promote reforms in ELT and in the classroom in particular,... |
PDF Full Text Request