Test Method Effect On Listening Comprehension Tests Of CET-4

Different test methods may exert their specific influence on the traits to be measured, resulting in different performances on the same language competence that is measured. This is test method effect. In measuring the same language competence, different test methods produce different results, which is related to the study of reliability and validity.According to the skills involved in finishing the task, we divide tests into integrative and discrete-point tests. There has been a debate on the method effect of these two methods. It is argued that integrative tests enjoy higher validity than discrete-point tests, while discrete-point tests are better than integrative tests in terms of reliability.Due to their advantages in efficiency and reliability, discrete-point tests are widely adopted in large-scale language testing. However, the large proportion of discrete-point tests has caused serious negative backwash effect and gets criticized by increasing researchers. With development of communicative language testing, CET-4, a nationwide English test once with 85% of hits questions in MC (Multiple-Choice) method, was reformed in 2006 with more integrative tests.With instruments of two listening comprehension tests taken from final exams, the present research, by analyzing the results of different test methods, examines the difficulty, internal consistency and concurrent validity of the integrative test (compound dictation) and discrete-point test (MC test). The paper also explores the reasons for the test method and backwash effect by conducting the questionnaire survey.The study reveals that respondents performed differently in integrative tests and discrete-point tests. Compound dictation is relatively difficult compared with multiple-choice questions. Reasons include the different constructs, responses and familiarities to different methods. However, dictation enjoys both high internal consistency and concurrent validity, which challenges the statement about the low reliability of integrative tests. From the perspective of backwash, compound dictation brings more positive effects than MC tests. In practice, compound dictation is more likely to enhance learners' overall language competence.It is concluded that the integrative test is a good method in testing listening comprehension and therefore it should be advocated in English test either nationwide or school-wide. It is suggested that the quality of MC items be improved. It is also recommended that the scoring method of compound dictation be adjusted to distinguish those who fail to understand the listening from those who merely fail the word spelling.