A Retrospection Study On The Construct Validity Of Multiple-choice Listening Comprehension Tests

This thesis"A retrospection study on the construct validity of multiple-choice (MC) listening comprehension tests"explores the construct validity of MC listening comprehension tests through examining the test-taking processes of test-takers. The construct validity of MC listening comprehension tests is defined as the extent to which the tests measure the right listening constructs or abilities they intend to measure. The research focuses on the following three research questions: 1) what constructs are tested in the MC listening comprehension tests; 2) how the constructs are tested in the MC listening comprehension tests; and 3) what effects the MC format has on test-takers'performance. To answer these questions, retrospective verbal report methodology is employed and eight Chinese EFL test-takers are involved in the study. The major findings of the present study are briefly summarized as follows:First, the grammatical knowledge and discourse knowledge of language competence and all the cognitive and metacognitive strategies proposed in the framework of listening constructs are measured in the MC listening comprehension tests, which supports the construct validity of the tests. But the MC listening comprehension tests may have some defects to measure test-takers'pragmatic and sociolinguistic knowledge, which threatens their construct validity to some extent. Second, the characteristics of test-takers'application of knowledge and various strategies in their test-taking processes are revealed as follows: 1) such characteristics as compensation and using the internal context are found with their application of language knowledge; 2) background knowledge is applied by test-takers to infer the meaning of unknown vocabulary and sometimes to guess the answer of the test item; 3) some strategies, such as inferencing, prediction, contextualization and reconstruction facilitate test-takers'listening comprehension, while the translation and visualization strategies actually slow down the listening process.Third, such three effects as superficial matching, reading and uninformed guessing of the MC format are investigated, which also threatens the construct validity of the MC listening comprehension tests. In addition, the influences of the long dialogue input are also revealed, providing some insights for the application of the input to listening comprehension tests.