Comparing The Validity Of The Global And Local Questions In English Listening Tests For Junior High Students

Although the validity of multiple-choice question has been doubted,it has been widely used in various English listening tests at home and abroad.Nowadays,the trend of using multiple-choice question as a test method cannot be changed.Therefore,it is necessary to search for ways to improve the validity of multiple-choice question.Multiple-choice question has two question types,including the global and local questions.Based on the literature review,question types affect the validity of listening testing,however,comparatively few studies focus on question types.Besides,inconsistent findings are reported in the existing researches of question types.Some studies demonstrate that question types have a significant effect on listening comprehension while some studies demonstrate that the effect is neutral.In addition,these studies do not explain the results from the perspective of test validity.Based on this,it is necessary to compare the construct validity of the global question and the local question to find out reasons.The current study adopted the Buck's(2001)framework for describing listening constructs to compare the construct validity of the global and the local questions in the English tests for junior high students.Firstly,based on results of theoretically analysis,it was found that the global question measured more listening abilities,which indicated that construct validity of the global question was better than that of the local question.Hence,hypothesis of the present study was put forward: The global question measured more constructs,so the global question was more difficult than the local question and score of the global question was lower than that of the local question.To testify to the hypothesis,the study employed empirical methods to collect data from both quantitative and qualitative aspects.The purpose is to find out which question type has better validity in English tests for junior high students.Study A is a listening test,including global questions and local questions.Considering that ensuring the authenticity of test tasks is the assurance of the validity of listening testing,especially construct validity,this study used question stem preview format,which referred to only allowing test takers to preview question stemsbefore listening.The study was conducted for 100 junior high students.By analyzing their test scores,it was evident from the results that question types had a significant effect on listening comprehension.The score of global questions was lower than that of local questions.Therefore,the hypothesis was verified from the quantitative perspective.For examining whether the validity of question types was mediated with language proficiency,the 100 subjects were divided into high language proficiency group and low language proficiency group in accordance with their language ability.Each group has 50 subjects.The result showed that question types and language proficiency had no interaction effect,so the validity of question types was not mediated with language proficiency.Study B is a think-aloud task,which purpose is to further verify and interpret the results of Study A.By analyzing the verbal reports of 3 subjects in the high language proficiency group and 2 subjects in low language proficiency group,it was found that the quantitative data of Study B was consistent with that of Study A.Besides,compared with Buck's(2001)framework for describing listening constructs,the result of think-aloud task was the same as that of theoretical analysis.Subjects in both language proficiency groups did not need to apply pragmatic knowledge,sociolinguistic knowledge and strategy of summarization when dealing with local questions,so the score of local questions was higher than that of global questions.Subjects in high language proficiency group applied all constructs of Buck's framework when addressing global questions.Thus,construct validity of the global question was better than the local question and the global question can measure listeners' listening ability more accurately.Taking the authenticity of language tests into account,the test method that only allows test takers to preview question stems before listening can be strongly promoted.At the same time,in order to accurately measure the listening ability of listeners and improve construct validity of listening tests,considering that the global question has a better construct validity than the local question,test developers can increase the number of or the proportion of global questions when designing listening tests.