A Comparative Study On Item Validity Of Multiple Choice Questions & Short Answer Questions

The Multiple Choice Question (MCQ) format is probably the preferable choice that test developers would like to turn to in their construction of English language testing. Its reliable scoring and efficient administration had once made people regard it as the only feasible as well as practical test format. However, it has aroused, with the advancement of language testing theories, more and more disputes among experts in this field, while another test format– Short Answer Questions (SAQ) has got more and more favorable attention along with the reforms of the CET and TOEFL tests.In order to make a comparative study about the item validity of the two formats, this study applies the evidences of validity, through a criterion-related experiment conducted among 60 sophomore students of the School of Politics and Public Management, Hunan University .Among these two testing formats, which is of higher validity and can better assess the ability which is supposed to measure?There are two testing papers in this study. Most parts of these two papers are the same in content and language points of the corresponding items. The only difference is the test formats. Paper 1 is mainly in the form of SAQ, which is the original Regina ESL Testing paper. And Paper 2 is mainly in the form of MCQ, which is revised by the senior teachers and professors of Hunan University. The procedures of the experiment fully follow the'counterbalanced'test design of Bachman (1990). Besides, this study makes full use of the statistics software SPSS13.0 to conduct statistics analysis, reporting on the item difficulty and discrimination index. Internal consistency analysis is reported as Cronbachα.The purpose of doing internal consistency analysis here is to provide supporting evidences to validity study. Paired Samples T Test is also carried out to check the differences. And Pearson Correlations analyses on each time of test and each paper are conducted to provide some validity evidences. Besides, as a validation analysis, this thesis makes full use of the concurrent CET4 results of the test-takers as an external validation criterion, and conducts criterion-related validity study.The study result shows that, compared with the Multiple Choice Questions , Short Answer Questions enjoys higher validity and can better assess the ability which is supposed to measure and can well discriminate students of all levels.