DIF Detection For Examinees With Different Academic Backgrounds And In Different Economic Regions In A Standard English Listening Subtest

Differential item functioning (DIF) analysis is a statistical technique used forensuring the equity and fairness of educational assessments. The presence of DIFindicates that factors related to group membership may affect the probability of acorrect response, thus fair assessment may compromise. Now abundant literatureshows that DIF screening has become an indispensable part of test development.However, this large scale and high stake listening test, making its debut only yearsago, is still a new test embracing more scientific assessment. This study, therefore, isexpected to contribute in this area.The purpose of the present study is to investigate whether there are DIF itemson the listening subtest for examinees with respect to different academicbackgrounds (humanities vs. sciences) and for examinees from different economicregions (economically-developed region vs. economically-underdeveloped region),and if there are DIF items, the study will attempt to trace the underlying factors. Theinstrument used is the listening subtest of a Standard English test held in2012. Thedata used here is collected from605examinees’ answer records of one of thelistening test. This test is chosen because it has the largest participant number out ofthe equated100tests. To achieve this purpose, two established DIF detectionmethods are used, namely the Mantel-Haenszel (MH) procedure and LogisticRegression (LR). They are chosen because they are computationally simple,inexpensive and easy to implement in practice.The research on examinees’ item performance interacted with their educationalbackground shows totally5DIF items (3with MH procedure, which all favor thefocal group, or, examinees from natural science background and2with LRprocedure), while in the second research on their item performance regarding to theirregional economic difference,4DIF items have been detected with MH, and3ofthem are overlapping with the result of LR procedure. MH result indicates that2 items favor focal group while the other two favor the reference group. Researchresult proves that in this standardized English listening test, examinees’ educationalbackground and their regional economic difference do have some effect on their finalperformance on the test items.The factors for DIF can be quite diversified. However, for the sake of testconfidentiality, there is no access to the test itself, so the exact cause of DIF cannotbe found. But the author has attempted to give some explanation from the itemcontent’s perspective. In the end, some pedagogical implications have been providedwith some illumination for future reference for DIF studies.