Comparison And Application Of The Robustness Of The Methods For Testing The Functional Differences Of Cognitive Diagnosis Models

In recent years,Cognitive Diagnosis Assessment(CDA)has become one of the research hotspots in the field of Cognitive psychology and psychological measurement(de la Torre,2009;Wang,Bian,&Guo,2015).Cognitive Diagnosis Models(CDMs)are the discrete latent variable models,which provide more fine-grained multidimensional diagnostic feedback about the ability levels of different subjects' understanding and mastery on a set of attributes.Differential item functioning(DIF)is an important method to evaluate the fairness of test in CDMs.In many analyses of cognitive diagnostic test data,Q-matrix is assumed to be correct,and its suitability and correctness will not be verified.However,some studies have shown that the accuracy of the Q-matrix has a great impact on the accuracy of CDA.Inaccurate specified of the Q-matrix elements in CDA will lead to deviations in the CDMs parameters,which will affect the diagnosis of subjects' diagnostic feedback(Chen,Culpepper,Chen,&Douglas,2018;Wang,Ding,&Luo,2019).And then the complexity of the Q-matrix and the structure of the items will produce different results for different methods(Svetina et al.,2018).According to the findings and deficiencies of previous studies,the purpose of this study is to explore the performance of six DIF detection methods under the specific types of Q-matrix and related factors affecting DIF.This study is divided into three parts.The first study investigates the performance among six DIF detection methods(denoted by MH,LR,CSIBTEST,WObs,WSw and WXPD,respectively)under the CDM framework when the Q-matrix is correctly.In most cases,the results show that WObs,WSw and WXPD are better than MH,LR and CSIBTEST.In addition,the test length will also infuence the performance of the WObs,WSw and WXPD.As the test length increases,the Type I error rates of the six detection methods decreases.The second study compares the performance of MH,LR,CSIBTEST,WObs,WSw and WXPD when the Q-matrix is misspecificed under the CDM framework.In this study,we manipulated two types of the Q-matrix misspecification.The results show that WObs,WSw,and WXPD still have better performance than MH,LR and CSIBTEST,when Q-matrix is misspecificed.And MH,LR,CSIBTEST have stable performance,and also have the inflated Type ? error rates and low power.The third Study is an empirical research.And using MH,LR,CSIBTEST,WObs,WSw,and WXPD to detect the existence of gender DIF in TIMSS 2007 Austrian fourth grade mathematics test booklet 3 and 4.The results show that TIMSS 2007 mathematics test booklet 3 and 4 about Austrian students might be gender DIF.But different DIF detection methods have different results.