Statistical Inference For Testlet Effect Based On Bifactor Structure

In educational and psychological measurement,a testlet-based test is a common format,especially in some large-scale assessments.A testlet is a cluster of items grouped together because they share the same stimuli,for example,a passage-based reading assessment.For such data,biased estimation of item parameters and inaccurate evaluation of examinees'abilities could arise when applying the standard measurement models(Sireci,1992).This is because the local independence among items within a testlet may be violated due to presence of the local testlet effect.In testlet response the-ory(TRT),Gibbons and Hedeker(1992)proposed a standard bifactor model to account for the testlet effect.The main idea of their approach is to include a common(main)effect measured by the whole test and the additional testlet effect within each testlet.Therefore,the contribution of items can be separately assessed in common effect and testlet effects and,as a result,a more reasonable evaluation of examinees'ability can be provided.The standard bifactor model assumes one common effect(factor)and a separate testlet effect(factor)for each testlet.Different testlet effects and the common effect are assumed to be independently distributed.Jennrich and Bentler(2012)showed that the correlation among the testlets could exist and thus needs to be incorporated while Reise(2012)argued that the independent factors have the advantage of good interpretation.For cognitive diagnostic models(CDMs),Hansen(2013)proposed an extension by adding to the latent attribute profiles the addition continuous testlet effects.Similar to the standard bifactor models,the proposed testlet CDMs assume that the latent attribute profiles are independent of the testlet effects.Consequently,to gain more accurate statistical inference of testlet effects,we proposed methods and models based on the bifactor structure under TRT and CDMs,respectively.In TRT,for data-driven learning of multidimensional testlet effects,we proposed a latent variable selection method based on the bifactor structure and developed a re-lated theory.Specifically,an L₁penalty was employed and the consistency in selecting the true structure was proven.Additionally,model identifiability was established un-der certain minimal conditions for the multidimensional two-parameter normal ogive testlet model.An empirical study was conducted on a data set from 2015 Program for International Student Assessment(PISA).Simulation studies are also conducted to assess the performance of the proposed models and to confirm the theoretical results.Central to capturing testlet effects in CDMs is how to incorporate the continu-ous testlet effects into the main(discrete)latent profiles.In this connection,testlet CDMs allowing for possibly dependency among different the CDM and the testlets is proposed and su cient conditions for the identifiability are discussed.In particular,a“conditionally independent”testlet CDMs is proposed that is flexible enough for allow-ing certain amount of dependence while also maintaining a relatively simple structure.Simulation results are reported,comparing performances of the existing models and the proposed ones.Results showed that ignoring either correlations or testlet effects could lead to biased estimates and increased misclassification of examinees.Finally,the proposed models were applied to a data set from 2015 PISA.