| In 1995, Raju, van der Linden, and Fleer introduced the Differential Functioning of Items and Tests (DFIT) framework. However, some concerns have been raised regarding the accuracy of DFIT (e.g., Meade & Lautenschlager, 2004, 2005). More recently, it was suggested that large differences in sample sizes might affect the sampling variance of the NCDIF statistic (e.g., Raju et al., 2009). The purpose of this study was to confirm if differing subgroup sample sizes affect the accuracy of the NCDIF statistic and to propose and evaluate a modification to solve this problem. Monte Carlo results indicated that the old method generally maintained fairly stable power, but tended to be overly conservative when the focal group was smaller than the reference group and exhibit inflated Type I error when the focal group was larger than the reference group. The new method generally maintained reasonable Type I error regardless of subgroup sample size and demonstrated comparable or better power except for conditions where the old method exhibited inflated Type I error rates. When impact was present Type I error rates were slightly higher and power was slightly lower but results otherwise conformed to the general pattern. |
