| For this dissertation, four item purification procedures were implemented onto the generalized linear mixed model for differential item functioning (DIF) analysis, and the performance of these item purification procedures was investigated through a series of simulations. Among the four procedures, forward and generalized linear mixed model (GLMM) iterative purification procedures attempt to remove the contamination of matching variables due to the inclusion of DIF items. The rank-based strategy and mean- DIF procedure are designed to locate DIF-free item(s) as anchor(s), and then the located DIF-free item(s) can be used to identify DIF items. Four variables were manipulated as simulation factors in this study: (1) sample size [500 examinees in each of the reference and focal groups (500R/500F), and 1,000 examinees in each group (1,000R/1,000F)]; (2) test length (20 and 50 items); (3) percentage of DIF items in the test (0%, 20% and 40%); and (4) DIF direction of DIF items (one-sided DIF, dominant DIF and balanced DIF). The type I error rate and power were calculated to evaluate the performance of the forward and GLMM iterative purification procedures. On the other hand, rank-based strategy and mean-DIF procedure were evaluated based on accuracy of DIF-free item identification. All four procedures were applied to simulated data and a real data set. The simulation results showed that the forward and iterative purification procedures were able to control type I error rates, and they were able to maintain a satisfactory power level when 20% of the test items were DIF items. When 40% of the items contained DIF, both procedures had good control over type I error rates and maintained adequate power under the dominant and balanced DIF conditions; however, both procedures lost control of type I error in one-sided DIF conditions. When larger amounts of DIF contaminations were in the tests, the iterative procedure performed better than the forward procedure by generating less error rates. Overall, the rank-based strategy and mean-DIF procedure were both promising for locating a set of up to four DIF-free items. By using a GLMM approach, this study compared the effectiveness of the four item purification approaches for the purpose of creating fairer tests. The comparisons provided practical knowledge that will benefit measurement professionals and enhance the psychometric literature. |
