Item Discrimination and Type I Error Rates in DIF Detection Using the Mantel-Haenszel and Logistic Regression Procedures

The inflation of Type I error rates can have damaging effects in DIF identification. This study primarily aimed to examine the performance of the Type I error rates in DIF analysis when using the Mantel-Haenszel (MH) and logistic regression (LR) procedures by simulating data based on two-parameter logistic (2PL) and three-parameter logistic (3PL) item response theory (IRT) models. Specifically, the focus of this study was to explore how item discrimination parameters affect the Type I error rates in both MH and LR procedures when other influencing factors such as, sample size, group mean difference, and matching method were manipulated. Several Monte Carlo simulation studies were conducted. The patterns of the false rejection rates under various conditions were displayed and the effects of influencing factors were evaluated.;The findings suggested that under thin matching, a small range of discrimination parameters for all items resulted in very little Type I error rate inflation for both MH and LR procedures, even with large sample sizes and large group mean differences. The results also indicated that when all items have relatively high discrimination parameters, there is less Type I error inflation regardless of the range of discrimination parameters for all items when using thin matching and deciles thick matching. Additionally, for the condition where the non-studied items did not include weak items, the false rejection rates were controlled fairly well when the studied item had a relatively larger discrimination value. When data were generated with a 3PL IRT model, the results confirmed that guessing was a nuisance determinant on the inflation of Type I error rates. This study also concluded that thin matching was preferable in controlling Type I error rates, deciles thick matching was acceptable in most circumstances, and quintiles thick matching was poor.