| The purpose of this study is to evaluate construct stability of the same certification test taken at different points in a test taker's career, taking into account changes is experience over time. A single medical certification exam administration was used to analyze the construct stability of the certification exam across testing cohorts at varied stages in their medical careers. The Rasch model was used for item analysis to calibrate the difficulty hierarchy of the exam items for each cohort. Correlations between the item difficulty hierarchies for each cohort supported the overall construct stability of the certification exam. Individual item function for each cohort was analyzed through a differential item functioning (DIF) procedure, which showed less than 5% overall DIF, again supporting the construct stability of the examination. The support for the stability of the construct measured by the exam is a necessary condition in the process establishing the validity of the exam, making the information in this study valuable for a variety of testing implications. |
