Investigating validity across two test forms of the Examination for the Certificate of Proficiency in English (ECPE): A multi-group structural equation modeling approach

The purpose of this study was to investigate the comparability of the underlying trait structure of the two test forms of the grammar/cloze/vocabulary/reading (GCVR) section of the Examination for the Certificate of Proficiency in English (ECPE). There were two parts to the study: The first part of the study separately investigated the hypothesized underlying constructs of lexico-grammatical knowledge and reading ability as measured by the GCVR section in the two different forms, using confirmatory factor analysis. The second part examined the extent to which the underlying constructs were invariant across the two different test forms when modeled simultaneously. A multi-group structural equation modeling technique was employed for this part of the study.;The results of the first part of the study showed that both test forms produced identical factorial models. In other words, the same model fit the data well for each form. Moreover, the parameter estimates were very similar in both forms, which suggested that the two forms were comparable when modeled separately.;When the underlying trait structure of each model was estimated simultaneously, the results showed that the invariance across forms was not supported in the data. In other words, when the equality constraints were imposed on the parameters, the differences in the parameters were found to be statistically significant.;Although the differences in the parameter estimates were statistically significant, the differences in many of the parameters appear to be marginal and the parameter estimates were substantively equivalent. Considering the large sample size of the data used in the study, even a tiny numerical difference in the parameter estimates can result in statistically significant difference. Based on the results of a multi-group SEM, it cannot be concluded that these two forms are strictly equivalent when they are simultaneously modeled. However, it may be reasonable to suggest that the two forms have the identical underlying trait structures despite the marginal differences in the parameter estimates.