A construct validation study of the GEPT reading and listening sections: Re-examining the models of L2 reading and listening abilities and their relations to lexico-grammatical knowledge

The purpose of the current study was to investigate the predictive validity of lexico-grammatical knowledge on L2 reading and listening abilities in the context of the General English Proficiency Test (GEPT), a high-stakes criterion-referenced test commonly used in Taiwan for selecting prospective students or screening job applicants. With the approval from the Language Training and Testing Center, one confidential form of the Intermediate GEPT reading and listening sections along with a background information questionnaire was administered to 609 Taiwanese college freshmen.;Multivariate generalizability theory analyses were first conducted to investigate the score and decision dependability of the GEPT. The results revealed that the test items worked consistently in distinguishing the examinees. Specifically, the semantic meaning items in the grammar and vocabulary (GV) tasks appeared to be more dependable than the grammatical form items. The reading items designed to measure the ability to understand the literal meaning of the text functioned better than the items for measuring pragmatic meaning. By contrast, the listening items designed to measure either literal or pragmatic meaning worked almost equally well. The current predetermined cut-score of 67% items correct on the GV and listening comprehension tasks seemed to perform better in making pass/fail decisions than on the reading comprehension tasks.;Afterward, confirmatory factor analyses were performed to examine the factorial structure of the GEPT. Supporting Purpura's (2004) model of language knowledge as a general theoretical foundation for assessment, it was found that the GV tasks appeared to measure grammatical form and semantic meaning; that the reading comprehension tasks were intended to measure the ability to comprehend literal meaning and pragmatic meaning; and that similarly, the listening comprehension tasks measured the understanding of literal and pragmatic meaning.;Lastly, the structural equation modeling approach and discriminant analysis were utilized to explore the relationships among lexico-grammatical knowledge, L2 reading ability, and L2 listening ability. Evidence was provided for lexico-grammatical knowledge as a strong predictor of reading and listening abilities, but lexico-grammatical knowledge seemed to have less predictability for listening than for reading. Moreover, semantic meaning had a much stronger relationship with reading and particularly listening comprehension than grammatical form.