| A call has been issued within the measurement community to integrate cognitive psychology with assessment to inform test design and validation and to provide detailed diagnostic feedback (National Research Council, 2001). One approach to achieving this integration is to model item statistics, in particular item difficulty, in terms of the cognitive processes underlying item solving (Huff, 2003). To date, only a few models have been developed linking item statistics to the cognitive structure of test items or tasks, and these models are limited by the concepts and methods used. The purpose of this study was to use a cognitive-psychometric approach to model the reading items included in the MELAB, a large-scale high-stakes assessment of English as a second language.; The model was developed and tested through four stages. First, based on the review of the processes associated with reading and reading test taking by ESL learners, an initial cognitive model was hypothesized to underlie the MELAB reading test item performance. Then, this model was validated using a three-stage procedure: analyzing cognitive demands of the test items by raters, collecting students' verbal reports of the processes they used to arrive at the correct responses, and examining the relationship between the proposed cognitive processes and item difficulty estimates using the tree-based regression.; A review of the literature suggested that seven reading processes, four test-management processes, and three testwise processes should be included in the initial cognitive model underlying the ESL reading test item performance. Triangulation of the three sources of evidence collected in the next three stages supported the conclusion that the seven reading processes and three test-management processes underlie successful performance on the MELAB reading test items.; This study demonstrated the value of using multiple sources of evidence to evaluate the performance of a cognitive model, and successfully demonstrated the union of cognitive psychology and assessment in the field of second/foreign language testing. While substantial evidence has been found in support of the cognitive model containing 10 processing components, the model warrants further research so that large-scale language testing programs can yield more meaningful results regarding examinees' reading abilities. |
