Examining Sequence of Contextualized Items in Science-Experimental Evidence on English Learners (ELs) and Non-ELs

Student performance on science tests is often substantially influenced by the context of individual test items. This study systematically describes the sequence patterns used in science test-item contexts and presents empirical evidence that demonstrate the ways in which the sequence patterns of contextual information support or hinder student performance on science tests. Contextual information includes supplemental information, such as a vignette or selected background facts that precede or follow a test item. This study addressed three gaps in the literature on contextualized items by providing the following in order to advance knowledge about how to achieve consistently equitable testing: a consistent theory-based framework for selecting test-item characteristics to be studied; new knowledge about how student performance is influenced by specific test item elements or features; and a better understanding of subgroups of students and their performance.;The design and data analyses were framed by three research questions: (1) How is providing the sequence of information presented in item contexts associated with student performance? (2) Considering different levels of linguistic demands in various contexts, how is the contextual sequence linked to science performance of ELLs and non-ELLs? ESLs and non-ELLs? (3) How do different dimensions of sequence of context influence how students perceive and respond to tasks? This study included multiple facets of inquiry, including test-item creation, psychometrics, and student cognitive interviews. It generated item-development guidelines based on dimensions of context sequence. Items resulting from this process were designed to evoke students' stored knowledge relevant to the content and/or process skills being assessed. The newly developed science items were field tested with a diverse population of students from middle schools and high schools in China and the U.S. A range of psychometric and statistical procedures were applied to student test scores.;In particular, this project produced both theory and empirical evidence regarding how the sequence of contextual information can be attributed to differential performance among English as Second Language Learners (ESL), English language learners (ELL) and non-ELL groups. Findings from the ESL, ELL, and non-ELL comparisons led to important improvements in how items are developed to ensure test fairness.