| Using Bachman's (2005) validation framework, this study investigates validity issues related to the use of the English Language Development (ELD) Classroom Assessment, a standards-based assessment of English proficiency used in a large urban school district in California to make high-stakes decisions about students. The purpose of this study is to determine the extent to which scores on the ELD Classroom Assessment can be interpreted as indicators of English proficiency as defined by the California ELD standards.; Quantitative methods were used to examine the ELD Classroom Assessment in relation to the California English Language Development Test (CELDT), another measure of the same ability, over three years. A confirmatory factor analysis of multitrait-multimethod data revealed evidence of convergence and discrimination in all three years. That is, the ELD Classroom Assessment measures the construct of English proficiency, and furthermore, it measures three distinct, though highly related, constructs of English proficiency: Listening/Speaking, Reading, and Writing. Substantial method effects were also found. Multilevel analyses pointed to an even greater method effect in the between-teacher model, suggesting inconsistencies among teachers in how they used the ELD Classroom Assessment. Longitudinal analyses, latent growth modeling and factorial invariance, revealed that the ELD Classroom Assessment measures these constructs consistently over time and measures student progress.; Verbal protocol reports of teachers while scoring the ELD Classroom Assessment shed some light on the quantitative findings. They revealed that the scores on the ELD Classroom Assessment reflect more than just students' English proficiency; scores also reflect teachers' interpretations of the standards, teachers' interpretation of the scoring criteria, students' personalities and behavior, pressure to advance students to the next level, and teachers' beliefs about assessment.; The study concludes that despite inconsistencies in teachers' scoring, the ELD Classroom Assessment measures English proficiency as defined by the California ELD Standards, does so consistently over time, and captures student progress. Classroom assessments such as the ELD Classroom Assessment have the potential to be useful assessment instruments both for accountability and to promote student learning. In order to be used more effective, professional development is needed as well as an explicit alignment between the curriculum and the ELD Standards. |
