An application of theory-driven evaluation in educational measurement

In the 2004-05 school year, the Canton City School District embarked on a plan to increase student achievement by giving teachers the tools to conduct formative and value-added student assessments in reading and mathematics. These initiatives were the major thrust of continuous improvement plans created at both the district and building levels. The purpose of this study is to report on the findings of the Canton City School District's theory-driven evaluation of its initial implementation of these assessment systems.; Five hypotheses were tested within two cohorts of students in the eleven elementary school buildings identified by the Ohio Department of Education in school year 2004-05 as at risk or in school improvement. This study found that the relationship between predicted achievement scale scores generated by the state's value-added assessment system and observed reading Ohio Achievement Test scale scores in the sample groups of students were positive, significantly strong and linear in the 2004-05 and 2005-06 school years; these predicted scores correctly classified proficient readers more than 75 percent of the time across cohorts. The use of value-added predicted scores as level-1 variables in unconditional and random effects ANCOVA hierarchical linear models were associated with varying levels of ICC values dependent upon model and cohort parameters. The study found no statistically significant difference between reading achievement proficiency levels in the year before and year of the implementation of the state's value-added assessment system.