Measuring Teacher Effectiveness with the Pennsylvania Value-Added Assessment Syste

The purpose of this research was to determine if the Pennsylvania Value-Added Assessment System Average Growth Index (PVAAS AGI) scores, derived from standardized tests and calculated for Pennsylvania schools, provide a valid and reliable assessment of teacher effectiveness, as these scores are currently used to derive 15% of the annual effectiveness rating assigned to PVAAS eligible mathematics and English language arts teachers. The research also sought to determine if teacher perceptions indicate that the validity and reliability of the PVAAS AGI score included in the Classroom Rating Tool is important. The PVAAS AGI scores, average Normal Curve Equivalent (NCE) scores, and demographic data from 260 Pennsylvania middle schools serving grades six through eight were utilized to determine the extent to which PVAAS AGI scores assigned to a school change statistically over time and if there is a relationship between the PVAAS AGI in mathematics or English language arts for each school and the percentage of economically disadvantaged, learning disabled, English language learners, and minorities attending the school. A Likert scale survey of twenty PVAAS eligible teachers of mathematics and English language arts employed in two middle schools serving grades six through eight in the same district yielded information regarding teacher perceptions. Results of this study indicated that the mean gains for PVAAS AGI scores significantly decreased from 2013 to 2016 for all schools, regardless of achievement level, in both mathematics and English language arts. The demographic variables analyzed did not impact PVAAS AGI scores for schools. Teacher morale was significantly negatively impacted by the inclusion of PVAAS AGI scores in the Classroom Rating Tool, the Pennsylvania mandated rubric utilized to measure teacher effectiveness. Additionally, teachers indicated that they believe important decisions such as changes to instructional practices and scheduling are made based on PVAAS AGI, even though teachers do not understand how it is calculated or trust the validity of PVAAS AGI scores. The literature review surfaced concerns regarding how the average NCE scores are treated in the PVAAS statistical model, as certain treatment of these scores could potentially force invalid results. The literature review also revealed that the Data Recognition Corporation (DRC), the company that designs the standardized tests from which the data utilized in the PVAAS statistical model is derived, has warned that scores at the maximum and the minimum end of the scale may not be accurate due to the design of their test. This is a matter of interest due to the fact that the SAS Institute claims that the tests utilized by their model must be able to measure the performance of students at the lowest and highest ends of the achievement spectrum, begging the question of whether it is even possible for the PVAAS model to accurately determine student growth. Finally, discrepancies in the reported grade levels for schools were found on the state data reporting site.