| Over the years, teachers, administrators, and policy makers have been concerned with optimizing learning for all students. The No Child Left Behind Act put an emphasis on summative assessments, which measure what students have learned. In contrast, formative assessment has been shown in many studies to improve student achievement and motivation because it is applied while students are learning. The purpose of this study was to investigate, for middle and high school mathematics and science instruction, the validity and reliability of a newly developed observation instrument called AssessToday, which is used in a single class period to assess a teacher's use of short-cycle formative assessment.;The content validity of the instrument was supported through an extensive literature review, feedback from experts in the field of formative assessment, and an examination of 98 classroom observations. For assessing reliability of the instrument, inter-rater reliability coefficients were calculated using data collected by trained observers who independently rated teachers during the same class period using three measures: percentage of agreement between raters, Cohen's kappa, and Fleiss kappa. Cohen's kappa (N = 36 pairs) ranged from .62 to 1.00 for all observer pairs with an average kappa of .75 for mathematics (n = 16 pairs) and .76 for science (n = 20 pairs). The recommended threshold for kappa is k = .70. An exploratory factor analysis was conducted on the observation data and the determined factors related to the theoretical framework established in the literature. The results affirmed that the instrument is a tool to be utilized in short-cycle formative assessment with middle and high school science and mathematics teachers. |
