The validity of the Martinello Open -ended Science Task: A performance assessment of children's scientific thinking

The Martinello Open-ended Science Task (MOST), a performance assessment of children's scientific thinking, was subjected to tests of reliability and validity and to factor analysis. Results indicated that the MOST is capable of yielding scores that are (1) stable over time, (2) related to criterion scores, and (3) sensitive to instruction designed to improve students' scientific thinking skills.;The past fifteen years have seen the development of a variety of assessment techniques designed as alternatives to norm-referenced, standardized science achievement tests. The MOST was developed as a diagnostic tool to measure children's observational and inferential thinking skills with minimal time and resource requirements. Students' responses to questions about an unfamiliar natural object presented to them are coded for the number of occurrences of descriptors, metaphors, incorrect inferences, correct inferences, information-seeking questions, and hypothesis-testing questions, yielding six scale scores and a grand total.;Data for the study came from the evaluation of the Minority Mathematics and Science Education Cooperative (MMSEC), a federally funded teacher training project designed to improve science and mathematics instruction. Elementary students (n = 230) from four elementary schools, one participating in MMSEC and one control school in each of two urban Texas cities, completed the MOST in Year One of the study. A subset of these subjects (n = 172) was recovered for a second administration of the MOST in the spring of Year Two. Standardized science achievement test scores were available for a subsample of these students.;Significant test/retest reliability was identified for the MOST Grand Total and three of its scales. Significant criterion-related validity evidence was provided by Year One data only. Evidence of predictive validity was found for the combined subjects and the control group taken alone. Construct validity (instructional sensitivity) evidence was found for the Grand Total and three scales, although findings were mixed for the youngest subjects. Year One and Year Two factor analyses indicate that there may be two primary groups of skills tapped by the MOST, titled here "Observing" and "Inferring." Results are discussed in the context of current criteria for performance assessment.