Identifying student mental models from their response pattern to a physics multiple-choice test

Previous work has shown that students present different misconceptions across different but similar physical situations, but the cause of these differences is still not clear. In this study, a novel analysis method was introduced to help to gain a better understanding of how different physical situations affect students' responses and learning. This novel analysis groups students into mental model groups (MMG) by similarities in their responses to multiple-choice test items, under the assumption that they have similar mental models. The Mass and Energy Conservation test was developed to probe the common misconception that objects with greater mass fall faster than objects with lesser mass across four physical situations and four knowledge sub-domains: information, dynamics, work, and energy. The test was applied before and after energy instruction to 144 college students in a large Midwestern university attending a calculus-based introductory physics course. Test time along with instruction and physical situation were the two factors. It was found that physical situation did not have a significant effect on mental models: The number of MMGs identified and the fraction of students belonging to the same MMG were not significantly different (p > .05) across physical situations. However, there was a significant effect of test time on mental models (p < .05): the fraction of students belonging to the same MMG changed from the pretest to the posttest, in that the MMG representing higher performance became predominant than the MMG with lower performance for the posttest results. A MANOVA for the average scores for each sub-domain and physical situation combination was applied to validate the previous results. It was found that a significant effect (p < .01) by physical situation resulted due to a lower average dynamics sub-domain score for the friction physical-situation attribute when compared to the no-friction physical-situation attribute. A significant effect (p < .01) was found for test time. This was due to an increase of the average energy sub-domain score from the pretest to the posttest. No significant interaction effect ( p > .05) was found. The MANOVA results obtained can be explained through the change in proportion of the MMGs present in the sample.