| This study was conducted to investigate the stability of 3 Pathfinder measures: relatedness, similarity, and coherence. To this end, a Soloman (1949) 4-group design was employed to answer 3 research questions: (a) how do instruction and premeasurement sensitization affect postmeasurement relatedness, similarity, and coherence scores; (b) what is the reliability of Pathfinder-related measures; and (c) to what extent is the test-retest reliability of students' relatedness, similarity, and coherence scores explained by prior knowledge?; In all, 236 students in 15 preservice educational technology classes participated in the study. Depending upon random group assignment, these students completed a premeasurement set of 435 concept ratings relating to Microsoft's PowerPointTM software, and/or received instruction in using PowerPointTM, and/or completed a postmeasurement set of concept ratings. The Appendices contain a complete set of stimulus materials for use by future researchers.; To answer the first question, a 2 (premeasurement by no premeasurement) x 2 (treatment by no treatment) mixed-effects MANOVA was conducted on students' postmeasurement scores, which were averaged across the class sections. The treatment effect was statistically significant (Wilks' Lambda = .174, F(3, 9) = 14.238, alpha = .0125, p = .001). There was no evidence of premeasurement sensitization. The results of a post hoc stepwise discriminant analysis revealed that approximately 76 percent of the variance in students' Pathfinder-related measures could be explained by their relatedness scores alone. The discriminant function correctly classified all 15 cases comprising the sample. Coherence scores were found to be collinear with relatedness and similarity scores.; To answer the second question, test-retest reliability coefficients were calculated for each student's Pathfinder measures. Correlations for Control Group 1 are as follows: relatedness = .812, similarity = .406, and coherence = .608. Although the reliability of relatedness scores meets Carmines and Zeller's (1979) recommendation of .80, similarity and coherence scores do not.; In answering the third question, it was found that the variance in students' postmeasurement ratings could be explained by their premeasurement ratings and group assignment. Test-retest reliability could not be explained using any of the 4 prior knowledge measures employed in this study: application experience, computer experience, application knowledge, or GPA. |
