| Assessment of cognitive models developed by students in complex scientific disciplines ideally captures the progressive and dynamic nature of learning. A computer-based performance assessment system (Stevens, et al. 1991) was developed to study student problem-solving skills. This system is able to track student problem-solving strategies through a process called "search path mapping", which reveals differences between successfully executed performances and unsuccessful strategies. Search path map analysis can be automated by using artificial neural networks (ANNs) trained with data from previous student performances.;Hypothesis formation during problem solving was studied with a combination of search path map and ANN analysis. We compared the number of hypotheses utilized by students on practice and examination problems in the same specific area of immunology. We found that medical students used information more efficiently during the examination, resulting in fewer hypotheses generated. This demonstrates the utility of ANN-based assessments, and suggests additional studies needed for this approach to become part of a comprehensive evaluation system (Hurst et al., 1997).;Our ANNs are highly reliable. By training multiple supervised ANNs, we have started to examine how medical student performances in immunology are clustered and derive estimates of "inter-rater" reliability of the networks. For ;Using several different problem sets, we have begun to address the differences between problem-solving performances of students and experts. In a high-school level genetics problem, our expert population was UCLA lower-division biology students, while our study population was high-school students. ANN analysis was able to distinguish between student and expert problem-solving strategies as well as between successful and unsuccessful strategies. We found that students utilized information items which were seldom ordered by the experts.;These studies examine the development of computerized tools for delivery and analysis of student problem-solving strategies. The pattern-recognition abilities of ANNs allow for both deeper (hypothesis formation) and broader (larger student groups across grade level and discipline) analysis of student learning has been than previously possible. |
