Determining structure in test performance: An artificial neural network approach

The Kohonen Self-Organizing Map (SOM), a kind of artificial neural network, is evaluated for its efficacy in determining test structure in educational measurement applications. It is argued that the SOM may be particularly useful for this function since it can reveal both the dimensional (latent trait) and class (latent state) structure of complex data. A series of monte carlo experiments assessed the capacity of one- and two-dimensional, small and large SOMs to determine the structure of data composed of dichotomously-scored test items. These data were simulated to comprise latent classes and varied with respect to the discrimination of the individual items and the dimensionality of the data as a whole. In addition to the important role for item discrimination in producing high quality projections and low quantization error, the relationship between characteristics of the map and the complexity of the data was found to be critical for the SOM to effectively represent test data. In particular, it was determined that SOMs most accurately preserved adjacency and proximity relationships when the intrinsic dimensionality of the data matched the number of co-ordinate axes of the map. Implications for future applications of SOMs in educational measurement are discussed, as well as suggestions for further research.