Realization And Validation Of Matrix Reasoning Testing Based On Item Generation Algorithm

Because of its advantages of easy operation and free from the cultural influence, matrix reasoning test, as an effective measurement instrument of g factor, is widely applied in clinical evaluation, recruitment and placement, and so on. However, matrices reasoning test, represented by raven’s progressive test, have the weakness of overexposure. Item generation is a very promising measurement technique that reply to the overexposure. In this research, item generation based on Raven’s progressive matrices, was preliminarily realized by systematically manipulate the influence characteristics of item difficulty on matrices design and improving matrix reasoning item of algorithm. The research was an attempt to develop the underexposure and rich feedback matrix reasoning test.In the study, matrix reasoning items were generated based on Raven’s Standard Progressive Matrices and Advanced Progressive Matrices, which were all three rows and three columns. First of all, the response data from the college students on the items of the two Raven’s Progressive Matrices were analyzed in normality, unidimensionality assessment, IRT model fit and measurement precision, in order to investigate the feasibility of the combination of the different matrix reasoning test versions. Then the researchers adopted a hierarchical regression model to explore the influence of the item design features on the Raven’s reasoning item difficulty, cognitive model of problem solution on matrix reasoning items was obtained.The matrix reasoning item generation and test system were developed on the basis of the obtained cognitive model and cognitive design system approach, through systematically manipulate the variation of 8 graphic attributes to implement six different rules, a matrix reasoning bank with large quantity of items were generated. According to the 31 item design structures, items were drawn from the matrix reasoning bank to assemble three parallel tests, each test consist of 36 items, the response data of college students on the items of one of the three parallel tests were analyzed to examine each test of its normality, unidimensional, IRT model fit and measurement precision, at the same time investigated the influence of test versions, item design structures, answer location on item reaction time as well as item difficulty, and the influence factors in item design structures were also explored.In order to investigate the differences between items from Raven’s test and generated bank, the researchers based on 16 common item design structures, drawn items respectively from the Raven’s reasoning test and the generated bank to assemble validation test, based on the item difficulty, item discriminate and reaction time, as well as the test score and time-consuming at the subject level, the differences between the different sources items were examined.The present study based on college students with the number of 1602, the main research conclusions are as follows:1.The psychometric characteristics of combined matrix reasoning with items come from two different sources reached the desirable level.2.The prediction of item difficulty by item design features is better than the prediction of item discriminate by the same variables. The visual-perception variables could predict item property significantly based on cognitive complex variables, the model that the rule of logic XOR separated from the rule of Distribution of 2 values performed better prediction.3.Three parallel tests of generating items conform in normality, unidimensionality, IRT model and its measurement precision has met the psychometric demand, basic requirements of the item design features can significantly predict the difficulty and the reaction time, however, the test version and answer location of items could not significantly influence the reaction time and item difficulty. The effect of Cognitive complex variables on the project is item difficulty is significant, however, the model adding visual perception variable didn’t have significant improvement on prediction.4. There are not significant difference of reaction time and item difficulty among different sources of matrix reasoning items under the same item design structures, the validation test is the unidimensional measurement structure, and the items fit the IRT model.On the basis of above research, the researchers reflect the innovation and deficiency of in the field, and propose practical suggestions of test evaluation.