The Performance Evaluation Studies Of RUM Which Have A Function To Assess The High-order Ability

Cognitive diagnostic assessment is based on cognitive science and combinedwith psychometrics. Cognitive diagnostic assessment is designed to explore human’slatent cognitive processes and cognitive structures. Cognitive diagnostic models aredefined as: probabilistic, confirmatory multidimensional latent-variable models with asimple or complex loading structure. They are suitable for modeling observablecategorical response variables and contain unobservable (i.e. latent) categoricalpredictor variables. The predictor variables are combined in compensatory andnoncompensetory ways to generate latent classes. Selecting a suitable cognitivediagnosis model plays an important role for a success cognitive diagnosis. In the thehundreds existing cognitive diagnostic models, RUM is considered as a successfulcognitive diagnostic model and uses widely. Soon after, a lot of Psychometricscientists is based on RUM and develop another models. In those models, HO-sRUMis the most similar to RUM, because both of them are measuring the high-orderability.In this study, we have used the simulation study and empirical experiments toevaluate the performance of RUM and HO-sRUM. The simulation study consists ofthree sub-study: First, performance evaluation of RUM and HO-sRUM. Second, faulttolerance evaluation of RUM and HO-sRUM. Third, how the consistency degree ofparameter pc i (θ j)influences HO-sRUM’s estimated performance. The empiricalstudy use real data to check the accuracy of both RUM and HO-sRUM.The simulation results show that (1) Data which is generated by RUM andHO-sRUM and estimated by the same model have an accurate estimated result.Except the ability parameterθ j.(2)Different models between the data generated andestimated bring an error in performance estimate, especially in performancer*.(3)The low consistency degree of parameter pc i (θ j)influences the accuracy ofHO-sRUM, especially in performancer*. Empirical experimental results show that (1) Reducing the useless attribute willimprove the accuracy of RUM.(2) Students who fail to answer the item is no masteror low master of the attribute in RUM.(3) Reducing the useless attribute both in RUMand HO-sRUM have a small influence in the parameter estimated.(4) The results ofparameter estimated using RUM and HO-sRUM have a high correlation coefficient.(5)There is a high classification consistency of attribute master pattern between RUMand HO-sRUM.