Researches On The Development Of The Revised DINA Model And Automatic Matching Three Cognitive Diagnostic Models With Attribute Structure

With the coinciding developments in psychometrics and cognitive science in thepast fifty years, more and more researchers are interested in combining these twofields to a new psychometric area, often called Cognitive Diagnostic Assessment(CDA). As the new testing paradigm in the21st century, CDA have called forincreasing research and application. As Fu (2005) noted, the number of existingcognitive diagnostic models (CDM) had been more than60. Recently, a series of newCDMs are proposed, such as G-DINA, P-DINA and so on. Having so much CDMs,one of the key emerging challenges facing both research and practitioners is thefinding of a parsimonious model with enough cognitive skill information to be usefulfor student and instructor. So model selection and evaluation of CDMs are requiredcloser attention in future research.The DINA(deterministic inputs noisy “and” gate) model and the R-RUM(reduced reparameterized unified model) are fairly popular among the cognitivediagnostic models. Although holding same assumption on knowledge structure,response strategy and Q-incompleteness, the two models are totally different on theirrepresentation of attribute structure. The DINA model assumes equal probability ofsuccess for all subjects lacking some attributes for an item, no matter which attribute.That is to say, only examinee who masters all the required attributes on a given item,can he/she respond correctly on that item. Lack of any skill will lead an incorrectanswer. The R-RUM assumes an increasing probability of success as the number oftask-relavant attributes the examinee masters increases. Moreover, it holdsheterogeneity assumption among attributes in responding items. According to thehypothesis of R-RUM, examinee that lacks some relative attributes will decreasehis/her probability of success on a given item, and the decrease depends on the importance of the absent attributes in item responding. It is thus clear that thecomplexness of attribute structure in item responding assumed by the DINA modeland the R-RUM are totally different, with the former being very simple and the latterbeing complicated.Obviously, from the representation of the complexness of the cognitive process intask solving, these two models can hardly meet the requirements in the practice ofCDA. Taking this point into account, this study introduced the R-DINA (reviseddeterministic inputs noisy “and” gate) model which relaxes the DINA model’sassumption of equal probability of success for all subjects lacking some attributes foran item by redefining the ideal response ηijof the DINA model. It assumes that theprobabilities of success are consistently increasing as the number of task-relevantattributes the examinee masters increases. On an given item j, the R-DINA modelpartitions the latent classes into kj+1latent groups while DINA model alwayspartitions into two, where kjrepresents the number of required attributes for item j.Compared to the R-RUM, the R-DINA model imposes restrictions of attributeshomogeneity on item responding, and thus it is much more simple than the R-RUM.The representation of cognitive assumption of the DINA model, the R-DINA modeland the R-RUM can be recognized as from simple to complex.The selection and evaluation of cognitive diagnostic model for data analyzing is afundamental step in CDA. Matching CDM with the underlying cognitive process oftask solving is the prerequisite of a valid and accurate CDA. Considering thatpsychological and educational diagnostic assessment are usually knowledge intensiveor skill intensive, the attribute structure in item responding within one test maycomplicated. Under these circumstances, several CDM, such as the DINA model, theR-DINA model, the R-RUM and other models are need simultaneously in testanalyzing.In order to inspect theoretically and empirically the performance of the newmodel, and to select appropriate CDM to match the attribute structure underlying item responding, this thesis mainly conducted three researches as follows:Part three firstly defined and explained the item response function of the R-DINAmodel, and then achieved model estimation using EM algorithm and MCMCalgorithm. The two algorithms of the new CDM were examined according severalcriteria, such as accuracy of item parameter recovery, correct classification rate andconsistency of classification. Results showed that (1) programs of the two algorithmswork well and can provide stable and reliable estimates;(2) according to the accuracyof item parameter recovery, correct classification rate and consistency of classification,the R-DINA model with EM and MCMC algorithms performs well with EM slightlybetter than MCMC.Part four compared the DINA model, the R-DINA model and the R-RUM throughsimulate and real data analyses in order to examine the performance of the new CDM.In the simulation study, model-data fit indexes, accuracy of item parameter recovery,correct classification rate were employed to make comparison among the threemodels. In the empirical study, the three models were used to analyze Tatsuoka’sfraction subtraction data. Simulation and real analysis results indicated (1) in somesituations, especially when attributes hold relatively homogenous effect oupon itemresponding, the R-DINA model can be recognized as a simple model of the R-RUM;(2) when model data misfit occurs, the decrease on accuracy of item parameterrecovery and correct classification rate observed from using DINA model to analyzedata of R-DINA model or R-RUM is more serious than that of the vice versa;(3)Results of the three model’ analyses on the fraction subtraction data implied that theattribute structure in solving items may not the same.In fact, attribute structure in solving items in a given test may diverse owning thecomplexity of human cognitive process. The main purpose of the fifth part wasdedicated to design an effective method to check the attribute structure. A reversiblejump markov chain monte carlo (RJMCMC) algorithm was developed to achieveattribute structure checking and automatic model determination as well as model estimation among DINA model, R-DINA model and R-RUM. Simulation study wasconducted and verified the feasibility and validity of this method. Finally, applicationof the RJMCMC on the analysis of the fraction subtraction data showed that (1)RJMCMC holds relatively good model-data fit on the test;(2) from a quantitativeperspective, the attribute structure of this fraction subtraction test may more likelysatisfy the hypothesis of the R-RUM and R-DINA model, that is, the probabilities ofsuccess are increasing as the number of task-relevant attributes the examinee mastersincreases.