Development And Application Of GRCDM

Cognitive diagnostic assessment (CDA) becomes more and more popular in recent years with the rise of education idea of "No child left behind". CDA can measure students’knowledge structures and processing skill to provide information about their cognitive strengths and weaknesses and get more efficient remediation to achieve the education idea of "No child left behind". Cognitive diagnostic assessment need a tests which design based on cognitive model and an appropriate psychometric model. Using an appropriate psychometric model to measure students’observation response date come from cognitive diagnostic tests can offer and remedy students’cognitive structure. A lot of cognitive diagnostic models have being put forward and revising, most of them are parametric latent class modeling. Though parametric latent class modeling for cognitive diagnosis has advantage, it requires specialized software to estimate parameter and consume excessive time to use EM algorithm or MCMC to fit model parameters, which will make it inconvenience. So, researchers are interesting in nonparametric approach to cognitive diagnosis in recent years. Nonparametric cognitive diagnosis approach requires no statistical parameter estimation, so it is time saving and can be used on a small sample size. The characteristics of nonparametric cognitive diagnosis approach decided it good for class assessment.Cluster diagnosis method as a kind of nonparametric cognitive diagnosis approaches only needs a Q-matrix which implied by the item-by-attribute association. The research of cluster diagnosis method all based on 0-1 scoring, but, Education test often contains multi-level scoring items which reflect students thinking process in solving problems. Using cluster diagnosis method based on 0-1 scoring to analysis these items will lost a lot of information. Although there is no research focus on the influence factors of class accuracy of cluster diagnosis method. On the basis of previous research, this paper combined with the demand of the examination reform, put forward a method named grade response cluster diagnosis method (GRCDM), which are suitable for multi-level scoring and investigated its utility.Our study includes five parts. First, we introduced GRCDM. Second, we investigated the performance of the method and learn how different factors are act on class accuracy of this method, like the number of attribute, sample size, sample distribute, attribute hierarchy structure. Third, we investigated the impact of Q-matrix misspecification on classification accuracy of the method. Fourth, we investigated the impact of attribute hierarchy structure misspecification on classification accuracy of the method. Fifth, we investigated the performance of the method when it used in analyze a sample of 1240 fifth grade students’observed item response for arithmetic word problem solve.We got the following results:1. GRCDM has high pattern match ration has high pattern match ratio (PMR:96.08%) and high marginal match ratio (MMR:99.04%). This method was proved to be feasible in cognitive diagnosis assessment. The more attribute the test has, the higher classification accuracy this method has.2. The classification accuracy under divergent or unstructured attribute hierarchy structure were better than other attribute hierarchy structure, especially under unstructured attribute hierarchy structure had high classification accuracy. The classification accuracy will not influence by sample distribute.3. The classification accuracy (MMR and PMR) of this method is not dependent on the size of sample. This method had high classification accuracy even in a sample of 100 or a sample of 500, so it can be used in small assessment and classroom assessment.4. Only under linear, Q-matrix has big misspecification will lead classification accuracy decrease, others structures’PMR fall less than 5%, which means linear is sensitive to Q-matrix misspecification, others have good robustness.5. Attribute hierarchy structure has mistake will lead classification accuracy decrease, except "have hierarchy structure turn into no hierarchy structure", which the mean of MMR drop 0.006. So, if we are not sure two attributes have relationship, we see they are independent.6. We used this method to analysis empirical data find that the attributes difficulty by the classification of the method is suitable for attributes nature. The proportion of mastery in each attribute is different in different type school. The good one has more student mastery each attribute than the bad one. So, this method has high internal and external validity to apply to practice.