Comparison Of Cognitive Diagnosis Attribute Reliability Indexes And Its Improvemented Indexes

Reliability is an important indicator for evaluating the test results stability and consistency.Based on the characteristics of Cognitive Diagnosis tests is to divide examinees into different categories, reliability index of criterion-referenced test based on contingency table were introduced into the cognitive diagnosis,Some Psychometric scientists developed attribute reliability indexes of cognitive diagnosis, mainly Templin's attribute retest consistency, Wang's attribute classification consistency. Taking DINA model as an example,this paper improve the attribute reliability indexes,and compare the performance on estimation accurary of the reliability indexes before and after improvement.Then, kappa coefficient which is important on the reliability index system of the contingency table of the criterion-referenced test is introduced into the cognitive diagnosis.This paper contains two studies.Study one compare estimation accuracy of attribute retest consistency and attribute retest consistency improved under the condition of different test length, different test quality and different examinees sample sizes. Setting different test length and qulity, and different examiness sample,Study one also compare estimation accuracy of attribute classification consistency before and after improvement.Changing the length and quality of the test,the sample size of the examinees, Study two compare estimation accuracy of attribute kappa coefficient and attribute kappa coefficient improved.The main results showed as follows:(1)Whether tests contain three or five attributes,estimation accuracy of improved attribute classification consistency is better than the original attribute classification consistency.When the test quality is below average level,Improved index can effectively change the phenomenon of the original index overestimated real attribute classification consistency. Estimation of attribute classification consistency influenced by test length and qulity, and different examiness sample. With quality of test quality become better, and the increase of the test length and the examinees, attribute classification consistency estimation precision become higher.(2)Whether tests contain three or five attributes, in the case of high test quality, attribute retest consistency and the improved index estimation accuracy is better. Under the condition of poor test quality, As the sample size increases, the test becomes longer,estimation accuracy of atrribute classification consistency is become better.Estimation accuracy of improved attribute retest consistency is better than the original attribute classification consistency.(3)Estimation accurary of attribute kappa coefficient influenced by test length and qulity, and different examiness sample. With quality of test quality become better, and the increase of the test length and the examinees, attribute kappa coefficient estimation precision become better.When the test quality is below average level,Improved index can effectively change the phenomenon of the original index overestimated real attribute kappa coefficient.