The Reliability Of Cognitive Diagnosis Test And Its Confidence Interval Estimation Method

Only when the reliability of test score is good can the test score be further analyzed.The study of reliability is of great significance in psychometrics and educational measurement.Cognitive diagnosis theory aims to evaluate individuals' cognitive processing,knowledge structure and cognitive skills.Plenty of cognitive diagnosis tests have been developed based on cognitive diagnosis theory.Further research on the reliability of cognitive diagnosis test will be beneficial to the application of cognitive diagnosis tests in mental health education and subject teaching.The existing researches on the reliability of cognitive diagnosis mainly focus on the reliability of attribute level,such as attribute retest consistency,attribute classification consistency,and improved Cronbach's ? coefficient,etc.However,there is little research on the reliability index of cognitive diagnosis test score.In fact,the reliability of test score indicates the reliability of participants' responses to the items.Only those tests with high reliability of test can be reliable when researchers analyze the participants' responses.Therefore,the reliability of test in cognitive diagnosis test was promoted in this paper.There were three studies in this research.In the first study,the formula of the reliability of test was derived.And a simulation experiment was designed to evaluate the performance of the reliability of cognitive diagnosis test and its estimation precision.In simulation experiment,there factors were manipulated: test quality,test length,sample size.In addition,the relationship among the reliability of test,pattern correct classification rate(PCCR),and classification accuracy(CA)was compared.In the second study,the variance of the reliability of test was derived using Delta method,and then a method for calculating the confidence interval of the reliability of test was given.The effects of test quality,test length,and sample size on confidence interval of the reliability of test were studied.In the third study,empirical data was used to examine the performance of the reliability of test and its confidence interval estimation methods in practical applications.The results were showed that:(1)The estimated accuracy of the reliability of cognitive diagnosis test were affected by test quality and test length,but not significantly affected by sample size.With the increase of test length,the estimatedvalue of reliability of test tended to increase.Mean bias between the estimated value and the true value of reliability of test decreased,as well as the root-mean-square error(RMSE),indicating that estimation accuracy was improved.With the improvement of test quality,the estimated value of reliability of test tended to increase.When the test length was medium or long,the root-mean-square error decreased and the estimation accuracy improved.(2)The reliability of cognitive diagnosis test was significantly correlated with pattern correct classification rate and classification accuracy.Under the condition of different test quality,test length and sample size,pattern correct classification rate and classification accuracy varied.What's different is that the reliability of cognitive diagnosis test was not affected by sample size.In this study,the reliability of cognitive diagnosis test merely increased along with the promotion of test quality and the lengthening of test length.(3)The confidence interval of the reliability of test was affected by test quality and test length,but not affected by sample size.The length of the confidence interval decreased and the coverage of the confidence interval increased as the test quality increased.The length of the confidence interval decreased and the coverage of the confidence interval increased as the test length increased.The Delta method could be used to estimate the confidence interval of the reliability of cognitive diagnosis test.(4)In the empirical study,the reliability of cognitive diagnosis test was high and the length of its confidence interval was short.