Point And Interval Estimation Of Cronbach’s Coefficient Alpha For Attribute Reliability In Cognitive Diagnosis

Cognitive diagnosis(CD)is an important topic in psychometrics,which is a form of testing that employs a cognitive model to provide examinees with information about their cognitive strengths and weaknesses.The first step is to develop items that measure specific knowledge and skills,and then using this model to guide the psychometric analyses of the examinees’ item response patterns to measure these specific knowledge structures and skills.In the field of cognitive diagnostic tests,attribute reliability refers to the precision of score decisions about examinees’ attribute mastery level.Gierl,Cui and Zhou(2009)have proposed the attribute reliability of the Cronbach’s alpha coefficient,which is a weighted score reliability of subjects weighted items.This idea has extented the Cronbach’s alpha coefficient into CD.However,the above point estimation method gives a single value of attribute reliability and will inevitably produce errors when sample size is small.Interval estimation is the use of sample data to calculate an interval of plausible values of attribute reliability.As there is no research to discuss the interval of Cronbach’s coefficient alpha for attribute score in CD.Therefore,it is very necessary to develop an interval estimation method of Cronbach’s alpha for the attribute score in cognitive diagnosis.Cronbach’s alpha in CD inherited the reliability definition of the classical test theory,can be calculated using the ratio of two weighted score variance on the items that probe specific attributes or the correlation coefficient of retest.So the cognitive diagnosis reliability coefficient is regarded as the correlation coefficient,and interval estimation methodS of the reliability in classical test theory,can be used to obtain the interval estimate of the Cronbach’s alpha attribute reliability coefficient(Fisher,1950;Feldt,1965).Therefore,based on interval estimation methods of the alpha coefficient in classical test theory,this study explored confidence interval estimation methods of Cronbach’s alpha attribute reliability in CD,and scientifically promoted the progress of the interval estimation method of attribute reliability in CD.There were three studies in this study.In the first study,explored the performance of point estimation of Cronbach’s alpha for attribute reliability in CD and evaluated the accuracy of Cronbach’s alpha for attribute reliability in CD.In the second study,ten interval estimation methods in classical test theory were introduced into confidence interval estimation methods of Cronbach’s alpha for attribute reliability in CD.Then results of the ten interval estimation methods were examined to recommend optimal methods among ten interval estimation methods.In the third study,we compared the performance of ten interval estimation methods in empirical data.The results were as follows:(1)Study 1 showed that the quality of the test,sample size and the length of test affected the average reliability of Cronbach’s alpha coefficient in cognitive diagnosis.Mean reliability increased as the test length and sample size increased.Mean reliability also greatly enhanced as the quality of test increased across each item set condition.Bias and RMSE of reliability of Cronbach’s alpha coefficient in cognitive diagnosis decreased as the test length,sample size and the quality of test increased.(2)Study 2 showed that the quality of the test,sample size and the length of test affected the performance of ten interval estimation methods of attribute reliability of Cronbach’s alpha coefficient in cognitive diagnosis.The length of confidence interval decreased as the test length and sample size increased.The confidence interval also decreased as the quality of test increased across each item set condition.(3)Study 2 also showed that the fifth method(Feldt method)and the seventh method(accurate Koning-Franses method)were better than other methods.Their results were also slightly different,and their calculation process was also relatively simple.In order to calculate confidence interval of Cronbach’s alpha for attribute reliability in cognitive diagnosis,we would need to know Cronbach’s alpha value for attribute reliability in cognitive diagnosis,number of subjects,F-threshold value,and number of items.(4)Study 3 showed when using Tatsuoka fractional subtraction data as empirical data,confidence interval length of the fifth method(Feldt method)is the shortest.These results were almost the same as the results of the simulation study(Study 2).