An Adaptive Design For Parameter Estimation And Knowledge Point Calibration Of Items In Cognitive Diagnosis

The implementation of cognitive diagnostic computerized adaptive testing(CD-CAT)often depends on a large item bank with high quality.The item bank of cognitive diagnosis test not only ensures that the content of the item conforms to the curriculum standard,but also estimate the parameters of cognitive diagnosis items and calibrate the knowledge points or attribute vectors of the item according to the answer data,which is the basis of realizing the selection of items according to the students' knowledge mastering state and evaluating the students' knowledge mastering state efficiently and accurately.At the same time,with the frequent use of some items in the item bank,as well as the emergence of problems such as curriculum standards and textbook revision,some items in the item bank need to be retired,modified,and even some new items need to be supplemented.Therefore,how to estimate the parameters of items and calibrate the knowledge points required by items based on item response data becomes an important problem to be solved for personalized adaptive learning and constructing high-quality large-scale item bank.In the early stage,the related research mainly focused on the calibration method,but the discussion about the online design method was relatively less.Especially in the estimation of item parameters and the calibration of attribute vectors of new items,many kinds of research adopt the method of random assignment of new items,which is not in line with the idea of "adaptive" in the CD-CAT.In order to solve the defects of random design,proposing an adaptive design for online calibration of attribute vectors and online estimation of item parameters for new items.Starting from the practical application and conditions,we mainly carried out three aspects of research:(1)because some cognitive diagnosis classification methods(such as nonparametric classification methods)needs the item attribute vector without item parameters for knowledge state classification,the first simulation study focuses on an optimal design of the new item attribute vector calibration based on Shannon entropy;(2)based on item attribute vectors for new items specified by subject experts or online methods,an optimal design for online estimation of item parameters based on Fisher information matrix is proposed;(3)to calibrate the attribute vector and item parameters for new items,a hybrid optimal design of the former two methods is proposed for both online estimation and online calibration.Under the condition that the number of attributes is 5 and the cognitive diagnosis model is deterministic inputs,noisy-and-gate(DINA)model,the above three optimal designs are applied to calibrate and estimate the attribute vectors and item parameters of 12 new items.Besides,in the first study,we discuss the influence of the number of students(100,200,400,800,1600)on the calibration of the attribute vector of the new items.In the second study,we discuss the impact of the small number of examinees(20,40,80,160,320)on the estimation of item parameters.In the third part,we mainly discuss the influence of two kinds of adaptive design on the accuracy of attribute vectors and item parameters.Three studies were compared with random design(i.e.examinees randomly selected items from 12 new items).The simulation results showed that:(1)with the increase of the number of examinees,the adaptive design of new online calibration and online estimation is better than that of random design;(2)using two kinds of adaptive design is better than using only one kind of adaptive design or neither,and more accurate results can be obtained when estimating item parameters.Whether the new adaptive design can be applied to different cognitive diagnosis models,different attribute hierarchies and so on,these problems are worthy of further study.