A Multiple Logistic Regression-based(MLR-B) Q-matrix Validation Method For Saturated Cognitive Diagnosis Models

As an essential component in cognitive diagnostic assessment,cognitive diagnosis models(CDMs)intend to provide researchers with comprehensive assessments for examinees’ proficiency.This method potentially classifies examinees into different groups with diverse mastery profiles on a set of specific skills or attributes that have been possessed or not.Most CDMs rely on a basic element,called the Q-matrix which represents the relations between the attributes and items.Q-matrix is usually defined by domain experts in the process of developing test,therefore,it’s specifications can be subjective,controversial or even incorrect.The misspecifications of the Q-matrix make an adverse impact on model fit,classification of examinees and parameter estimation.To estimate and correct the Q-matrix,a host of methods were proposed.Nevertheless,many existing methods cannot be applied to saturated CDMs,which limits their applications in practice.Other Q-matrix validation methods could be applied to saturated CDMs,but they have some disadvantages and have great room for improvement.To address the above issue,this study intends to develop a multiple logistic regression-based(MLR-B)method which can be used for general CDMs.This method is based on the concepts:(1)whether individuals master the required attributes of a given item influences their observed responses on this item;and(2)an attribute intends to be regarded as being measured by an item when the mastery of it takes a significant positive impact on individuals’ correct response probability on this item.Therefore,we can identify or determine the required attributes by item j by modeling the multiple logistical regression models between individuals’ observed responses on item j and their mastery probabilities of attributes.If attribute k plays a significant positive effect on the observed response of item j,we can say that attribute k is very likely measured by item j,else not.The performance of the MLR-B method was evaluated in two simulation studies with diverse conditions including the sample size,different kinds of misspecified Q-matrices and attribute distribution of individuals.Two existing method,Stepwise method and GDI method,were applied to compare with MLR-B method.In addition,a real data is used as an example to illustrate its application.Results show that the proposed MLR-B method outperforms the existing GDI method and the Stepwise method in different experimental conditions,especially in the condition of small sample size or high percentage of misspecifications in the Q-matrix.Real data analysis also suggested that the suggested Q-matrix by the MLR-B method is more reasonable than the original Q-matrix and the Q-matrix suggested by the Stepwise method in terms of model-fit indexes.As showed,the main contribution of this article is to propose a more effective method of Q-matrix validation,which can not only be applied to saturated CDMs but also has higher accuracy of Q-matrix validation than the existing methods.Moreover,the proposed method has an extra advantage of robustness due to that it still has acceptable accuracy of Q-matrix validation even under the condition of small sample size.