Methods And Model Selection Of Analyzing Dichotomous Scored Self-report Questionnaire Using Item Response Theory

Aim: To explore the analyzing method and model selection for dichotomous scored self-report questionnaire based on Item Response Theory (IRT).Methods: Our research is from bottom to top, through analyzing typical data to discuss the methodology problems of applying IRT in the realm of self-report questionnaire. Firstly, we primarily classify self-report questionnaire based on their aims and characteristics. Then we selected one typical scale from each type, and analyze some of their subscales. Because the unidimensional models are more popular now for IRT application and the main models of analyzing non-cognitive questionnaires are accumulating models and unfolding models, we use Principal Component Analysis (PCA) to verify the unidimensionality of the data first, and use Mokken Scale analysis to check Monotonicity, and then apply 2-parameter Logistic model (2PL, accumulating model) and Generalized Graded Unfolding Model (GGUM, unfolding model) to analyze the data, and after that we use MODFIT software to detect the fit between the model and data, and at last we summarize the steps of applying IRT in self-report questionnaire based on analyses of reasonableness of the parameters and the model fit. For symptom rating scales and trait proneness scales, only one group of data is used to compare the effects of different models. For clinical personality scales, we also include comparison between high and low trait level, besides comparison of different models. In all the analysis, PCA is conducted using SPSS13.0 software; Mokken scale analysis is conducted using MSP5 software; 2PL model analysis is conducted using Bilog-MG software; and GGUM analysis is conducted using GGUM2004 software.Results: (1) Based on the characteristics and aim of questionnaires, we divides self-report questionnaires to three different types: symptom rating scales, trait proneness scales and clinical personality scales. In this study we choose Cognitive changes, Emotional responses and Psychiatric symptoms in Acute Stress Response Scale (ASRS) as representatives of symptom rating scale. And we choose Warmth (A scale) and Social Boldness (H scale) in 16PF as representatives of trait proneness scales. And we choose Dit scale in Chinese Soldier Personality Questionnaire (CSPQ) as representative of clinical personality scales.(2) Most items from ASRS can satisfy unidimensionality and monotonicity hypotheses. Parameters of 2PL for most of these items are reasonable, but all the"location"parameters are above 0, manifesting quasi-trait characteristic. And parameters of GGUM for some items are near extremeness. And results of model fit analysis using MODFIT software show that 2PL model can fit better with the data than GGUM model. So analyzing symptom rating scales can get better effects based on 2PL model.(3) Subscales of 16PF in this study can satisfy the unidimensionality hypothesis on the whole. But answering patterns of some items can not satisfy monotonicity well. And we find that discrimination parameters of 2PL model for many items are small, and tests'information is relatively small. But parameters of GGUM model for most items are reasonable. Accordingly, results of model fit using MODFIT show that GGUM model can fit better with the data than 2PL. So we may get better results if we use GGUM model to analyze trait proneness scales.(4) Data of CSPQ in this study can satisfy the unidimensionality hypothesis on the whole. And data from low trait level group can also satisfy the monotonicity hypothesis for many items, while data from high trait level group show answering processes of many items have slight violets of monotonicity. Results of 2PL model analysis show that, although parameters from low trait group are reasonable, they manifest quasi-trait characteristic, which should not exist, and test information for average subjects are relatively small; parameters from high trait group are also reasonable, and test information for average subjects are large. And results of GGUM model analysis show that, parameters from low trait group are reasonable, with good ability to discriminate average subjects, while many parameters from high trait group can't be estimated, which may affect the construct of the test. Additionally, results of model fit analysis show that for data from low trait group, GGUM model is better, and for data from high trait group, 2PL model is better. So when we analyze clinical personality questionnaires, we should choose proper model based on the characteristics of target subjects and the aims of investigation.Conclusion: We can't just transfer the method used in the analysis of cognitive scale to self-report questionnaire when we choose IRT models. We should first classify the scales based on their aims and characteristics, and then detect the unidimensionality and monotonicity of the data. And choose proper model based on the results above. However, we have just discussed the methodology problems for dichotomous scored scales primarily. The generalizability and accuracy should be explored further in the future.