Back Random Responding Detection Method Under MIRT Framework

Aberrant responses,which are caused by examinees’unusual behaviors（e.g.,carelessness,speededness,warm-up,plagiarism）,are frequently seen in various testing programs.The appearance of aberrant response seriously damages the test validity and the generalizability of research findings,so it becomes very important to detect it.The most frequently encountered type of aberrant response behavior is back random responding（BRR）in the real-life testing.The detection rate of traditional methods for BRR is around 0.5 or even lower,while Yu and Cheng（2019）obtained ideal power using change-point analysis（CPA）to detect BRR.Previous studies about the BRR detection under CPA are conducted with unidimensional assessment using Unidimensional Item Response Theory（UIRT）,however,multiple latent traits are often measured in psychological assessment,and the traditional CPA methods are not suitable for these conditions due to that items measuring the same trait in multidimensional surveys,questionnaires or tests are not sequentially but alternately arranged with items of other traits.To fill this gap,the main purpose of this paper is to develop a CPA method that can detect BRR correctly and estimate BRR starting position accurately with multidimensional assessment using Multidimensional Item Response Theory（MIRT）.This study proposes an optimized CPA method(SR_max)based on the residual between the sum of squares of deviation to detect BRR.In order to verify the performance of the proposed method(SR_max)in the detection of BRR,this study used Monte Carlo simulation research to simulate the null distribution of SR_max,and systematically calculated the corresponding critical value（SR_c）of the method under different test lengths and different interdimensional correlations.At the same time,under different test lengths,interdimensional correlations,BRR severity,and BRR prevalence rates,the new CPA method and the existing methods(R_{ma x}、L_{ma x}和W_max)are compared in terms of Power,Type-Ⅰ error rates and the estimated bias of the starting position for change point.Results indicated:（1）The critical values of the four methods all increase with test length.However,except for W_max,interdimensional correlations have little influence on the critical values of other three methods.（2）Across all conditions,the Type-Ⅰ error rates for the four methods are close to the nominal level 0.05.therefore,the new CPA method proposed in this paper can result in a well-controlled Type-Ⅰ error rate for the BRR detection.（3）The new CPA method proposed in this paper has higher Power than other existing CPA methods in BRR detection.Power is mainly affected by CPA statistics,test lengths and BRR severity,while BRR prevalence rates and interdimensional correlations show little effect on it.（4）When the severity of BRR is high（start in the middle with random responses）,compared with other CPA methods,the bias of the starting point for the change estimated by the new CPA method proposed in this paper is the smallest.The mean（standard deviation）of the estimated bias of the change point position is also affected by CPA statistics,test lengths and BRR severity,while BRR prevalence rates and interdimensional correlations show little effect on it.（5）In the empirical study,the performance of the new method was verified by calculating the detection overlap between the new method and the existing method,and the graph showed that the provisional estimated latent traits of the respondent affected by the BRR fluctuates more than normal respondent.The CPA method based on the residual between the sum of squared deviations proposed in this study has important theoretical and practical significance for the detection of aberrant response.Finally,the article further discusses the deficiencies and defects in the research,as well as the prospects for further improvement and perfection in the future.