Research On Quantile Multilevel Item Response Theory And Aberrant Response Behavior

In the context of educational and psychometrical measurement,many questions revolve around the analysis of persons’ latent traits.Item response theory(IRT)has many advantages in measuring the characteristics of items and latent traits of persons,so it has a wide range of applications.In IRT,the latent trait usually refers to a person’s ability.This paper focuses on two questions,the first one is to explore the correlation between students’ characteristics,such as gender,intelligence,family background and students’ academic performance(ability).Grasping the influence effect of these factors on students’ ability is helpful to provide theoretical support for the formulation and implementation of educational policies,so as to improve students’ academic performance.The other question is to make an objective and accurate assessments of abilities,and to exclude undesirable factors that may affect the results of the assessments.Here,we mainly focus on examinees’ aberrant response behaviors in the test,including the warm-up behavior caused by the nervousness at the beginning of the test,or the test speededness caused by the time limit,and so on.The responses under aberrant response behaviors will affect the validity of the test,and then affect the analysis and evaluation of examinees’ abilities.Therefore,it is of great significance to detect aberrant response behaviors.Based on IRT,this paper studies the above two problems,which are divided into the following two parts:In the first part,many studies proposed the Multilevel IRT(MLIRT)model based on IRT,and analyzed the influences of factors(explanatory variables)on the average abilities of students.However,when the distribution of ability is asymmetric,the analysis of average abilities is not representative.In recent years,many studies have paid much attention to the whole distribution of abilities,but there is no literature to study the effect of explanatory variables on the whole distribution of abilities under the framework of IRT.In this paper,quantile regression(QR)analysis is first introduced into IRT,a QR model is established for the relationship between ability and explanatory variables,which is called the quantile MLIRT(Q-MLIRT).Q-MLIRT has the excellent properties of both IRT model and QR model.It not only takes into account the errors of ability measurement,but also makes a more complete analysis of the relationship between explanatory variables and abilities.In addition,the distribution of abilities is asymmetric,the measurement of the center position of ability is more accurate.In addition,QR does not depend on the assumption of homogeneity of variance,so it is more robust.In this paper,a Gibbs sampling algorithm is proposed under the Bayesian framework to estimate the parameters in the Q-MLIRT model,and the uncertainty of all parameters in the estimation process is considered.The simulation study shows that the Gibbs sampling algorithm can accurately estimate the parameters of Q-MLIRT model under different conditions.Finally,the Q-MLIRT model was used to analyze the relationship between students’ math achievement and students’ individual and family background factors in the 2018 Program for International Student Assessment(PISA),and the application of the model was verified.In the second part,many studies aim to detect aberrant response behavior,and usually delete the responses of the examinees with aberrant response behaviors in order to improve the calibration of item parameters and the estimation of the abilities.In recent years,the detection method of change-point analysis(CPA)has attracted the attention of researchers.CPA can not only detect aberrant response behavior,but also provide information of change point(the location of aberrant response behavior),so data can be filtered in a targeted way.In this paper,a Bayesian change point analysis detection method is proposed.Different from the current CPA methods,this method uses the response times(RTs)information of the examinees on each item,which can improve the detection efficiency of aberrant responses.At the same time,the numbers and locations of change points are taken as random variables,combined with the prior information and data information,they are inferred from the posterior distributions.Then,according to the positions of the change points,the speed of the examinees in each stage of the test can be estimated.The advantages of this method are as follows :(1)it allows each examinee to have multiple aberrant response behaviors or multiple change points;(2)It is less dependent on the positions of the change points and does not depend on the maximum likelihood estimation(MLE)of the parameter;(3)The aberrant response behavior can be judged more accurately by combining the information of change points and examinee’s speed.The simulation results show that,the method can not only effectively detect the aberrant response behavior,but also accurately estimate the numbers and locations of change points,meanwhile,the false alarm rate(the proportion of examinees who have no change points but are detected to have change points)can be controlled within a reasonable range.Finally,the paper analyzes data from computerized adaptive testing(CAT)using Bayesian CPA,and detect the aberrant response behaviors of examinees,which verifies the practicability of the proposed method.