| Educational and psychological measurement is an important means in the studyof education for evaluating and realizing the level of developing for individual, whichwas applied in education and psychology for diagnosing, consulting, evaluating, select-ing and sorting personnel widely. The development of the practice for measurementmust be obey the theoretical direction. In the development history of educational andpsychological measurement, there are two primary theories. One is classical test theory(CTT), with the core of true score model (Gullikson,1987; Lord&Novick,1968), ituse the concept system such as reliability, validity to assess the quality of a test work.The other is modern test theory, the core content of this theory is item response theory(IRT; Lord,1980; Hulin, Drasgow&Parsons,1983; Hambleton&Swaminathan,1985;Hambleton, Swaminathan&Rogers,1991; Baker&Kim,2004). Along with the rapiddevelopment of computer technology, item response theory has been the main objectof the study of modern education measurement. It has been attended and favored bymore and more researchers and educators. At present, the two test theory coexist inthe modern test domain. Compared to classical test theory, item response theory hasobvious advantage. The item parameters (such as difculty parameters, discriminationparameters and guessing parameters) it takes are an index which don’t be influencedby samples, that is, the estimation value of these parameters will not change along withthe change of the samples which were taken as testing items. Just because of theseadvantages, item response theory is widely used in psychological and educational mea-surements. For example, the computerized adaptive testing (CAT; Wainer,1990)which based on item response theory are adopted one after the other by some large examssuch as TOFEL、GRE、GMAT, etc. However, this doesn’t mean that item responsetheory is a perfect measurement theory. Owing to item response theory is constructedon the following assumption with strong restrict condition:(1). The unidimensionality assumption of the latent traits space;(2). The local independence assumption of the test items;(3). The monotonicity assumption of response probability of examinee.In view of above drawback, there are something need to be perfected and mended,thus, some new model and method must be presented. In recent years, the centralresearch topics in the classical item response theory literature are multidimensionalitem response theory, testlets response theory, nonparametric item response theoryand cognitive diagnostic theory.Although the test system was found in our country, but we don’t investigate thisknowledge via scientifically analyzing quantificationally, so that psychometrics developin overseas. Along with the introduction of psychometrics, our country’s educationaland psychology evaluation system and method are underway stage. The present examin our country mainly tests the students’ grasping knowledge, which can’t reflect thelearning ability of the examinee, the reason is that the investigation base is very feebleand there are little researchers working on educational and psychological measurement(Tao Xin,2005). Many current standardized educational tests and evaluation havethe phenomenon that groups of items based on a common material or stimulus. Suchgroupings, usually called testlets (Wainer&Kiely,1987). Simulation studies showthat when there are dependence relationship exiting in a test, applying item responsetheory model which ignore the dependence relationship will result in an overstatementof precision of proficiency estimates as well as a bias in item difculty and discrimi-nation parameter estimates. Overstatements of precision and biased estimation leadto inaccurate inferences about the parameters (Sereci, Wainer&Thissen,1991; Yen,1993; Wainer,1995; Wainer&Thissen,1996; Chen&Thissen,1997; Wainer&Wang,2000). Therefore, investigator generalize item response theory to testlets response theory (TRT), for test data with dependence relationship among items, investigatoradopt the testlets model which acknowledging the dependence relationship among items(Bradlow, Wainer&Wang,1999; Wang, Wainer&Bradlow,2002; Wang,&Wilson,2005; Li, Bolt&Fu,2006). In our country, there are many testlets items such as thereading comprehension items, audition comprehension items, matching items and clozein Chinese test and foreign language test, judgement choice item for situation in thetest for choosing and sorting person, etc. But, there are little theoretical study on thetest item which don’t satisfy the local independence (Tu Dong-bo, etc.,2009).Dichotomously scored testlets response theory model was firstly proposed by Brad-low, Wainer&Wang(1999) by introduced a random efect parameter in order to expanditem response model. According to theirs idea, the two-parameter normal ogive testletresponse theory model (named primary two-parameter normal ogive testlet responsetheory model) should bewhere, yijdenotes the response for the examinee i and item j is coded as1or0according to whether the examinee answers the item correctly or incorrectly. Φ denotesthe cumulative distribution function (CDF) of the standard normal distribution, ajdenotes the discrimination parameter of item j, using it to discriminate the ability ofexaminee, bjis the difculty parameter of item j, θiis the latent trait of examinee i(also named ability), γid(j)is a random efect that represents the interaction of personi with testlet d(j)(i.e., testlet d that contains item j). The random efect γid(j)canbe interpreted as a random shift in examinees’ ability or another ability dimension.Li, Bolt&Fu (2006) think that the discrimination power of a item for examinee’sability and the testlet random efect shouldn’t be same, so they propose a generaltwo-parameter normal ogive testlet response theory modelwhere, item parameters aj1and aj2indicate the discriminating power of an item withrespect to the primary ability θ and the random efect γd, respectively, the two param- eters can be defined independently. tjis a threshold parameter related to the difcultyof the item.In this paper, we mainly discuss the parameter estimators and actual applicationof dichotomously scored testlets response theory model. In what follows, we introducethe main results of this paper.Firstly, because the secondary dimension γid(j)is a random efect that representsthe interaction of person i with testlet d(j), it is believed that the loading of the sec-ondary dimensions γdshould be discriminating power of the testlet with respect to it,and it should be related to the discrimination parameters of the items in the testletwith respect to the intended ability θ. Therefore, we introduced a new item param-eter as testlet discrimination parameter and proposed a new two-parameter normalogive testlet response theory model based on the testlet discrimination parameter fordichotomously scored items as followswhere, the meaning of parameters aj, bj, θi, γid(j)are the same as above. αd(j)is anew parameter introduced in this paper firstly, named testlet discrimination parameter,it is relevant to the discrimination parameters of all items in a testlet. The definitionof testlet discrimination parameter as followswhere, the token nd(j)denotes the numbers of items in testlet d(j) and Sd(j)is the setof the serial numbers of items in the testlet.Secondly, we study the problem of parameter recovery of parameter estimationand the fitting for the new two-parameter normal ogive testlet response theory modelby Gibbs sampling scheme (a number of Markov Chain Monte Carlo scheme, MCMC)under the data augmentation scheme (DAGS) from Bayesian framework. We evalu-ate the recovery characteristic and accuracy of the traditionary two-parameter normalogive item response theory (2PNOIRT) model, the primary two-parameter normal ogive testlet response (2PNOTRT) model and our new two-parameter normal ogive testletresponse (2PNOTRT) model based on the testlet discrimination parameter using meansquared error (MSE) and mean absolute error (MAE) between true and estimated itemparameters, results show that MSE and MAE of the proposed2PNOTRT model aresmaller than that of the traditional2PNOIRT model and primary2PNOIRT generally.The item parameter estimates of the proposed2PNOTRT model are overall satisfac-tory.Finally, we apply the proposed2PNOTRT model to a set of actual item responsedata which from an English reading comprehension test with testlet structure of univer-sity of Maryland College Park, USA. In order to explain the actual data perfectly, wefit and compare this data by the traditionary2PNOIRT model, the primary2PNOTRTmodel and our new2PNOTRT model. Bayesian deviance information criterion (DIC)result indicate that our new2PNOTRT model provide a better explanation. Base onthis data, we obtain Bayesian estimation and95%credibility interval for item parame-ters of28items and ability parameters of1289examinees. Generally, the computationalresult is reasonable. |
PDF Full Text Request