A Comparison On The Method Of Kernel Equating And Other Equating Methods Based On HSK Data

The equating of test scores derived from different test forms is significant. When equating is being carried out, test scores could be reported and explained more accurately. Also, equating keeps the evaluation criterion stable so that the quality of tests could be controlled.The kernel method of test equating is a new equating method. It integrates the linear methods and equipercentile methods based on classic testing theory into one frame. It converts the scores of the given testees' population on test form X into that of the observed score distribution on test form Y, so it is an observed score equating method. The kernel method of test equating has five steps, including presmoothing, estimation of test score probability, continuation, equating and calculation of standard error of equating. Kernel equating has been in use at Educational Testing Service (ETS) for some time.Is there any difference between the new methods under the KE frame and the traditional equating methods based on CTT? To what extent are KE methods different from those corresponding CTT methods? Is there any difference between methods under KE frame? Shall the KE equating methods be used in HSK equating? In order to answer these questions, this study constructs new test forms based on real HSK data to remove error, and comparison has been done in line with some equating criteria.Sixteen methods have been compared in this study, including 8 CTT methods—Tucker, Levine, Braun-Holland, Chain linear equating, presmoothed and unpresmoothed chain equipercentile method, frequencey estimation method, and 8 KE methods—KE chained with optimal bandwidth (CE optimal), KE chained with large h bandwidth (CE linear), KE poststratification with optimal bandwidth (PSE optimal), KE poststratification with large bandwidth (PSE linear)—each method under KE has two treatments, either presmoothed or unpresmoothed.The result shows that KE methods approximate their corresponding methods based on CTT under NEAT design. With random group equipercentile method as a criterion, the equipercentile methods under both CTT and KE frames perform well, but the chained equating method should be avoided for small samples; kernel linear mehods could produce the same results as the CTT methods without presmoothing. For large samples, the CE and PSE methods, the corresponding methods with optimal and large h values yield different results, and the differences are significant from zero. For small samples, the corresponding methods might produce similar results without presmoothing. Presmoothing plays an important role in the equating of smaller samples.Since the present test forms of HSK are more difficult than that of 1989, and the testing groups are higher achieving than before, this study makes the following proposal: the frequency estimation equipercentile and presmoothed PSE with optimal h value are better choices for small samples; the frequency estimation equipercentile methods, the chained equipercentile methods, PSE and CE with optimal h values work better for large samples.In this study, different equating criteria and statistic indexes are also discussed, and it is found that the comparisons based on different equating criteria might lead to different conclusions.