Are Parallel Tests Really Parallel?

Because of the great number of candidates, the Computerized Oral Test of the National Matriculation English Test (NMET-COT) in Guangdong province entails multiple administrations. To prevent leakage of test content between administrations, for any one test several test papers are employed, which are called parallel forms of the same test. In the 2007 test, six parallel forms (Form A to Form F) were used. One potential problem of using parallel forms is that the different forms might not be comparable in difficult level, reliability, and so forth. In that case, the fairness of the test is threatened. So it is highly desirable to carry out studies to address the issue of parallelness of different forms used in one test. The present study investigated the equivalence between Form B and Form C of the NMET-COT 2007 with the application of the many-facet Rasch model (MFRM).The participants in the study are 45 Senior III students from Zhixin High School in Gaungzhou who were asked by the present researcher to take a COT test using both Form B and Form C of the NMET-COT 2007 in class. Before the test, the researcher trained them with a sample test in 2004 to make them familiar with the format. After the test, all the participants were required to complete a questionnaire about the test content, which provided further data for analysis.The results showed that the two forms were not statistically parallel either on the form level (overall) or on the individual task level. Form C was more difficult than Form B. Moreover, sources of variation to account for the variable difficulty of tasks identified by the questionnaire were: the number of new words and long sentences, and the delivery speed in Part A of the test; familiarity with the topic and the degree of informativeness for Part B and Part C. Besides, contrary to common belief, too much information in Part C negatively affected students' performance because the time for them to complete the task was limited.The findings of the study provided feedback on the comparability of the two forms of the NMET-COT 2007, which was valuable information for future test development. Further, the study proved that MFRM is a suitable method to employ for research into oral tests, which could be considered by other researchers and examination board that are responsible for large-scale tests.