Comparing Reading Subtests In China’s Secondary English Test And TOEFL Junior

For a test, validity is one of the important factors determining its quality. Therefore, the quality of a given language test depends on a series of scientific and reasonable validation. At present, domestic validity studies in the field of language testing have focused much more on CET4, CET6, TEM4 and TEM8, but less on the overall validation of China’s Secondary English Test, which is also one of the large-scaled and high-stake tests in China. The present study attempts to explore the similarities and differences of the reading comprehension subtests in China’s Secondary English Test and TOEFL Junior on the basis of Weir’s validation framework, which is expected to enrich the research literature about China’s Secondary English Test in the field of language testing on the one hand; on the other hand, to provide some meaningful references for the improvement of English test as well as English teaching for junior high school students, and promote language testing in China to integrate with international standards at an early date.“An evidence-based approach” put forward by Weir(2005) is employed as the theoretical core of this study to carry out the exploratory validity comparison between the reading comprehension subtests in China’s Secondary English Test and TOEFL Junior so as to address the following research questions:(1) What are the physical, psychological and experiential characteristics of the test-takers involved in the test taking process? To what extent are the cognitive processes and interactional activity of the test takers authentic during the process of completing the two reading subtests?(2) To what extent are the testing conditions and operations of the two reading subtests fair to all the test-takers?(3) To what extent are the scores of the two reading subtests reliable?(4) How are the two reading subtests interrelated?At first, an entire authentic test paper of China’s Secondary English Test in 2015(Chongqing) and a reading comprehension subtest from an authentic TOEFL Junior were respectively taken as the instruments of this study. Then, 97 grade-three students in a junior high school in Chongqing were recruited as the subjects of the study. On the basis of the validation framework by Weir(2005), the validity evidence of the two sample reading tests was gathered from the following aspects:(1) prior to the test administration, the physical conditions of the classrooms for testing were checked and examined to ensure the context validity of the two tests.(2) At the end of test performance for each sample test, 3 randomly selected students were asked to do immediate introspection experiment and recall their cognitive processes and interactional activities during the process of completing the two reading comprehension subtests to validate the theory-based validity.(3) After the test administration of the two sample tests, all the participants were required to finish a questionnaire survey not only to collect evidence of their characteristics, like physical, psychological and experiential characteristics, but also to have a deep understanding of their viewpoints of the content of the two sample tests, such as task setting, task demands as well as test setting and administration, all of which is expected to offer further evidence for context validity and theory-based validity.(4) Finally, of all the participants in this study, 77 were considered to provide valid data, and all their test scores in the two reading subtests were used to examine the scoring validity and criterion-related validity. To be specific, with the help of the statistic tool SPSS 19.0, index of difficulty, discrimination and reliability of the both reading comprehension were calculated and compared. Meanwhile, the correlation between them was also investigated by descriptive statistics analysis and inferential statistics analysis.In the process of comparing and analyzing the validity of reading comprehension in China’s Secondary English Test and TOEFL Junior with reference to Weir’s evidence-based approach, the findings of the current study are as follows. The data from the questionnaire survey manifests that in the test taking process of the two reading subtests some test takers feel uncomfortable physically and nervous psychologically. The number of the participants who are familiar with the reading comprehension in China’s Secondary English Test is larger than that in TOEFL Junior. Both of the reading comprehension subtests measure the similar reading skills, but the immediate introspection experiment proves that the test-takers’ cognitive processes in China’s Secondary English Test is more authentic than that in TOEFL Junior. As for the context validity, the physical conditions for both of the sample subtests are fair to all the subjects. However, for the operation conditions, the reading comprehension in China’s Secondary English Test is fairer than that in TOEFL Junior for its own difficulty and students’ unfamiliarity with it. Item analysis reveals that compared with the items in China’s Secondary English Test, those in TOEFL Junior reading subtest have higher difficulty and lower discrimination. The Equal-length Spearman-Brown reliability coefficients for the reading subtests in China’s Secondary English Test and TOEFL Junior are 0.676 and 0.546 respectively. Therefore, the former is more reliable and stabler than the latter. In addition, the Pearson Product-Moment Correlation of the two versions of reading subtests is 0.583, which indicates that they are interrelated with each other significantly, but still not ideal.