| The present study is conducted to investigate the effect of test methods on the construct measured by two grammar tests and on test-takers at lower and higher proficiency levels by comparing test-takers' grammatical performance including their test scores, individual responses and test-taking processes.One of the two test methods involved in this research was the gap-filling grammar test(the G/F test) adopted from 2007 NMET A(Guangdong version) and the other test method was the multiple-choice grammar test(the M/C test) with the distractors coming from test-takers' top three incorrect responses in the G/F test.Another gap-filling grammar test adopted from 2008 NMETA(Guangdong version)was used as a control test. 164 senior two students from one high school in Guangzhou participated in this research, including two experiments. The role of the control test was to assure that participants didn't enhance their scores through having already seen the item stems in Experiment One and to divide participants into lower and higher group. Participants first took the G/F test and the control test, then four weeks later they were required to take the M/C test and also the control test. 12 of them from lower and higher group were interviewed about their test-taking processes in the G/F test and the M/C test immediately after they finished each experiment.Both quantitative and qualitative analysis were conducted. For quantitative analysis,test-takers' scores and individual responses were directly compared and for qualitative analysis, their test-taking processes were examined and compared, with the focus on grammatical knowledge and strategies used in taking the G/F test and the M/C test.It was found that test-takers' scores in these two types of tests were significantly different based on Matched Pairs T-test results and their responses had changed between two tests in about 45% of the overall items answered. Examination oftest-taking processes found that in the G/F test, test-takers needed to activate their grammatical knowledge to reach an answers that were both grammatically accurate and contextually appropriate and test-takers used more strategies, indicating that these two test formats involved different test-taking processes. Therefore, it was concluded that test methods affected the construct measured by the G/F test and the M/C test and these two tests did not measure the same construct. The results from two-way repeated measures ANOVA showed that there was significant difference of scores in these two tests and their responses had changed between the two tests in about 54.9%and 34.6% of the overall items answered respectively. In addition, for higher group,more difference was found in their use of cohesive knowledge in two tests while for lower group, more difference was found in their use of morphosyntactic knowledge in two tests. With regard to strategy use, for higher group, more difference was found in their use of associating, monitoring and guessing in two tests, while for lower group,more difference was found in their use of associating and translating in two tests. A conclusion was drawn that test formats not only affected grammatical performance of test-takers at lowerand higher proficiency levels, but also to varying degrees.On the basis of the above findings, it was found that the G/F test and the M/C test did not measure the same construct, the G/F format better tested test-takers' ability to use grammar in context and that the G/F test better measured higher-level grammatical ability. The findings also revealed that lower group and higher group were affected by test method effect to different extent, but which group was more affected would need to be further investigated. |
PDF Full Text Request