Computer-or Paper-Based Test: Does Delivery Mode Affect Reading Test Performance And Test-Taking Strategies?

This paper reports on an empirical study which set out to find out the effect of computer delivery on reading test performance and test-taking strategy use by comparing two different modes of presenting reading comprehension tests: computer-based and paper-based. The test material was adapted from the TOEFL reading sections. The paper-based test was a shortened version of the traditional TOEFL reading test, with the same test instructions. It consists of 30 multiple-choice question items on 4 passages of 300-350 words. The computer-based version was a web page copy of the paper-based one, imitating the interface of the usual tests in the students' New Horizon College English computer-assistant program.One hundred and eighty-one second-year non-English majors participated in the study. They were two homogeneous groups in terms of their English proficiency, which was verified by their performance on PETS-3 (The Public English Test System Level 3). One group took the computer-based test and the other the paper-based one. The two tests were administered at the same time and for the same period of test time. At the end of the tests, the students were required to complete a questionnaire about their attitudes toward computer-based language testing and test-taking strategy use in computer- and paper-based language tests. Six students were interviewed and encouraged to reflect on how they took the tests.It is found that their performance in both tests demonstrates no statistically significant difference. In other words, computer delivery mode has no effects on the students' performance in reading comprehension tests, which runs contrary to the students' expectations. The reason for their mistaken belief probably lies in the fact that they do not get accustomed to taking tests on the computer. Furthermore, test-taking strategies in taking computer- and paper-based reading comprehension tests are also probed into. The finding is that the students do develop new strategies different from those taken in the paper-based reading comprehension test, such as highlighting the concerned portions, copying the passages into a Word document.The study is enlightening in that it offers some insights to teachers and learners engaged in tests presented on computers about the influence of delivery mode on examinees' test performance and use of test-taking strategies. It also provides fresh evidence that computer delivery mode does not have effects on test-takers' test performance but affects their use of test-taking strategies in taking reading comprehension tests on computers.