A Comparative Study On Candidates’ Cognitive Processes When Completing IELTS,TOEFL IBT And CET-6 Reading Tests

The International English Language Test System(IELTS),Test of English as a Foreign Language Internet-based Test(TOEFL i BT)and College English Test Band 6(CET-6)are three English tests which are highly influential home and abroad.Although the three tests have differences on test purposes,test formats and score interpretations and use,they still have relatively strong comparability for high-stakes and large-scale characteristics,same candidature who are taking or will take higher education,and similar test contents with an academic focus.Validity and validation has long been argued as the core of language assessment studies.Cognitive validity refers to the extent to which candidates’ cognitive processes engaged in test context are similar to those in real-life language use situation which stresses the important role of test takers’ cognitive processes in validation procedure.However,traditional validation studies in language assessment field have relied heavily on test results while test takers’ cognitive processes are neglected to a larger extent.None of existing studies has endeavored to explore candidates’ cognitive processes engaged in large-scale and high-stakes testing context with a comparative analysis.To address the research gap,this study is conducted to investigate test takers’ cognitive processes comparatively when completing IELTS,TOEFL i BT and CET-6 reading tests with an innovative combination of eye-tracking and retrospective interview,hence demonstrating the cognitive validity of the three tests.This thesis coupled quantitative data(14 eye movement measures)and qualitative data(interview data with the eye-traces clips as stimulus on each item)of 10 participants with higher English proficiency so as to examine their cognitive processes comparatively during the test-taking experience with reference to the cognitive model of reading proposed by Khalifa and Weir(2009).And it is mainly conducted to address the following three research questions: 1)What cognitive processes do test takers employ when completing the three reading tests? 2)Are there any similarities and differences in cognitive processes elicited by the three reading tests? If yes,what are they? 3)To what extent and in what ways can the three reading tests succeed in eliciting a full range of cognitive processes expected in real-life reading context?This study revealed the following findings:Firstly,IELTS,TOEFL i BT and CET-6 reading tests elicited a variety of cognitive processing at word,sentence,paragraph and text levels including word recognition,lexical access,syntactic parsing,establishing propositional meaning,inferencing,building a mental model,creating a text-level representation(with an exception of IELTS reading test).Nevertheless,this study also found participants’ use of other processing which are not included in the cognitive model of reading such as guessing,analyzing item stem,comparing given options,using clues from other items and using topical knowledge when they responded to some items.Secondly,the three reading tests successfully evoked a full range of cognitive processes that are similar to those in real-life academic reading.Among these cognitive processes,establishing propositional meaning was frequently used by participants in all the three reading tests.It suggest the similarity on participants’ cognitive processes when engaged in three reading tests,namely,they activated plenty processing at sentence level.Meanwhile,their differences on cognitive processes are also found in this study.As a result of text input,item design and English proficiency,participants used more syntactic parsing and lexical access in CET-6 and TOEFL i BT reading tests respectively.And IELTS reading test elicited more use of inferencing at sentence level in this study.According to eye-tracking data,there are significant differences on participants’ eye-tracking metrics of fixation,saccade and regression on Ao Is of text processing,task processing and global processing.IELTS and TOEFL i BT reading tests appeared to impose a higher cognitive load on participants than CET-6 reading test.It closely corresponds to the interview result that IELTS and TOEFL i BT reading tests evoked more higher-level cognitive processing than CET-6 counterpart.Finally,this study provides valid evidences to demonstrate the cognitive validity of IELTS,TOEFL i BT and CET-6 reading tests.However,it is also shown that in some correctly-answered cases,participants also made use of other processing which might cause threats on test validity and fairness,indicating possible limitations in test design.To be specific,the gap-filling task of CET-6 reading test was more likely to test participants’ syntactic knowledge rather than their comprehension and use of words in text context.Depending on lexical categorization and option elimination,participants could come to the right answer rapidly in this study;in IELTS reading tests,there were two or even more items testing participants’ understanding of a specific part of text,and participants were able to answer correctly with the use of clues from previous items;some cases of TOEFL i BT reading test were limited to elicit participants’ use of their previous topical knowledge and lexical meaning without references to textual information.This study investigated participants’ cognitive processes and evaluated the cognitive validity of IELTS,TOEFL i BT and CET-6 reading subtests carefully through a mixed method.It is hoped to provide valuable suggestions for test design of the three reading tests,implications for learners’ reading development and insights for future related studies theoretically and methodologically.