Skill Integration In Language Assessment-a Comparative Study Of Test Takers’Cognitive Processing In Pearson Test Of English-academic And Internet-based College English Test Band-6

In an attempt to deepen the understanding of independent and integrated tasksand gain insights into score interpretation from the perspective of cognitiveprocessing, a comparative study of Pearson Test of English-Academic(PTE-Academic), a computer-based language test predominantly composed ofintegrated tasks, and Internet-Based College English Test Band-6(IB CET-6), whichmainly contains independent tasks, was conducted. This thesis reports the findingswith respect to test takers’ cognitive processes and use of metacognitive strategies inintegrated and independent tasks. The following two questions were addressed in thethesis:1) How and to what extent can test takers’ cognitive processes and use ofmetacognitive strategies differ in integrated and independent tasks?2) How and towhat extent do high and low scorers perform differently in integrated and independenttasks in terms of their cognitive processing?43college students from Shanghai and Beijing volunteered to sit forPTE-Academic and IB CET-6within a time span of one month. Survey questionnaireswere developed on the basis of theory-based validity (cognitive validity) of thesocio-cognitive framework for validating language tests (Weir,2005). Thesequestionnaires were delivered after each test to elicit test takers’ immediate post-testreflection on their cognitive processing in completing listening, speaking, reading,and writing tasks, as well as their executive resources of language knowledge andinternal/external content knowledge of the tasks.Paired-sample t-tests of the survey data elicited19items with significantdifferences (p<.05) and large effect sizes (2=.14~.68) between the two tests,indicating that different executive processes and strategies were adopted in the twotests. Results seem to suggest that independent tasks relate more to test takers’internal content knowledge while integrated tasks tend to associate more with themanipulation of inputs.One-way ANOVA analyses identified25items with significant differences(p<.05) between the high-and low-scorer groups in IB CET-6and PTE-Academic(2=.20~.63), suggesting that the high scorers were more skilled in manipulatingsource materials and using metacognitive strategies than the low scorers. A higherscore in the independent tasks, accordingly, may indicate a stronger capability ofoperating executive resources including internal content knowledge and languageknowledge, conducting executive processes such as goal-setting and time management, as well as using appropriate global-level strategies especially in expeditious reading.Research results of the integrated tasks, on the other hand, reveal that high scorers canbe relatively skillful and effective in carrying out integrative operation ofinformation/external content knowledge and performing revising activities insummary writing tasks.It is hoped that the present study can contribute to the understanding of skillintegration in language assessment to some extent. Theoretically, the current researchdemonstrates the feasibility to apply theory-based validity (cognitive validity) to thestudy of cognitive processing in language assessment. Based on the theory-basedvalidity, a questionnaire that is applicable to relevant studies is developed. On thepractical side, the identified differences between independent and integrated tasksmay enable test users to gain more insights about test takers’ different ways ofprocessing tasks as well as the underlying meaning of test scores.