The Validation Of A Language Performance Test With A Multi-dimensional Construct

Language performance tests based on a multi-dimensional construct(MDC)have been a new practice to meet the trends of fostering international talents who are not only proficient in language,but also are equipped with global vision,critical thinking ability(CT)and Intercultural Communicative Competence(ICC).However,considering the subjective nature of language performance assessment and the complexity of the construct,the validation of such a test would be of primary concern,especially the issue of whether the MDC is effectively realized in rating.Whereas,few studies has been conducted on this issue.This study aims to validate such a test from the rating perspective under the guidance of Bachman&Palmer's(2010)Assessment Use Argument(AUA).A mixed-method design has been employed to collect evidence both from the rating process and the rating results to answer the center issue——whether the MDC is effectively realized in the rating.Qualitatively,two raters' think aloud verbal protocols when each rating 2 writing samples were recorded,transcribed and coded to figure out whether the raters pay attention to all aspects of the construct and how their attention distribute.Quantitatively,this study identifies whether the raters maintain a uniform level of severity across different dimensions of the construct,and how well they can spread test takers into different levels on each dimension by analyzing 8 raters' rating results of 30 test takers'writings via Many-facet Rasch Model(MFRM).Results show that(1)in the rating process,raters distributed different weight of attention to each aspect of the construct,with the most focus on test takers' CT,then language ability,and finally ICC.The decisive factor for the unbalanced attention given to the concerned abilities is whether the indicators of certain aspect of construct are explicitly presented in the rating criterion;(2)as manifested in the rating results,two aspects of findings have been yielded.First,raters were most severe when discriminating test takers' CT and ICC,and most lenient on their language ability,which indicates that the MDC has been realized in the rating.However,raters exhibited many problems in discriminating the concerned abilities,especially CT.This may be due to the quality of the rating criterion,or raters' insufficient understanding of the rating criterion.Several theoretical and practical implications have been put forward in this study.