Towards A Framework Of Diagnostic Score Reporting Of College Students’ English Reading Ability

The traditional score reporting of language tests usually cannot provide enough diagnostic information for remedial teaching and learning,for they usually serve a range of purposes,such as selection,placement,accountability,or issuing certificates.Cognitive Diagnostic Assessment(CDA)is a relatively new assessment approach that is aimed at providing formative diagnostic feedback through a fine-grained score reporting of learners’ cognitive skill and strategy mastery profiles.However,Most CDA researchers focus their attention on the Q-matrix construction and Cognitive Diagnostic Models(CDMs),as a result of which less emphasis is laid on the score report development,though some researchers have mentioned the process of designing the diagnostic score report.The goal of the present study is to construct a theoretical framework of diagnostic score reporting of college students’ English reading ability by linking the theory of CDA with the practice of score reporting.In order to achieve this goal,the present study tries to answer the following three questions:1)What are the cognitive attributes that affect college students’ English reading ability?2)How can the existing reading comprehension test be retrofitted into a CDA to provide diagnostic information about college students’ English reading ability?3)How can the diagnostic score reports of college students’ English reading ability be developed and validated?To answer the research questions above,a set of reading comprehension test was retrofitted into a CDA and both students and teachers’ diagnostic score reports were developed and validated.First,relevant documents and studies were reviewed to identify the cognitive reading attributes required in the teaching syllabus,test specifications and scales of language proficiency.Then,a set of reading comprehension test was analyzed and the identified cognitive reading attributes were empirically validated through students’ think-aloud verbal protocols and expert panel’s judgements.Five students participated in the think-aloud verbal protocols and 5 experienced teachers participated in the expert panel’s judgments.Their major task was to extract the cognitive reading attributes assessed in the reading comprehension test.Second,2225 students from one provincial key normal university provided test item response data.These data were analyzed with the GDINA package in R Studio through model estimation,model-fit evaluation,Q-matrix validation and attribute prevalence to provide diagnostic information.In addition,the distractors were also analyzed for mining more diagnostic information.Finally,in the application phase,the retrofitted diagnostic reading assessment was applied in one non-English major class with 54 students and 1 English teacher.After finishing the cognitive diagnostic assessment,they got a diagnostic score report,which inform the students of their mastery of reading attributes.In order to collect evidence to validate the diagnostic score report,the 54 students participated in the selfassessment and questionnaire survey.Based on the studies above,a framework of diagnostic score reporting of college students’ English reading ability was proposed,which consists of three components: 1)retrofitting the existing reading tests into CDAs;2)developing and validating the diagnostic score reports;and 3)remedial teaching and learning.In addition,they are an on-going improving and validation process and it needs to be further pointed out that the diagnostic remedial teaching and learning activity is an important component of CDA,because the ultimate purpose of the diagnostic score report is to make teachers’ teaching and students’ learning more targeted,so as to promote students’ learning.Theoretically,the present study has improved the framework of diagnostic score reporting of college students’ English reading ability.Practically,the evidence of students’ and teachers’ perception of the diagnostic score reports has been collected to investigate the validity of the diagnostic score reports.