An Exploratory Study Of Assessing EFL Critical Reading Abilities: From Specification To Implementation

Critical reading (CR) is high-level reading which calls for reader's active involvement. Critical reading analyzes not just what texts say, but how they say it. Critical ability is regarded as the fourth dimension of literacy. One of the higher education objectives in developed countries is to raise students'critical awareness.Critical reading is playing increasing roles in modern EFL learning and teaching. High-level learners have ready access to authentic magazines, newspapers, journals, leaflets, etc. They turn to the Internet for news, knowledge and arguments. It is wise for them to read critically. Accordingly, there is a pressing need to assess critical reading ability (CRA).However, no specific test suits such a need. Therefore, this study aims to develop a critical reading test (CRT) intended for high-level EFL learners. The test is conceived as a communicative language test, a proficiency test, an integrative test, and a criterion-related test.The theoretical objective of the study is to identify and establish the construct of critical reading ability. The construct is explicated in virtue of three frameworks: a framework of critical reading, taxonomy of critical reading ability and an inventory of CRT focuses. The practical objectives are to design CRT specification and compile CRT prototype paper.Extensive literature review lays solid underpinnings for developing CRT. By integrating definitions of being critical, critical reading and critical reading ability, three frameworks are sketched. The critical reading framework highlights'reading at reader's stance','rhetoric-based reading','social-relevance reading'and'responding to texts.'The CRA taxonomy is composed of four levels: structure analysis level, rhetoric analysis level, social relevance level and holistic evaluation level. The CRT focus inventory has four components—analyzing paragraphs, discovering meaning, evaluating reading and responding to text. Each component embraces several facets, such as distinguishing among opposing viewpoints, explaining figurative language, assessing arguments, recognizing appeals and logical fallacies, and expressing a personal viewpoint. Specification is a crucial part of test construction. It is also the core product of this exploratory study. The specification is designed on the basis of CRA construct. It depicts deliberately essential elements of CRT, such as target test taker, purpose of the test, test focus, test structure, etc.A CRT prototype paper is compiled according to the CRT draft specification. The paper has five sections. Each section includes a text of a certain style, such as exposition, argumentation, advertisement, and opinion-editorial. Each text is followed by four or five questions respectively. The last section is intended for cross-text reading, which requires responding to two texts of the same topic but with opposite viewpoints. The tasks are designed in accordance with text genres. The CRT focuses are sampled in the test. Five task types are adopted in the paper to elicit CRA. They are MCQ, SAQ, matching, essay writing and summary writing.To check up the validation of CRT, a research is designed with six research questions as follows.1) Can CRA be elicited by the items devised according to the specification?2) Is it reasonable to divide CRA into four levels?3) How do materials entail CRA?4) What item types can effectively elicit CRA?5) Who and when is suitable for taking CRT?6) What will be the backwash of CRT?Altogether 229 students and 6 experts took part in the research. To triangulate the research methods, three main research instruments were employed—questionnaire, interview and CRT paper. Three questionnaires are designed to collect quantitative data concerning students'attitude to text genres, test focuses, and their evaluation on the CRT test paper. The interview questions aim to collect qualitative data, which may reveal underlying reasons for the quantitative data. Scores from the CRT paper administration are analyzed to offer statistical evidence.The pilot study was administered during the training session. 33 subjects sat in the lecture. They were offered a handout which included materials concerning critical reading. They were introduced to the definitions of CR, CRA and CRT first. Next they were taught with useful terms in evaluating text, such as appeals and logical fallacies in arguments. After practicing critical reading with a speech text, the subjects were required to complete the questionnaires and the CRT paper. The analysis on the pilot study led to minor modifications to the handout, the questionnaires and the test paper.In analyzing the following experimental data, multiple techniques are exploited. Descriptive analysis, cluster analysis, correlation analysis, T-test and ANOVA are employed to deal with the quantitative data in the questionnaires. Cronbach's Alpha value, factor analysis and scree plots are used to justify the reliability and validity of the questionnaires. Test score data are analyzed in terms of reliability analysis, correlation analysis, ANOVA test, covariate ANOVA test and T-test. The qualitative interview data are transcribed to provide further interpretation.The conclusion part of the dissertation elaborates on the necessity and the feasibility of CRT in the first place. Answers to the six research questions are then summarized respectively. Findings show that the CRT paper has well-accepted validity and its purpose of testing CRA is successfully fulfilled. The 4-level CRA taxonomy is justifiable owning to both theoretical basis and empirical support. Reading materials do entail CRA, and a certain kind of genre may call for CRA at a certain level. Essay writing and summary writing are equally accounted as the most effective types to elicit CRA, but matching is chosen as the most suitable method in CRT paper. CRT should not be intended only for senior English-majors but for high-level learners with critical reading experience. Finally, qualitative data disclose that CRT will have positive backwash on learning and teaching reading.How the CRT specification may be modified is also included in the conclusion part. A few research implications are provided. The contributions, the limitations and the future research orientations of the study are pointed out as well.