A Study On Content Validity Of Deep Reading Of HSK-5

Bachman(1990)proposes that content validity is reflected in the consistency between test content and test purpose,and also in the representativeness of test content.At present,a theoretical framework has not been put forward in the domestic studies,and the studies of language tests are focusing on the test of English,Japanese,German and Russian while few studies involve the test of Chinese.As an international Chinese language test,Hanyu Shuiping Kaoshi is in an attempt to measure the ability of non-native Chinese learners who use Chinese in their study,work and life.For Chinese learners with different language proficiency,HSK has been divided into HSK-1 to HSK-6,among which HSK-5 is the threshold for the overseas students to apply for scholarships,which has attracted more and more attention from the overseas students.However,the studies on HSK at home and abroad tend to explore its development prospects and the reform of examination form but pay little attention to the content validity of the test and the evaluation of the candidates.Given this,the study aims to explore the content validity of deep reading of HSK-5 by combining the theoretical study with the empirical study through analyzing the consistency between the test content and the test purpose,illustrating the representativeness of the test content,and surveying the washback effect of HSK-5 on learning.First,based on Bachman and Palmer's(1999)language task characteristic framework,a revised framework is established to verify the consistency between test content and test purpose in terms of input characteristics and expected response.Second,a reading test is organized to collect the scores of 50 overseas students,the scores are used to draw a Rasch model to obtain the fitting degree of the item's difficulty level and students' proficiency level by the help of Ministep,the fitting degree reflects the representativeness of the test content(Beglar,2010).Third,questionnaires are distributed to the candidates to obtain the washback effect of deep reading of HSK-5 on students' learning attitude,purpose,materials,content and methods.The results of this study are as follows:(1)The consistency between test content and test purpose is analyzed according to Bachman's(1999)language task Characteristic framework.In terms of the characteristic of the input,the average length of the article is 390 words,which is between 250 and 570 words specified in the syllabus;the sentence patterns of the article are 2.4% for performative sentences,11.8% for simple sentences,and 71.8% for complex sentences,which meet the requirements of sentence diversity;the genres in the articles are 44% for narrations,52% for exposition,4% for argumentation and no application,so the genres meet the requirements;the topics in the articles are 36% for daily life,4% for literature,12% for nature,16% for economics;4% for occupation and no culture-related topic,so the topics are in line with the requirements basically;the actual average reading speed is 98 words/minute,which is lower than the 128 words/minute specified in the syllabus.In terms of the expected response,the reading skills examined in the articles(24%for understanding the main idea,74%for interpreting the details,2%for understanding the background information)are consistent with the requirements;the expected response types are all selective outputs,that is,the multiple-choice questions,which do not meet the requirements of the syllabus for the diversity of test forms.On the whole,the test content is well consistent with the test purpose.(2)The representativeness of the test content is analyzed based on three aspects of Beglar's(2010)theory.First,the Rasch model reflects that the distribution of research subject's proficiency is corresponded to the distribution of the difficulty of the test items,indicating that the test items are well represented,and there is no ceiling effect and floor effect,but there are 3 test items(Q77,Q72,Q75)whose difficulty is above the average,3 test items(Q71,Q73,Q82)whose difficulty is below the average and 4 test items(Q88,Q90,Q75,Q73)which have no corresponding proficiency of the research subjects,these are invalid items.Second,the separation coefficient of the test item is 3.06,and the difficulty span of the test item reaches 4.41,indicating that the difficulty of deep reading comprehension can distinguish students' reading proficiency(Beglar,2010).Third,the difficulty gap of most questions is evenly distributed except for invalid items,indicating that the existing items can diagnose the Chinese learner's reading proficiency.(3)The washback effect lies in six aspects of research subjects' learning based on the results of the questionnaires.87.5% of the research subjects hold that the test stimulates their motivation and interest on learning attitude;72.92% of the research subjects realize that the preparation test is for future study and work on learning objectives;75% of the research subjects present that the test prompts them to train their reading skills on learning methods;72.83% of the research subjects believe that the test forces them to learn Chinese reading skills and strategies on learning content;56.25% of the research subjects proactively extend the time of learning Chinese before the exam on time allocation;80% of the research subjects express their willingness to purchase reading materials for the test on learning materials.Therefore,in the design of the upcoming HSK,the designer should enrich the inspection form of the deep reading,the number of argumentation and application should be appropriately increased;besides,although the test content is relatively representative,the test designers need to improve the quality of the test in accordance with the actual proficiency of students;in addition,teachers should make full use of the washback effect of the test to help Chinese learners adjust their learning methods and content,and learners should accumulate Chinese cultural knowledge from various aspects to improve communication skills.