A Rasch-based Validation Study Of The Business English Vocabulary Size Test

The validity examination of vocabulary size test is the prerequisite for designing a scientific vocabulary measurement.At present,the validity investigation of general English vocabulary size test has been relatively common while the research on business English vocabulary size test is few.Thus,this study aims at providing preliminary validity evidence for a self-developed business English vocabulary size test(BEVST)in the hope of constructing a valid and accurate measurement and exploring effective validity examination methods for vocabulary size test.This study employed a series of analytical methods of the Rasch model to explore the validity of BEVST.Validation analyses mainly focus on the three aspects of the Rasch model.First,the item difficulty is examined by the Wright map to have a general understanding of the whole test's difficulty level.Second,the item fitness is explored through fit order table and bubble chart to figure out whether test-takers' responses to items are corresponding to the model's expectations.Finally,the single dimension exploration by Principal Component Analysis(PCA)of item residuals is carried out to examine whether the test is a unidimensional construct.The overall research objective is decomposed into three closely-related research questions: 1)Does the item difficulty hierarchy spread sufficiently to measure learners at different ability levels? 2)To what extent do the items display fitness to the Rasch model? 3)Is there a single dimension underlying the BEVST measurement? In order to address the above research questions,the BEVST was conducted twice.All the participants in the two tests are from Business English Program of Guangdong University of Foreign Studies in China.The collected data are analyzed based on the three aspects of the Rasch model using the software Winsteps 3.72.3.In the first study,the participants are 63 undergraduates(17 freshmen,19 sophomores,15 juniors,and 12 seniors).After the Rasch model validation analysis,the results indicate that: 1)the spread of item difficulty hierarchy is not even or continuous,namely,some participants cannot be measured with items at corresponding difficulty level;2)there are four items which display misfit to the Rasch model,indicating the observed results are misaligned with the expected;and 3)the multidimensionality investigation shows that the test is not unidimensional.After some revision and modifications of those flawed items,a second study was conducted.The participants are 69 undergraduates(18 freshmen,19 sophomores,12 juniors,and 20 seniors).The Rasch-based validation results reveal that 1)the item difficulty hierarchy spreads sufficiently and evenly to be able to measure learners at a wide range of proficiency levels,and the mean value of the test's difficulty is close to that of persons' ability level,indicating that the revised test is suitable for the majority of learners.2)Except for item 10,the majority of the items display good fitness to the Rasch model and function as what the measure model expected.3)The items measure along the same empirical dimension and the test is unidimensional enough for practical purposes.In contrast with the first study,more positive validation results are achieved in the second study.The main reasons for this improvement are that some comparatively easy items are replaced with more difficult ones and the distractors of misfitting items are modified.This study lays a sound foundation for the business English vocabulary measurement and enriches the relevant vocabulary size studies.Besides,it is expected that the validation investigation analysis will lead to a better understanding of or provide references for the validation examination of vocabulary size measurement.Last but not least,the BEVST test can provide teachers and researchers with a valid measurement that assesses learners' receptive vocabulary size,which in turn would be conducive to business English vocabulary teaching.