| Teachers and test developers in Japanese university language programs have hitherto taken little notice of the impact of the results of placement testing. The primary purpose of this study was to revise the 2002 in-house placement test used in a Japanese university foreign language program in order to more precisely measure and classify 5,500 incoming students' proficiency levels. Following Messick's theoretical view of validity as a unitary concept, a framework for the systematic validation of the placement test was created. The revisions that were made to the 2002 test were based on this framework. Through the validation procedures, more plausible evidence was provided to support test score interpretations and uses for the revised 2003 test.;Six approaches for determining the validity of both the original 2002 and the revised 2003 tests were conducted. Descriptive statistics, a three-parameter IRT analysis, and phi lambda dependability index indicated that the 2003 test more accurately measured and classified the incoming students, especially those students whose scores were around the higher cut-point. The three-parameter IRT analysis also indicated that the number of problematic items on the revised test decreased from 31 to 6, and the test information curve at the average ability level on the revised test increased from 6.0 to 7.0 points. The dimensionality of the internal structure of the tests was investigated through correlation and PCA analyses. They indicated that the Listening subtest and its subsections serve a useful purpose because they tap a domain of language proficiency that is distinct from the Grammar and Reading subsections on the test. Finally, questionnaires from students and teachers showed that teachers' attitudes toward placement decisions were more positive than the students'. Both teachers and students who were in classes around the higher cut-point showed more positive attitudes toward placement decisions than those in classes around the lower cut-point. |
