Validation Study Of The Computer-based College English Placement Test

The Computer-based College English Placement Test (CCEPT) in Hunan University is a new test that aims to develop a scientific placement system tailored to Hunan University to facilitate the academic success of students in the University, and to meet the requirements of establishing an international research university. The study is a comprehensive validation of CCEPT, and several suggestions are given to the improvement of the test.The study is firmly rooted in the modern paradigm of test validation. The author employs the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association & National Council on Measurement in Education, 1985) to build arguments. Based on the theory of language testing, particularly that of communicative language testing, this study focuses on the analysis of content validity, construct validity and face validity to evaluate the test to ensure that it reflects the English language performance of the test takers in the light of the test specifications. Quantitative and qualitative forms of data are gathered from test takers. All statistical analysis is performed using SPSS 13.0. Logical analysis is used to evaluate the adequacy with which the test content represents the content domain. Discrimination Index and Facility Value of items are calculated. Correlation study and factor analysis are conducted to test the construct validity. Students'responses to the questionnaire are analyzed to see the face validity of the test.Results show that the validity of CCEPT is convincing. CCEPT has relatively high construct validity. The correlations between each subtest with another subtest and with the total test are generally satisfactory. However correlation between Reading Comprehension and Writing Test is as low as +0.169 which could be due to the marking reliability of the Writing test. The factor analysis shows that the extracted two factors in the test can be named as"ability of listening and writing"and"ability of reading", and they can explain general test format of CCEPT. However, speaking test is available only to those whose final written score is 050 or above. As a communicative language test, it has not tested all the aspects of English language competence.Analyses of the content validity also show generally satisfactory results. The Facility Value of Listening Comprehension is 0.464, around the ideal value 0.5. However the listening component focuses mainly on specific details rather than on function of utterance, which fails to meet the specifications adequately. Items tested in Reading Comprehension reflect a harmonious balance among different micro-level skills. The overall Facility Value is 0.569 with the lowest 0.465 and the highest 0.708. The discrimination index of this component is 0.27 (around the standard value 0.3). The writing test topics are close enough to everyday life to encourage students to write on. However, study of inter-rater consistency evaluation shows the Scorer Reliability is rather low in the Writing Test.Responses to questionnaires concerning computer familiarity and computer anxiety, general impression of the test, fairness of the test, possible backwash effect, and skills tested in the test and so on, show that most of the test takers find CCEPT satisfactory and they treated it seriously.In the end, the thesis puts forward several suggestions: CCEPT should enlarge item bank in order to ensure the quality and stability of the test. Mode effect should be decreased to the lowest degree. In addition to the analysis of validity, the thesis further points out that CCEPT should improve its reliability, especially scorer reliability in writing scoring.The findings will have important practical implications for implementing a placement English test on computers and they will also be applicable in other academic contexts.