Measurement approaches in the development of functional ability level language tests: Norm-referenced, criterion-referenced, and item response theory decisions

This dissertation investigates the contributions of norm-referenced, criterion-referenced and item response theory in the development of tests designed to measure functional language ability. Such tests attempt to identify real-world tasks of linguistic interest in order to report what learners can and cannot do. Development of the tests and interpretation of results thus present several measurement problems associated with item analysis, decision consistency and reliability.;The study involved analysis of the test results of 830 subjects from Japan, Mexico and Saudi Arabia. The instruments were two multiple-choice forms of a pilot version of the General Test of English Language Proficiency developed by National Education Corporation, International and TENEC International. These pilot versions contained three proficiency levels on each form. Functional tasks were distributed in terms of hypothesized difficulty across the three proficiency levels.;The results of the study indicate that each approach contributes some information to test decisions. Item response theory is useful in initial item analysis and refinement of the tasks and test levels. Criterion-referenced measurement indices of item discrimination provide information when levels are adequately specified. Additionally, the study indicates that functional tasks frequently have a band of difficulties which overlap those of tasks designated as being at a higher or lower level. Further, the study indicates the need for statistical analysis to inform decisions made by content experts in the construction of the tests.