| To date most researchers examining multidimensionality in English language proficiency tests have investigated tests with dichotomously scored items which typically measure linguistic features such as grammar and vocabulary. There has been very little dimensionality research conducted on polytomous data from tests of oral language proficiency, a construct no longer defined in terms of linguistic features but rather as the mastery of functional, sociolinguistic, discourse, and linguistic competence (Canale, 1983; Canale & Swain 1980) and as an interplay between (a) language ability, topical knowledge, and affective schemata and (b) the characteristics of the language use situation (Bachman & Palmer, 1996). The revised Speaking Proficiency English Assessment Kit (SPEAK) incorporated these new theoretical definitions of oral language proficiency as the premise for its test tasks and holistic scoring rubric.; This local validation study contributes to the construct validation of the SPEAK (Messick, 1989) by evaluating the dimensionality and parallelism of four versions of the test from a qualitative and a quantitative approach in an attempt to merge the pragmatic and the theoretical traditions of oral proficiency assessment.; Results from the substantive analysis, ANOVAs, GENOVA, exploratory factor analyses using Pearson and polychoric correlation matrices, and a hierarchical cluster analysis using PolyHAC support the conclusion that SPEAK Forms A, B, and C are statistically unidimensional and parallel. Results for Form D were conflicting. Form D had significantly higher test-level mean scores as well as numerous significantly higher mean scores on items intended to measure the same language function; however, its content, variance, and reliability are similar or identical to those for Forms A, B, and C. A dimensionality assessment using Poly-DIMTEST did indicate multidimensionality in Forms A, B, and D, but results are preliminary and merit further investigation.; In addition to arguing that the theoretical basis of the revised TSE/SPEAK inadequately serves the test's practical purposes, the author recommends the scoring rubric be changed from holistic to multitrait. It is hypothesized that the multidimensionality believed to exist in the construct "oral English proficiency" might be statistically determined through detailed analyses of rater behavior and examinee responses rather than through analyses of holistic test scores. |
