An investigation of operationalized constructs of second language listening tests

This study investigated the operationalized constructs of listening measured by the Test of English as a Foreign Language (TOEFL). The first phase of this research comprised theoretical analysis to define the listening abilities that constituted the operationalized constructs. The second phase encompassed empirical analyses to validate the effectiveness of the listening abilities and to test a hypothesis about the operationalized listening construct.;The first research question aimed to define the listening abilities measured by TOEFL tests by analyzing three aspects: language knowledge unique to listening, cognitive and comprehension processes implicated by items tasks, and stimulus features. The phonological characteristics of text from the test were transcribed to identify the language knowledge unique to listening. The cognitive processes were identified following Bloom's taxonomy and comprehension stages. Stimulus features were examined for text redundancy, propositional density, and characteristics of item alternatives. The analysis produced thirteen listening ability types measured by TOEFL listening tests.;The second research question aimed to validate the effectiveness of those thirteen listening abilities as primary variables for analyzing the internal structure of operationalized listening constructs. The TOEFL listening test was administered to approximately 450 Korean English language learners in Korea. The exploratory factor analysis demonstrated that those listening abilities were effective in extracting the meaningful number of factors and factor patterns.;Finally, the hypothesis posited two factors of operationalized listening constructs: one was related to language knowledge and the other to cognitive comprehension factors. These two factors represented the theoretical construct and test method facets, respectively. The confirmatory factor analysis successfully verified a model that substantiated the hypothesis. The model displayed moderate proportions of variances accounted for by the two factors and covariance of certain measurement errors. The model implied that the operationalized listening constructs involved sources other than language and cognitive comprehension, supporting the complex nature of listening. Additionally, the nature of the two factors implied a continuum of operationalized listening constructs. Each end of the continuum may represent theoretical construct and test method effects. The notion of a continuum was discussed in relation to the model of language ability, and top-down and bottom-up processing.