Does program quality matter? A meta-analysis of select bilingual education studies

This meta-analysis was a review of previous reviews of bilingual education. The purpose of this study was to refine the current accumulated knowledge base about language acquisition theory, bilingual education, and best instructional practices for linguistically and culturally diverse students. For the six narrative and vote count reviews, and three quantitative reviews reviewed, bilingual education was found to have generally positive outcomes, ranging from extremely weak to strong (Cziko, 1991; Greene, 1998; Okada et al., 1982; Troike, 1978; Willig, 1985; Zappert & Cruz, 1977;). These nine reviews primarily focused on the quality of research design, rather than on program quality, in relation to student outcomes.; In this meta-analysis, both the quality of the bilingual education program as well as the quality of research design were examined. No review to date has examined the relationship of program quality (as informed by language acquisition theory) and student outcomes, using meta-analysis.; In selecting the 10 primary studies reviewed in this meta-analysis, an effort was made to include studies from various sources, such as dissertations and published studies, given that published studies may favor statistically significant outcomes (Rosenthal, 1991; Wolf, 1986). For the 12 cohorts included in the analysis, the average effect size was .35, and .31 for the one cohort with strong bilingual program and sound methodology. Moreover, average effect sizes were consistently positive for each bilingual program quality category analyzed (strong, weak, and undefined).; Fail-Safe N calculations indicated that 408 studies of small or negative effect sizes would be needed in order to bring the overall average d = .35 down to d = .01. This is impressive, given that only ten primary studies were included in this meta-analysis.; This meta-analysis also demonstrated the utility of one definition of bilingual education, that posited by Krashen (1996). Specifically, using Krashen's three components of comprehensible input in English, primary language literacy, and primary language subject matter instruction to identify program quality, allowed for a focused review of both program efficacy and research efficacy.