| Assigning education services targeting certain subpopulations often involves eligibility cutoff points. This study discusses three different design and analysis strategies that can potentially help approximate experiments in multi-site studies involving eligibility cutoffs. I discuss key issues that should be considered in conducting studies with eligibility cutoff points, which help identify important limitations and strengths of the suggested designs. For illustrative purposes, I examined the effectiveness of a reading intervention called Voyager Passport on student performance on a statewide assessment.;First, this study addresses theoretical and practical issues of regression discontinuity (RD) designs from a potential outcomes perspective. I focus on analytical challenges in both "sharp" and "fuzzy" RD designs using the Voyager data. In particular, I stress various challenges that arise in the use of instrumental variables (IV) methods in the context of fuzzy RD designs. Given the challenges connected with RD designs, I present two alternatives termed a between-school design and a within-school cohort control design for forming a comparison group comprised of individuals who are highly similar to the individuals in the treatment group in multi-site settings. The underlying idea of the alternative approaches is that the eligible students in the control group - who are drawn from schools assigned to the control conditions that just missed the school-level eligibility cut off, or from prior-year cohorts in treatment schools - can provide sensible counterfactuals for individuals in the treatment group. In the Voyager example, there are some students who were assigned to Voyager who did not participate in the program. Thus one aspect of this study is to outline strategies for handling non-compliance in the context of fuzzy RD designs, between-school designs and within-school cohort designs.;This study, highlights that decisions on which design might be preferable in a certain situation should be made based on an array of factors, including careful consideration of the target populations, the treatment effects of interest, the degree of the association between the eligibility cutoff and actual participation, and sample sizes around the eligibility cutoff points at both the student and school levels. |
