Comparing the efficiency and effectiveness of substance abuse treatment programs using a two -stage data envelopment analysis

The claim that substance abuse treatment programs with high overall client dropout rates operate at comparatively low organizational efficiency and effectiveness levels is examined. Using data from 40 treatment programs participating in the National Institute on Drug Abuse's Drug Abuse Treatment Outcome Studies (DATOS), program treatment efficiency and effectiveness frontiers were established to identify the "best organizationally performing" community-based treatment programs. Substance abuse treatment was modeled as a two-stage service delivery process and a two-stage data envelopment analysis (DEA) was conducted to estimate this model. The first stage DEA calculated the ratio of outputs (e.g. treatment services provided by programs) to inputs (e.g. programs resources), and the second stage DEA calculated the ratio of outcomes (e.g. reduced substance use, decreased unemployment, reduced illegal acts, etc.) to outputs. Outcomes were case-mix adjusted with a problem severity index that produced a percentage of client problems present in each program. Programs were classified into one of four performance groups: low efficiency/low effectiveness; high efficiency/low effectiveness; low efficiency/high effectiveness; and high efficiency/high effectiveness. Using these categories and DEA measurement scores under conditions of front-loading of treatment services and fixed resources, no consistent relationship between dropout rates (and its inverse, stay-rates) could be established. This methodology provides the capability of conducting large-scale comparative evaluations across many programs to determine both organizational efficiency and effectiveness and has the potential to produce important policy-relevant information. States and other regulatory entities could use this methodology to improve treatment practices and outcomes in public behavioral health systems.