Bayesian models for stable subgroup treatment effects: an analysis of six graduation studies

About this event

Broadly successful interventions such as the Graduation program may still exhibit treatment effect heterogeneity across subgroups and across study contexts. Yet effectively using estimated subgroup effects to inform targeting of future interventions requires that these effects be reasonably stable across contexts. We develop a set of Bayesian hierarchical models to capture the extent of heterogeneity in subgroup effects across the six flagship Graduation programs. The models address the inherent power challenges of subgroup analysis by implementing an adaptive Ridge-style regularization that strengthens the penalty when isolated large estimated effects are not mirrored across sites, reducing false positives and type-M errors. We find evidence that larger and female-headed households generally benefit more, while households with pre-existing assets benefit less. However, the differences implied by subgroup effects are substantially smaller than the differences in average effects across settings. Even under oracle knowledge, targeting particular contexts is likely to be more effective than targeting particular households.