With the growing number of impact evaluations worldwide, the question of how to apply this evidence in policymaking processes has arguably become the main challenge for evidence-based policymaking. How can policymakers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the policy's design and implementation? This paper introduces a simple and flexible framework to address these questions of external validity and policy adaptation. Failures of external validity arise from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented. I discuss the strengths and limitations of existing approaches to external validity, and develop a method of "mechanism mapping" that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy, there is a fundamental informational trade-off between the strength and relevance of evidence on the policy from other contexts and the policymaker’s knowledge of the local context. This trade-off can guide policymakers’ judgments about whether policies should be copied exactly from elsewhere, adapted, or invented anew.
All are welcome at this CEBI seminar; please arrive early to guarantee a seat.
