Theory of change and impact policy evaluation in cross-national settings

Project outline

 

The research in international in conception, design and execution with the large majority of the work is being undertaken by local academics working to evaluate locally developed policies in three countries: Uganda, India and China. The substantive core of the project is to establish whether shame associated with poverty reduces human agency while the policy dimension is to examine whether policies that do not stigmatise (i.e. are 'shame-proofed') have better outcomes in part because they do not add to the shame of poverty. Both propositions being tested are global in construction and are explored in the context of a maximum difference design; if a consistent story emerges in very disparate settings in countries at different levels of economic development, this adds force to the notion of the propositions having universal applicability.

ESRC/DFID funded research (RES-167-25-0557) suggested that shame associated with poverty is ubiquitous and structural being imposed by others in their dealings with people in poverty. Shame may serve to perpetuate poverty through eroding individual agency, while policies that stigmatise could be counterproductive in adding to the debilitating effects of shame.

This research has proved very influential, spawning replications and adding poverty-related shame to the topics covered by the ESRC 'Understanding Society' study, following the successful inclusion of questions in the ESRC 'Poverty and Social Exclusion in the UK' survey. It also led directly to the International Labour Organisation (ILO) including the principle of governments having respect for the rights and dignity of social protection recipients in Recommendation 202 on social protection approved in 2012. The UN has cited the research to support its guiding principles on extreme poverty and human rights.

However, while the previous research yielded a model hypothesising how shame and stigma might exacerbate poverty, the research was not designed to provide evidence of casual relationships. Hence, new research is proposed to rectify this omission by testing the effectiveness of social protection policies that have been 'shame-proofed' against those which have not. The robustness of the test is much enhanced by simultaneous replication in Uganda, India and China; each evaluation is sensitive to local circumstances, but shaped by a common theoretical framework with a core of standardised measures and comparable methods. In Uganda, theory of change evaluation, which employs diagnostic techniques to examine whether underlying logic of the policy works in practice, is combined with impact evaluation to measure the relative effectiveness (and value for money) of shame-proofing.

Theory of change evaluations will be conducted in China and India with the prospect of impact evaluations being added with financial support from the ILO.

The research builds on a current study undertaken by the same research team, funded by the Norwegian Research Council, to develop the shame-proofed policies that are to be refined, implemented and evaluated in the proposed research. The evaluation will entail a mix of surveys, depth interviews, focus groups, investigative research and extraction of administrative records.

Â