National Centre for Research Methods

Project Details

Research Cluster: CEBI
Start Date: 01.01.2016
End Date: 30.06.2018

Related Staff

GRADE Extension for Complex Social Interventions

DSPI Principal Investigator: Professor Paul Montgomery
Funded by: Economic & Social Research Council, Department for International Development

Principal Investigator: Prof Paul Montgomery (

Research Officer: Ms Ani Movsisyan (ani.movsisyan(at)

General contact email: grade(at) 

International Co-Investigators

Dr Sean Grant – RAND Corporation, Santa Monica, USA
Dr Jane Dennis – Research Synthesis Ltd, Bristol, UK
Dr Erik von Elm – Institut Universitaire de Médecine Sociale et Preventive (IUMSP), Lausanne, Switzerland
Dr Eva Rehfuess – Institute of Medical Informatics, Biometry and Epidemiology Ludwig-Maximilians – University, Munich, Germany
Prof Geraldine Macdonald – University of Bristol, Bristol, UK
Dr Susan Norris – World Health Organization (WHO), Geneva, Switzerland 


This project, funded under the NCRM Methodological Research Projects funding call, brings together an international collaboration that aims to develop a methodology for rating the quality of evidence and the strength of recommendations for complex interventions.

It is now standard practice for policy makers to draw on systematic reviews as the superior source of evidence to inform decision-making regarding effective interventions. Social interventions, commonly applied within the disciplines of international development, psychology, education, social work and welfare, criminology and public health, are most frequently complex. This means they might involve a number of interacting components, multiple outcomes, and diverse delivery formats. In addition, they might be more amenable to contextual factors and intervention implementation issues. For the research synthesis endeavours to be effective in guiding policy and practice, they must utilise adequate methodology that reflects the unique features of these interventions.

The most prominent system to guide evidence-informed decision-making has been developed by the Grading of Recommendations Assessment, Development and Evaluation Working Group (GRADE) in clinical medicine.

  • The GRADE approach provides a systematic and transparent process of rating the "best-available evidence" to inform recommendations for practice.
  • Having been endorsed by more than 80 organisations worldwide, including the Cochrane Collaboration and the World Health Organisation (WHO), the applicability of the GRADE approach outside of clinical practice has been questioned, in part because of differences of the evidence base in biomedical and social practice domains.  
  • Many researchers argue that GRADE does not provide a systematic approach to capture a wide range of evidence important for complex social interventions, including evidence on intervention implementation, mechanisms and contexts.

This project aims to:

  • Develop and disseminate an extension to the GRADE approach for complex social interventions.
  • With the coordination of a Steering Committee of leading experts in the field, the project team will first organise an international online panel involving multidisciplinary expertise to identify aspects that need to be modified in the GRADE approach for complex interventions.
  • Following the online expert panel, a consensus meeting will be hosted with a select group of participants to build consensus for the GRADE extension.
  • Throughout all the phases of the project, the project team will closely collaborate with the GRADE Working Group to assure the project output is an official extension to the GRADE approach.
Related Publications
  • Movsisyan, Ani, Melendez-Torres, GJ, Montgomery, Paul (2015) “Users identified challenges in applying GRADE to complex interventions and suggested an extension to GRADE”, Journal of Clinical Epidemiology In Press. doi: 10.1016/j.jclinepi.2015.09.010
  • Movsisyan, Ani, Melendez-Torres, GJ, Montgomery, Paul (2016) “Outcomes in systematic reviews of complex interventions never reached 'high' GRADE ratings when compared to those of simple interventions”, Journal of Clinical Epidemiology In Press. doi: 10.1016/j.jclinepi.2016.03.014 
  • Movsisyan, Ani, Melendez-Torres, GJ, Montgomery Paul (2016). "A harmonised guidance is needed on how to 'properly' frame review questions to make the best use of all available evidence in the assessment of effectiveness of complex interventions", Journal of Clinical Epidemiology In Press. doi: 10.1016/j.jclinepi.2016.04.003

More information

The Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group website

How National Centre for Research Methods projects are called and funded

Cochrane Switzerland

World Health Organization

WHO Guidelines