USING SYSTEMATIC REVIEWS: AN EXAMPLE
Let’s say you want to know if taking young people who live in at-risk neighbourhoods to visit prisons can help to prevent those young people from committing crimes later in life. Here is how to find out:
STEP 1 – FIND GOOD EVIDENCE (IF IT EXISTS)
Another really good source for plain-language summaries of reviews of research is the Cochrane Collaboration, which has some of the highest standards for systematic reviews and a large library of reviews on many topics.
STEP 2 – DO A SEARCH
Search in the ‘Cochrane reviews’ library for the keywords ‘prison’ and ‘youth’. You should find a number of examples, one of the most recent of which is a systematic review entitled ‘‘Scared Straight’ and other juvenile awareness programmes for preventing juvenile delinquency’. Cochrane provides a simple summary of this review, which concludes:
Results indicate that not only do these programs fail to deter crime, but they actually lead to more offending behavior. The intervention increases the odds of offending by between 1.6 to 1 and 1.7 to 1.
STEP 3 – DIG DEEPER
You could read more detail about this review by clicking the ‘Read abstract’ link.
The abstract would tell you that the review authors analysed nine studies of the Scared Straight programme – which took place in eight different states in the USA – and that these nine studies included nearly 1000 young people of different races. The average age of the participants was between 15 and 17 years, and most were male. Only seven of the nine studies could be meta-analysed (two of the studies could not, and the researchers explain that it is because those two studies did not report enough information about results).
STEP 4 – READ THE FOREST PLOT
You can find out even more information by clicking the link entitled ‘Find the research’. For example, if you click on the ‘Figures’ tab, you will find forest plots, labelled as ‘Analysis 1.1.’, ‘Analysis 1.2’, etc. Analysis 1.1 shows you a forest plot of the results of one of the measures the researchers analysed: recidivism rates.
The forest plot shows that most of the included studies had results that fall to the right of the line, in the ‘Favours control’ section of the graph. This tells you that the group that did not receive the crime-prevention intervention actually did better than the group that received the intervention. The forest plot also tells you that the studies that were included in this analysis all had fairly similar results. You can tell this by looking at the level of heterogeneity, which is measured using the I2 statistic (in the bottom left-hand corner of the forest plot). In this case, I2=29% (anything below 50% can be considered low). What this tells you is that any differences in the results among the trials were probably due to chance, so you can have confidence in the results. Learn more about heterogeneity.
STEP 5 – APPLY THE EVIDENCE TO YOUR QUESTION OR CONTEXT
Based on your exploration of programmes that introduce at-risk youth to prisons, it is apparent that such programmes have very little evidence to support them. So you may want to ask a new question, such as: which interventions can help to prevent at-risk youth from committing crimes in the future? You can investigate this by repeating the steps outlined above. Once you uncover information about other interventions, you need to weight the evidence against other factors. Learn more about using evidence to improve decision-making.