Skip to Main Content

Systematic reviews

A brief overview of systematic reviews and resources to support producing one.

Working with results

The post search phase of the systematic review process is to go through the results to select those that are relevant. The studies that are included are assessed for methodological rigour using an appraisal tool. Those studies that are included in the final step go through a process of data extraction and depending on the nature of the review and the results of the studies they may be synthesised in a process like a meta-analysis.

Selecting studies

The initial process for the systematic review is to collect all of the search results and citations collected from other resources to check the references for their relevancy to the review question. The first step on this process is to discard the duplicates as there will often multiple versions of the same reference from different databases.

The next step is to review the citations at the title and abstract level by two reviewers independently checking the references for inclusion. The citations which are included from this step are then accessed as the full article and appraised according to the review criteria.

Online tools such as Covidence  or JBI's SUMARI(Available from the Evidence tools within the JBI EBP database) can help in the screening process and also with data extraction. 

The numbers of citations included and excluded for this process need to be recorded for the review. There is a free tool called the PRISMA flow diagram which is commonly used to demonstrate the process of inclusion and exclusion and to record the numbers of references. The PRISMA flow diagram is available from the PRISMA statement website:

Data extraction

Data extraction is the process that describes the collection of relevant information about the findings and characteristics of a study that is included in a systematic review. This information is usually collected in a data extraction form which will consist of a number of elements dependent on the review question.

Data can mean any information from a study including:

  • Methods
  • Participants
  • Setting
  • Context
  • Interventions
  • Outcome measures

The Cochrane Handbook has a list of items to consider that may be useful for developing a data extraction form.

Useful references

 Meade MO, Richardson WS. Selecting and appraising studies for a systematic review. Annals of internal medicine. 1997;127(7):531-7.

 Elamin MB, Flynn DN, Bassler D, Briel M, Alonso-Coello P, Karanicolas PJ, et al. Choice of data extraction tools for systematic reviews depends on resources and review complexity. Journal of clinical epidemiology. 2009;62(5):506-10.

Higgins JPT, Deeks JJ. Chapter 7: selecting studies and collecting data. 2011. In: Cochrane handbook for systematic reviews of interventions version  510 [Internet]. The Cochrane Collaboration. Available from: www.cochrane-handbook.org. .

Study apprasial

Appraisal is the process of critically analysing a study for its inclusion in a systematic review. The process determines if there are any factors which may influence the outcome of the study. These factors can be termed bias. There are tools available to appraise a number of different study types available via the Equator Network - Equator (Enhancing the QUAlity and Transparency Of health Research) 

More information and links about critically appraising studies can be found in the Evidence-based practice in health sciences guide.

References

Higgins JP, Altman DG, Sterne JA. Chapter 8: Assessing risk of bias in included studies. 2011. In: Cochrane handbook for systematic reviews of interventions version  510 [Internet]. The Cochrane Collaboration. Available from: www.cochrane-handbook.org

Synthesis and analysis - Combining studies

The final part of the systematic review is to combine the result to answer the research question. This may be via a quantitative method using a statistical approach such as with a meta-analysis or it may rely on other methods of synthesis such those used in qualitative topics like meta-ethnography.

The final combination of results will be dependent on the nature of the question and the quality and homogeneity of the research.

Types of Synthesis:

Quantitative

  • Meta-analysis

Qualitative

  • Meta-ethnography
  • Thematic synthesis
  • Grounded theory
  • Content analysis
  • Qualitative comparative analysis
  • Qualitative research synthesis

In performing a systematic review utilising qualitative data you may find the software NVivo helpful. There is a library guide available to help you out. It is entitled: Using NVivo in systematic reviews

References:

Crombie IK, Davies HT. What is meta-analysis? : Bandolier.org.uk; 2009.

Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC medical research methodology. 2009;9:59.