Evidence Synthesis Guide : Risk of Bias

Bias by Nick Youngson CC BY-SA 3.0 Alpha Stock Images

Clinicians and policymakers rely on systematic reviews as “high-quality evidence for decision-making.”¹ Bias is one of the most significant weaknesses of systematic reviews that “can limit the quality of systematic reviews and can lead to erroneous conclusions.”¹ It is therefore imperative that systematic review authors recognize the risk of bias and understand methods & tools available to them to minimize bias in their reviews.

This guide focuses on two categories of bias. For additional information on bias, see Cochrane Handbook for Systematic Reviews of Interventions, Chapter 7.²:

1. Bias in locating studies occurs when not all relevant literature on a topic is identified or reviewed.^2-4 This bias is minimized by performing a comprehensive search of the literature through bibliographic databases, registers, hand searching, and grey literature searches.
2. Bias in primary studies: “Given that systematic reviews rely on data from other studies, the evidence in a systematic review is only as good as, or as free from bias, the primary data sources.”³ In other words, if the primary study results used for the review are flawed, so will the systematic review. Bias of this sort is minimized through the process of quality assessment and critical appraisal of each of the primary studies.

Consult Cochrane Interactive Learning Module 5: Introduction to Study Quality and Risk of Bias and Module 7: Interpreting the Findings for further information.  *Please note you will need to register for a Cochrane account while initially on the Mayo network. You'll receive an email message containing a link to create a password and activate your account.*

1.    Jackson JL, Kuriyama A. From the Editors’ Desk: Bias in Systematic Reviews—Let the Reader Beware. Journal of General Internal Medicine. 2018;33(2):133-135.

2.     Boutron I, Page MJ, Higgins JP, Altman DG, Lundh A, Hróbjartsson A. Considering bias and conflicts of interest among the included studies. In: Higgins JPT, Thomas J, Chandler J, et al., eds. Cochrane Handbook for Systematic Reviews of Interventions. version 6.2: Cochrane; 2021.

3.     Drucker AM, Fleming P, Chan A-W. Research Techniques Made Simple: Assessing Risk of Bias in Systematic Reviews. The Journal of investigative dermatology. 2016;136(11):e109-e114.

4.     Keenan C. Assessing and addressing bias in systematic reviews. In. Meta-Evidence Blog. Vol 2021. UK & Ireland: Campbell Collaboration; 2018.