Minimizing Bias
Multiple types of bias may impact health evidence. The Cochrane Handbook for Systematic Reviews of Interventions (Table 7.2.a)1 provides definitions of non-reporting biases that can be minimized by identifying all relevant literature on a research topic.
Note: *Prior to assessing Risk of Bias, consider use of the Research Integrity Assessment (RIA) tool for randomized controlled trials to validate the authenticity of studies. See:
Weibel S, Popp M, Reis S, Skoetz N, Garner P, Sydenham E. Identifying and managing problematic trials: A research integrity assessment tool for randomized controlled trials in evidence synthesis. Res Synth Methods. 2023 May;14(3):357-369. doi: 10.1002/jrsm.1599. Epub 2022 Sep 15. PMID: 36054583
Bias in Locating Studies
The publication or non-publication of research findings, depending on the nature and direction of the results, i.e., “the selective publication of manuscripts based on the magnitude or direction of the study results.”2
The rapid or delayed publication of research findings, depending on the nature and direction of the results, i.e., “[t]ime-lag bias occurs when the results of negative trials take substantially longer to publish than positive trials.”3
The publication of research findings in a particular language, depending on the nature and direction of results, i.e., language bias “introduces the risk of ignoring key data… as well as missing important cultural contexts, which may limit the review’s findings and usefulness.”4
The citation or non-citation of research findings, depending on the nature and direction of the results. Citation bias occurs during the process of citation searching for related publications to include in the review. Bias may be introduced by “selective inclusion of statistically significant studies with effect sizes similar to other published studies retrieved from database searching.”5
The multiple or singular publication of research findings. When “studies are published in more than one journal to maximize readership and impact of study findings,” they may inadvertently be included in the systematic review more than once.6
The publication of research findings in journals with different ease of access or levels of indexing in standard databases, depending on the nature and direction of results
The selective reporting of some outcomes or analyses, but not others, depending on the nature and direction of the results, i.e., “Selective reporting bias…, the incomplete publication of outcomes measured or of analyses performed in a study, may lead to the over- or underestimation of treatment effects of harms.”7
References & Recommended Reading
1. Boutron I, Page MJ, Higgins JP, Altman DG, Lundh A, Hróbjartsson A. Considering bias and conflicts of interest among the included studies. In: Higgins JPT, Thomas J, Chandler J, et al., eds. Cochrane Handbook for Systematic Reviews of Interventions. version 6.2: Cochrane; 2021.
2. Montori VM, Smieja M, Guyatt GH. Publication bias: a brief review for clinicians. Mayo Clinic proceedings. 2000;75(12):1284-1288.
3. Reyes MM, Panza KE, Martin A, Bloch MH. Time-lag bias in trials of pediatric antidepressants: a systematic review and meta-analysis. Journal of the American Academy of Child and Adolescent Psychiatry. 2011;50(1):63-72.
4. Stern C, Kleijnen J. Language bias in systematic reviews: you only get out what you put in. JBI Evidence Synthesis. 2020;18(9).
5. Vassar M, Johnson AL, Sharp A, Wayant C. Citation bias in otolaryngology systematic reviews. Journal of the Medical Library Association : JMLA. 2021;109(1):62-67.
6. Fairfield CJ, Harrison EM, Wigmore SJ. Duplicate publication bias weakens the validity of meta-analysis of immunosuppression after transplantation. World journal of gastroenterology. 2017;23(39):7198-7200.
7. Reid EK, Tejani AM, Huan LN, et al. Managing the incidence of selective reporting bias: a survey of Cochrane review groups. Systematic reviews. 2015;4:85.