Skip to main content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.


Systematic Reviews: Appraising Pool of Citations

This guide is for the clinician undertaking a systematic review.


You've finally narrowed the thousands of citations down to a smaller, more relevant group of articles. Now what do you do to make sense of this information? Take a look below at the various tools and guides to help you determine the quality of and risk of bias for each article.

"Critical appraisal skills enable you to assess the trustworthiness, relevance and results of published papers so that you can decide if they are believable and useful."  (

Critical Appraisal Checklists by specific Study Design type

Randomized Controlled Trials


RoB 2.0 Risk Of Bias Tool

New as of July 2019

RoB 2 Tool : A revised Cochrane risk of bias tool for randomized trials.

Archived: There are 3 variations of the tool based on trial design.  Choose the appropriate version.

1.  Individually randomized, parallel group trials (traditional RCT)

2.  Cluster randomized, parallel group trials (like traditional but randomized by groups)

3.  Individual randomized, cross-over trials (randomization occurs to determine the order the patient receives both treatments)​

CASP: Randomised Controlled Trial Checklist
Summary: Critical Appraisal Skills Program (CASP): RCT CAT is a methodological checklist which provides key criteria relevant to randomised controlled trials.
Authors: Public Health Resource Unit, NHS, England

The CONSORT Statement
Summary: The Consolidated Standards of Reporting Trials (CONSORT) Statement is a detailed document which outlines an explanation and elaboration of the CONSORT statement for reporting randomized controlled trials. It also includes (in table two) the critical appraisal tool.
Authors: The CONSORT Group, Canada


Non-randomized Studies or Observational Studies



Risk of Bias for non-randomized (observational) studies or cohorts of Interventions​



Risk of Bias for non-randomized (observational) studies or cohorts of Exposures other than interventions, including environmental and occupational exposures​


Newcastle-Ottawa scale (NOS) -most widely used for case control or cohort studies



AXIS-To assess the quality of cross-sectional (prevalence) studies


NIH – NHLBI – Study Quality Assessment Tools

Tools for Controlled Intervention Studies, Systematic Reviews & Meta-Analyses, Observational Cohort and Cross-Sectional Studies, Case-Control Studies, Before-After (Pre-Post) Studies With No Control Group, Case Series Studies.


IHE Case Series Studies Critical Appraisal Checklist


Methodological Index for Non-Randomized Studies (MINOR) is a tool for assessing non-randomized interventional studies


Joanna Briggs Institute

Critical Appraisal Tools


McMaster Critical Review Form - Quantitative Studies


Systematic Reviews


AMSTAR can be used to assess the methodological quality of a systematic review and as a guide to performing a systematic review. Two agreements are required during quality assessment ensuring lower risk of bias. AMSTAR has guidelines explaining each outlined item.



ROBIS is a new tool for assessing the risk of bias in systematic reviews (rather than in primary studies). Here you can find the tool itself, information to help you complete a ROBIS assessment, and resources to help you present the results of your ROBIS assessment.


CASP: Systematic Review Checklist
Summary: Critical Appraisal Skills Program (CASP): This is a methodological checklist which provides key criteria relevant to systematic reviews.
Authors: Public Health Resource Unit, NHS, England


CEBM:  Systematic Reviews Critical Appraisal Worksheet 
Authors: Centre for Evidence Based Medicine (CEBM), University of Oxford

Public Health


MetaQat - Meta-Tool for appraising all types of public health evidence

Quantitative Research (Public Health Research)


EPHPP- Quality Assessment tool for Quantitative Studies 

Also known as the "Hamilton" tool

Animal Research


Systematic Review Centre for Laboratory Animal Experimentation (SYRCLE) risk of bias assessment tool for assessing animal studies


(invivo studies) Review the ARRIVE guidelines


Critical Appraisal of Studies using Animal Models

Diagnostic Accuracy



QUADAS-2 is the current version of QUADAS and the tool that we recommend for use in systematic reviews to evaluate the risk of bias and applicability of primary diagnostic accuracy studies. 


STARD Checklist
Summary: STAndards for the Reporting of Diagnostic accuracy checklist is helps determine the accuracy and completeness of reporting of studies of diagnostic accuracy. The STARD Statement consists of a 25 item checklist and flow diagram.​

CASP: Diagnostic Test Checklist


CEBM: Diagnosis Critical Appraisal Worksheet
Authors: Centre for Evidence Based Medicine (CEBM), University of Oxford

Learn more about Critical Appraisal

Grading the Strength of Evidence

Additional Grading Tables & Scales


Study Design Flow Chart


  • Alan Schwartz: Diagnostic Test Calculator
    This calculator can determine diagnostic test characteristics (sensitivity, specificity, likelihood ratios) and/or determine the post-test probability of disease given the pre-test probability and test characteristics. Given sample sizes, confidence intervals are also computed.
  • Alan Schwartz: Risk Reduction Calculator
    Given information about the probability of an outcome under control and experimental treatments, this calculator produces measures of risk increase/decrease and number needed to treat or harm, including confidence intervals. If some patients were lost to follow-up, the calculator provides estimates for several different scenarios.
  • Alan Schwartz: Diagnostic Test Cutoffs
    A graphical demonstration of the effect of changing cutoff scores on sensitivity and specificity of a test.
  • Alan Schwartz: Statistical Testing Thresholds
    A graphical demonstration like the above, but written in terms of statistical test theory (type I and II error).