Skip to main content


Stony Brook University

Systematic Reviews: Grading Evidence

A guide to conducting systematic reviews.

Bias

Always address the issue of bias in the studies under consideration for your review, as well as the bias that may be inherent in your review itself.

There are a variety of types of bias to be considered in a systematic review including publication bias, reporting bias and included study bias.

Publication Bias

The peer-reviewed journal literature has a tendency to over-represent certain types of studies rather than others. In particular, peer-reviewed journals have a preference for publishing studies with positive (i.e. statistically significant) findings, thus studies demonstrating negative or equivocal findings are less likely to be published. There is also a publication preference for English-language studies. This is why it is better not to limit by language and to investigate alternative (grey) sources of evidence in order to locate the 'negative' or 'no difference' studies.

Reporting Bias

Chapter 10: Addressing Reporting Biases From the Cochrane Handbook for Systematic Reviews of Interventions

Bias Within Individual Studies Included Within A Review

Chapter 8: Assessing Risk of Bias in Included Studies From the Cochrane Handbook for Systematic Reviews of Interventions

A Bias is "a systematic error, or deviation from the truth, in results or inferences." Certain types of methodological flaws are more likely to lead to bias within a study or group of similar studies, thus the potential for bias can be assessed be examining study methodology. However, because the results of a study can be unbiased despite a methodological flaw, it is more common to report the risk of bias due to such flaws. Also note that bias is not random error, which is unavoidable. Bias is always systematic.

Quality Rating Scales

There are a dozens of scales and instruments used to evaluate the level of evidence quality in clinical studies included in a systematic review. Some journals and professional associations have their own rankings that they have developed.
 
The Equator Network maintains comprehensive searchable database of reporting guidelines and methodology standards for different types of research studies.
 
Some commonly used ones include:

The Cochrane Collaboration provides slidecasts on how to use GRADE and produce summary tables of results.

The HSL's LibGuide to Evidence-Based Medicine provides more information and resource links on levels of evidence.

Choose an evidence ranking scale that is appropriate for the specialty area and targeted journal for publication of the review. Be consistent in your use of the one you create or select.

Quality of Systematic Reviews

There are methodological quality reporting standards and assessments for systematic reviews. There is variation of quality among reviews, like any other type of research study, and it can sometimes be useful to do an "umbrella review" (a review of reviews).