Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.


Stony Brook University

How to Conduct a Systematic Review

This guide covers all of the information you need to know in order to prepare for and conduct a systematic review.

What is Grey Literature?

In 1997, The Luxembourg Convention on Grey Literature defined it as š“that which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers”  (Farace 1998)
š
Synonyms: Fugitive literature, gray literature

Forms of Grey Lit include: Reports, Conference Abstracts, Dissertations & Theses, Registered Clinical Trials, Interviews, Patents, Newsletters, White Papers, Book Chapters

Where to Search

Dissertations and Theses aren't always easy to locate but they can be valuable resources. Search options include:

Using the Grey Literature

Advantages

  • Minimizes bias
  • Finds valuable data that would otherwise be missed
  • Increases currency and accuracy,
  • Broadens applicability of review

ššDisadvantages

  • Takes time and effort
  • Difficult to find and access; may have associated costs
  • Indexing  and search interfaces may be poor quality
  • Often isn’t peer-reviewed

Bottom line ⇒ You are not doing a real systematic review if you don't include some gray literature searching.

Why is it important?

“Publication bias is the term for what occurs whenever the research that appears in the published literature is systematically unrepresentative of the population of completed studies.”  (Rothstein, 2005)
š
Systematic reviews aim to include ALL high quality studies about the research question. This is difficult to accomplish, but a missed study or studies may affect results and conclusions.
 

As the Cochrane Collaboration reports, studies that report statistically significant 'positive' results are:

  • more likely to be published (publication bias)
  • more likely to be published rapidly (time lag bias)
  • more likely to be published in English (language bias)
  • more likely to be published more than once (multiple publication bias)
  • more likely to be cited by others (citation bias)

Because of this, negative studies, equivocal results studies, and non-English studies are less likely to get published, less likely to get into the top journals, and less likely to get cited.

Not everything gets published in peer-reviewed journalsparticularly English peer-reviewed journals.

Rothstein H, Sutton AJ, & Borenstein M. (2005). Publication bias in meta-analysis prevention, assessment and adjustments. Chichester, England; Hoboken, NJ: Wiley.