Systematic Review Guide

In 1997, The Luxembourg Convention on Grey Literature defined it as š“that which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers”  (Farace 1998)

Grey literature is everything but peer-reviewed journals and academically or commercially published books. (Von Hendy, 2014)
š
Synonyms: Fugitive literature, gray literature

Forms of Grey Lit include: Reports, Conference Abstracts, Dissertations & Theses, Registered Clinical Trials, Interviews, Patents, Newsletters, White Papers, Book Chapters

Where to Search

Dissertations and Theses aren't always easy to locate but they can be valuable resources. Search options include:

• Proquest
• Worldcat
• Google Scholar (It picks up from many academic institutional repositories.)
  • Use domain limits (org, edu, int, gov, mil)
• Networked Digital Library of Theses and Dissertations (NDLTD)

Advantages

• Minimizes bias
• Finds valuable data that would otherwise be missed
• Increases currency and accuracy
• Broadens applicability of review
• Sometimes only source of specific local/regional information
• May provide more concrete examples of implementing policies or procedures
• Often free
• Multidisciplinary

ššDisadvantages

• Takes time and effort
• Difficult to find and access; may have associated costs
• Indexing  and search interfaces may be poor quality
• Often isn’t peer-reviewed
• Overwhelming number of results
• Not suitable method for some grey lit document types 

Bottom line ⇒ You are not doing a real systematic review if you don't include some gray literature searching.

Be Transparent

• Document the search terms, number of results, limitations, inclusion/exclusion criteria, and document types.

“Publication bias is the term for what occurs whenever the research that appears in the published literature is systematically unrepresentative of the population of completed studies.”  (Rothstein, 2005)
š
Systematic reviews aim to include ALL high quality studies about the research question. This is difficult to accomplish, but a missed study or studies may affect results and conclusions.
 

As the Cochrane Collaboration reports, studies that report statistically significant 'positive' results are:

• more likely to be published (publication bias)
• more likely to be published rapidly (time lag bias)
• more likely to be published in English (language bias)
• more likely to be published more than once (multiple publication bias)
• more likely to be cited by others (citation bias)

Because of this, negative studies, equivocal results studies, and non-English studies are less likely to get published, less likely to get into the top journals, and less likely to get cited.

Not everything gets published in peer-reviewed journals, particularly English peer-reviewed journals.

Rothstein H, Sutton AJ, & Borenstein M. (2005). Publication bias in meta-analysis prevention, assessment and adjustments. Chichester, England; Hoboken, NJ: Wiley.