Systematic Reviews: Post-Searching Process

Reviewing/Weeding the search results to select the relevant studies is the most time-consuming part of a systematic review. It requires a significant investment of time and effort.

Once all of the searching and weeding is done and the final set of journal articles  have been selected, there are three additional techniques to be considered, along with grey literature searching:

• Handsearching (Journal TOCs)
• Snowballing (AKA Citation/Reference Tracking)
• Author Contacts

• Many, if not most, systematic review search protocols yield thousands of abstracts.
• Do NOT be surprised if you have somewhere in the neighborhood of 5000 abstracts to weed through in the first round. Quite often, the first round of abstract weeding will knock out 80-90% of the abstracts. That is normal.
• You MUST have at least 2 independent reviewers who make their inclusion/exclusion decisions separately. A comparison will give you a measure of the inter-rater reliability or concordance.
• This is reiterative process.
• Take advantage of any available software that can make this process easier.
• Keep very complete and careful records. TRACK YOUR NUMBERS! All articles retrieved by the search protocol must be accounted for in the final numbers. You will have a problem writing the paper at the end if this is not done carefully at all stages.
• Record reasons for exclusions at least at the Full-Text stage if not at the Abstract stage.
• A similar process should be employed for grey literature results if possible.
• Set reasonable, but definite, deadlines for this process. It can take a while. This is where most systematic reviews falter or fail.