Development of Views and
Guidance for Empty Reviews
The Cochrane Empty Reviews Project
Consensus Meeting
16 June 2011
The Empty Reviews Project is funded by the Cochrane Opportunities Fund
Questions for Discussion
• How do we define an empty review?
– A systematic review that results in no included/eligible studies
• Empty reviews are suitable for/when :
– Important questions for practice or policy
– Trials are feasible
– Though may need to amend the protocol as part of an iterative
process
• Do we need guidance for conducting and reporting empty reviews?
– Yes, by:
1- Reviews done and ready to report
2- Reviews in progress (Woops)
3- Anticipating low evidence reviews
4- Updates
• Our Focus is #1, perhaps, with implications for #2-4
Discussion of possible guidance for
the reporting of completed empty
reviews
Should guidance be generic or organized by review
question (e.g., parachute vs. RCTs unfound vs. side
effects vs. trials inconceivable) or CRG?
• Guidance should apply to the whole
collaboration
Which sections of empty reviews require guidance?
Abstract
*Main results – single sentence of no evidence
*Author’s Conclusion –
*Brief sentence – no evidence
*May highlight possible severe/prevalent adverse effects
Plain Language Summary
*Brief sentence – no evidence of effect…
*Explain why empty, if appropriate, referring to eligibility criteria
*May highlight possible severe/prevalent adverse effects
Summary of Findings
*Include even if empty
Background
Objectives
Methods
Criteria for considering studies for this review (types of studies, PICOT)
Search methods for identification of studies
Data collection and analysis (Study selection, data extraction, and analysis)
*Omit data collection/analysis sections
*Reference protocol
Which sections of empty reviews require guidance?
Results
Description of studies
*Flow Diagram
Results of the search
Included studies
*one sentence – no studies
On-going studies
Excluded studies
*usual text
*do not report excluded results here
Risk of bias in included studies
Effects of interventions
Discussion
*Follow handbook subheadings
*Provide examples
*May discuss excluded study results, but
*under heading
*reference
*discuss reason for exclusion
*bookend with caveats
*brief
Which sections of empty reviews require guidance?
Author’s conclusions
Implications for practice
*Follow handbook – no recommendations
*Add examples to Handbook
*Brief – statement of no evidence
*Other evidence is not to be reported here
*General statements referring to local practice
guidelines
*May flag severe/prevalent adverse effects
*cite evidence
Implications for research
*Signal for update (on-going, awaiting, etc.)
*General statement – research needed/follow
Handbook
Potential Guidance for Implications for
Practice for Empty Reviews
• Should there be a clear statement that absence of
evidence is not evidence of absence (i.e., effect or no
effect)?
• Should other types of evidence be discussed in the
absence of included studies?
– If so, what and how should it be presented? (i.e., formal
citation)
• Should there be minimum content in the Implications
for Practice?
• Should evidence of harmful effects be discussed?
• Should implications be targeted? (e.g., Schiz Group’s
subheadings-client, practitioner, policy-maker)
Writing and Dissemination Strategies
• Standard Handbook applies
– Possible special addenda for empty reviews,
section dependent (draft 2-3 pages to integrate)
•
•
•
•
David Tovey – editor training
Colloquia workshops, editors’ meeting
Website, webinars, etc.
Other forums? a stand alone empty review
reporting document?
Preliminary considerations of empty
reviews at title registration, at
protocol, in progress, and at update
Should contingency plans for empty reviews be defined
in the review TRF/protocol?
• At TRF – (decision tree)
1. Possible to trial? Y/N
2. Aware of at least one possibly includable trial?
Y/N
3. Why important to review even if empty?
• At protocol – (decision tree continued)
– provide a contingency plan in case no eligible
evidence?
Should there be a sliding scale of evidence depending
upon its availability? Decided a priori or post hoc?
• See decision tree discussion
• See EPOC
Should empty reviews be flagged in the Cochrane
Library?
• No
What are the views regarding the updating of empty
reviews?
• Same general principles of updating apply