UPDATING COCHRANE
REVIEWS: FINDINGS OF AN
NHS ENGAGEMENT PROJECT
AND FUTURE UPDATING
OPTIONS
Author: Rachel Marshall
Background

1.
2.
3.
Three objectives of the project:
To identify those Cochrane Reviews that NHS
stakeholders regard as the most important to update.
To develop a decision tool for determining whether
and when to update Cochrane Reviews.
To explore whether targeted consultancy for a small
number of CRGs could facilitate updating priority
Cochrane Reviews, and to identify whether
efficiencies could be made in the updating process.
Aims


We aimed to provide short and focused periods of
assistance, to incentivise authors, and to address
barriers to updating.
The intended approach was to retain both the
responsibility for producing the update within the
authors, and editorial support for within the CRGs.
Methods





One-year project (July 2010 to July 2011)
Involved 6 CRGs
Each CRG offered 37 days’ of assistance for updating
1 to 6 Cochrane Reviews (between November 2010
and April 2011)
Assistance provided by Bazian, the CEU, the
Cochrane Musculoskeletal Group and Karla SoaresWeiser (free lance)
Project managed by the CEU; tasks assigned to
updaters once a week
Tasks offered to CRGs and authors
Results: status of updates




We worked on 14 Cochrane Review updates
Mean time spent: 35.5 hours (median 26.71 hours, range
4.5 to 109.75 hours)
In June 2011
 4 updates had been submitted for CRG approval
 3 were having text revised before submission to CRG
 2 were inputting extracted data into RevMan
 2 were extracting data
 2 were abstract screening
 1 was unknown (no author response)
In October 2011, no updates yet published
Results: assessment of tasks
Results: efficiencies

Search: MEDLINE-only searches for suitable updates, peerreview and centralised storage


Appraisal: appraise in Endnote if lots of abstracts/papers




although note limited searches performed, and reluctance
among CRGs to use methods
Or all contributors to use same process
Data extraction: extraction sheets that follow the same
structure as RevMan, or possible double extraction into
RevMan.
Tasks: author to perform duplicate (where applicable)
Documents: store relevant documents (previous search
appraisals, peer-review comments, status notes, papers
[with copyright holder permission] ) in Archie
Results: project management







Project management sometimes time consuming
and required lots of organisation.
Peaks and troughs in work flow.
Sometimes more than one person on a review.
Some misconception from authors about the role
of the updating service.
Defining what needed to be done.
Background to last version of review.
Some unresponsive authors.
Results: CRG and author feedback



Evaluation form: all CRGs responded, most authors
responded (1 on holiday, 1 no response)
Scale 1-5 (1 greatly improved, 5 greatly worsened)
Authors
CRGs
Author motivation
1.6
1.8
Speed of update
1.6
2.5
Quality of update
1.8
2.75
Overall impression
of assistance
1.44
1.5
All authors and managing editors who responded to the
question on rolling out an updating service for The Cochrane
Collaboration were in favour of an updating service (2 CRGs
did not respond)
Results: CRG and author feedback



“The assistance was greatly appreciated and has really
helped us progressing both reviews.”
“Assistance with risk of bias assessment (especially of
included studies in the previous version) and summary of
findings table. These new methods seem to be daunting to
authors who have not used them previously.”
“There is potential for a disconnect between the review
authors, the CRG, the CEU and the updater. Whilst the
general communication was excellent, I did not get a sense
that if the author went back to ‘silent’ mode there was
anyone monitoring this to ensure the momentum was
maintained.”
Conclusions: lessons learned




Consider updating search efficiencies
Consider storage of documents relevant to an update
Consider the efficiencies highlighted
If a centralised service were rolled out, we would recommend the
following:






Employ staff gradually to avoid wasting resources (possible free lance
approach).
One project manager to oversee all updates, and one member of the updating
service to be assigned to each update.
Updating service potentially to make an initial assessment of tasks to be
performed during the update.
Updating service to call authors to discuss schedule of work, to highlight
information relevant to the update, and to identify any documents or Archie
permissions required for the update.
Authors to sign/agree work schedule.
Updating service to perform tasks highlighted.
Discussion
Comments and questions on the project
Group work

What incentives for updating are feasible and
effective, and how can we remove barriers to
updating?
 Assistance
 Process and
 Recognition
technology
Disclaimer

This project was funded by the NIHR Cochrane –
NHS Engagement Award Scheme (project number
10/4000/01). The views and opinions expressed
therein are those of the authors and do not
necessarily reflect those of the Department of
Health.
Acknowledgements
Cochrane Musculoskeletal Group
 Cochrane Pain, Palliative and Supportive Care
Group
 Cochrane Infectious Disease Group
 Cochrane Wounds Group
 Cochrane Neonatal Group
 Cochrane Airways Group
 Karla Soares-Weiser
