Automatize or perish: Towards user-led, instant and (at
least) semi-automated systematic reviews
1
Torres ,
1
Rana ,
2
Stark ,
2
Hagmann ,
2
Knahl
Mercedes Torres
Aakash
Benjamin
Sebastian
Constanze
2
2
2
3
Stefanie Polzmacher , Annabelle Wolff , Selin Nur , Clive E Adams
School of Computer Science1, Cochrane Schizophrenia Group3, University of Nottingham, UK
2
University of Applied Sciences, Ulm, Germany
,
Background
The reviewer’s hand-crafted masterpiece
• Current systematic reviewing, from protocol generation to review publication, is, largely, manual.
• This process is slow, expensive, repetitive, inaccurate and lacks transparency.
• A framework to procure, curate, and synthesise tailored evidence when needed has to be developed.
Karl Benz’s hand-crafted masterpiece
The idea
Cochrane’s mass manufacturing
“You can have any review as long as it’s systematic”
Writing the
protocol
Finding
evidence
Dissemination
Data
extraction
Write up
Henry Ford’s masterpiece of mass manufacturing
“You can have any colour as long as it’s black”
Results of extensive search for relevant work
Question
setting
Update
• We propose a short-medium term rejuvenation of the current framework for systematic reviewing
employing existing technology but tailoring and developing it for this purpose. This approach would
combine Biomedical Informatics, Document Engineering, Machine Learning, Human-Computer Interaction
and end-user needs to automatically and instantly:
• Relate to only relevant best evidence.
• Optimize PDF and HTML documents to make data extraction more accurate.
• Extract accurate quantitative and qualitative data from randomized trials efficiently using methods such
as Deep Learning, Support Vector Machines or ensemble classifiers.
• Create an online repository with all the data extracted from the optimized documents.
• Present data tailored to users’ previously-defined priorities/values and specifications.
Synthesis
Activities in the reviewing cycle with
already developed systems of automation
• Much relevant expertise and software exists.
• Aspects of this work are ongoing in many areas relevant to health care.
• Current approaches, however, are entrepreneurial, idiosyncratic and disjointed. It is only a short matter
of time before teams unite to ensure the conveyer-belt of best current evidence runs swiftly.
Mercedes Benz’s current mass manufacturing
The future
Soon a new framework for systematic reviewing will:
• Support instant, efficient and accurate best evidence generation.
• Save time, costs and labour.
• Present data to all users via a thoroughly-tested user-friendly online repository.
• Allow users to access personalised data according to needs.
Conclusions
•
•
•
•
•
The production line of systematic reviews related to health should never aim to mass produce a
perfect cloned product
These types of reviews will always involve peoples’ values and judgements
However, the time between question-setting and interpretation of synthesised best evidence can be
seconds not months or years
Much effort has already been expended in this process and, with leadership, co-ordination and
investment, this investment could be amalgamated
This would produce a new system where repetitive processing involving high-precision drudgery is
left to machines and the personal value-linked consideration of what to ask and what data
mean left to people.
Contact:
Dr Mercedes Torres Torres (psxmt3@nottingham.ac.uk) School of Computer Science, University of Nottingham, Jubilee
Campus, Wollaton Road, Nottingham, NG8 1BB, UK.