2013 Nobel Laureates
• Physiology or Medicine: James E. Rothman,
Randy W. Schekman and Thomas C. Südhof
"for their discoveries of machinery regulating vesicle
traffic, a major transport system in our cells“
• Physics: François Englert and Peter W. Higgs
"for the theoretical discovery of a mechanism that
contributes to our understanding of the origin of
mass of subatomic particles…"
Randy W. Schekman
Peter W. Higgs
• sdf
Life sciences research in 2010
US$ 240,000,000,000
85% wasted
Lancet 2013;382:1286-307 & 2009;374:86–9
Avoidable waste or inefficiency in
biomedical research
Are research
decisions
relevant to
users of
research?
Appropriate
research
design,
methods,
and analysis?
Efficient
regulation and
management?
Fully
accessible
research
information?
Unbiased
and usable
research
reports?
Research waste
Lancet 2014;383:101–4
The 42 “Wasters”…
A Metin Gülmezoglu, Andrew Vickers, An-Wen Chan, Ben
Djulbegovic, David Moher, David W Howells, Davina
Ghersi, Douglas G Altman, Elaine Beller, Elina Hemminki,
Elizabeth Wager, Fujian Song, H Bart van der Worp,
Harlan M Krumholz, Iain Chalmers, Ian Roberts, Isabelle
Boutron, Janet Wisely, John P A Ioannidis, Jonathan
Grant, Jonathan Kagan, Julian Savulescu, Kay Dickersin,
Kenneth F Schulz, Malcolm R Macleod, Mark A Hlatky,
Michael B Bracken, Mike Clarke, Muin J Khoury, Patrick
Bossuyt, Paul Glasziou, Peter C Gøtzsche, Robert S
Phillips, Robert Tibshirani, Rustam Al-Shahi Salman,
Sander Greenland, Sandy Oliver, Silvio Garattini, Steven
Julious, Susan Michie, Tom Jefferson, Ulrich Dirnagl
1. Setting research priorities
Lancet 2014;383:156–65
1. Setting research priorities
The inefficiency of basic science research
1 intervention used widely
5 resulted in licensed clinical
interventions by 2003
101 claimed that new discoveries had
clear clinical potential
>25,000 reports in 6 basic science
journals 1979-83
Am J Med 2003;114(6):477-84
1. Setting research priorities
Thomas Edison
"Young man, why would I
feel like a failure? And why
would I ever give up?
I now know definitively
over 2,000 ways that an
electric light bulb will not
work. Success is almost in
my grasp.“
1. Setting research priorities
Best basic:applied research funding ratio?
Lancet 2014;383:156–65
1. Setting research priorities
Whose priorities?
Lancet 2014;383:156–65
1. Setting research priorities
Involve stakeholders
Lancet Neurol 2012;11:209
1. Setting research priorities
Set research in the context of systematic reviews
Lancet 2014;383:156–65
1. Setting research priorities
What systematic reviews could have shown…
tPA in animal stroke models
prone:supine SIDS:controls
Lancet 2014;383:156–65
1. Recommendations
•
•
•
•
More research on research should be done to identify factors associated with
successful replication of basic research and translation to application in health
care, and how to achieve the most productive ratio of basic to applied research
Research funders should make information available about how they decide what
research to support, and fund investigations of the effects of initiatives to engage
potential users of research in research prioritisation
Research funders and regulators should demand that proposals for additional
primary research are justified by systematic reviews showing what is already
known, and increase funding for the required syntheses of existing evidence
Research funders and research regulators should strengthen and develop sources
of information about research that is in progress, ensure that they are used by
researchers, insist on publication of protocols at study inception, and encourage
collaboration to reduce waste
Lancet 2014;383:156–65
2. Design, conduct and analysis
Incongruent statistical findings in publications in 2001
(rounding, transcription, or type-setting errors)
25%
38%
62%
75%
BMC Med Res Methodology 2004;4:13
2. Design, conduct and analysis
Failure to replicate published pre-clinical academic
results
11%
36%
64%
89%
Lancet 2014;383:166–75
2. Design, conduct and analysis
High effect-to-bias ratio
In vivo studies
Lancet 2014;383:166–75
2. Recommendations
• Make publicly available the full protocols, analysis
plans or sequence of analytical choices, and raw data
for all designed and undertaken biomedical research
• Maximise the effect-to-bias ratio in research through
defensible design and conduct standards, a well trained
methodological research workforce, continuing
professional development, and involvement of nonconflicted stakeholders
• Reward (with funding, and academic or other
recognition) reproducibility practices and reproducible
research, and enable an efficient culture for replication
of research
Lancet 2014;383:166–75
3. Regulation and management
“…the clinician who is convinced that a
certain treatment works will almost
never find an ethicist in his path,
whereas his colleague who wonders
and doubts and wants to learn will
stumble over piles of them”
Lancet 1990;336:846–7
3. Regulation and management
Delays and inconsistency in ethics and governance
Lancet 2014;383:176–85
3. Regulation and management
Is regulation proportionate, when the public approves?
23%
28%
72%
UK National Cancer Registry
including postcode, name and
address, and sending a letter
inviting them to a research study
77%
Finland national biobank of
existing diagnostic and research
samples
Lancet 2014;383:176–85
3. Regulation and management
RCTs recruited pre-specified sample size
31%
45%
55%
69%
114 RCTs funded by MRC or HTA in
the UK in 1994-2003
73 RCTs funded by MRC or HTA in
the UK in 2002-2008
HTA 2007;11:ix–105 & Trials 2013;14:166
3. Regulation and management
Better recruitment after UK clinical research networks
Lancet 2014;383:176–85
3. Recommendations
• People regulating research should use their influence to reduce
other causes of waste and inefficiency in research
• Regulators and policy makers should work with researchers,
patients, and health professionals to streamline and harmonise the
laws, regulations, guidelines, and processes that govern whether
and how research can be done, and ensure that they are
proportionate to the plausible risks associated with the research
• Researchers and research managers should increase the efficiency
of recruitment, retention, data monitoring, and data sharing in
research through the use of research designs known to reduce
inefficiencies, and do additional research to learn how efficiency
can be increased
• Everyone, particularly individuals responsible for health-care
systems, can help to improve the efficiency of clinical research by
promoting integration of research in everyday clinical practice
Lancet 2014;383:176–85
4. Accessible reporting
Proportion of funded/completed research that is reported
100%
1980 - 1996
80%
1997 - 2005
60%
40%
20%
UK
US
Sw A
itz
Ge erl.
rm
an
y
Sp
a
Ne i n
th
er
l.
U
Au K
st
ra
lia
Fr
an
D e ce
nm
ar
Ca k
na
Au da
st
ra
lia
Sp
ai
n
US
A
0%
HTA 2010;14(8):iii, ix-xi, 1-193
4. Accessible reporting
Reporting is selective
Lancet 2014;383:257–66
4. Accessible reporting
Associations with reporting
Oseltamivir
Rosiglitazone
Rofecoxib/celecoxib
etc
Lancet 2014;383:257–66
4. Accessible reporting
Language bias
Chinese
Biomedical
Literature
database
Chinese
Medical
Current
Content
China
National
Knowledge
Infrastructure
Chinese
Scientific
Journals
database
Chinese
Medicine
Premier
2,500 biomedical journals, <6% indexed in Medline
Health Info Libr J 2008;25:55–61
4. Accessible reporting
Lancet 2014;383:257–66
4. Recommendations
• Institutions and funders should adopt performance metrics
that recognise full dissemination of research and reuse of
original datasets by external researchers
• Investigators, funders, sponsors, regulators, research ethics
committees, and journals should systematically develop
and adopt standards for the content of study protocols and
full study reports, and for data sharing practices
• Funders, sponsors, regulators, research ethics committees,
journals, and legislators should endorse and enforce study
registration policies, wide availability of full study
information, and sharing of participant-level data for all
health research
Lancet 2014;383:257–66
5. Complete & usable reporting
Lancet 2014;383:267–76
5. Complete & usable reporting
Lancet 2014;383:267–76
5. Complete & usable reporting
Lancet 2014;383:267–76
5. Recommendations
• Funders and research institutions must shift
research regulations and rewards to align with
better and more complete reporting
• Research funders should take responsibility for
reporting infrastructure that supports good
reporting and archiving
• Funders, institutions, and publishers should
improve the capability and capacity of authors
and reviewers in high-quality and complete
reporting
Lancet 2014;383:267–76
Avoidable waste or inefficiency in
biomedical research
Are research
decisions
relevant to
users of
research?
Appropriate
research
design,
methods,
and analysis?
Efficient
regulation and
management?
Fully
accessible
research
information?
Unbiased
and usable
research
reports?
Research waste
Lancet 2014;383:101–4
How can, and will, we change?
Implementation Science 2011;6:42