Avoiding waste in research:
the role of public involvement
Iain Chalmers
Coordinator, James Lind Initiative
‘Putting people first in research’
INVOLVE Conference
Nottingham, 13 November 2012
The skeletons in academic
medicine’s cupboards
Questions relevant
to clinicians &
patients?
Low priority questions
addressed
Important outcomes
not assessed
Clinicians and
patients not involved
in setting research
agendas
Appropriate design
and methods?
Accessible
full publication?
Unbiased and
usable report?
Over 50% studies
designed without
reference to
systematic reviews of
existing evidence
Over 50% of studies
never published in full
Over 30% of trial
interventions not
sufficiently described
Biased underreporting of studies
with disappointing
results
Over 50% of planned
study outcomes not
reported
Over 50% of studies
fail to take adequate
steps to reduce
biases, e.g.
unconcealed
treatment allocation
50
%
Most new research
not interpreted in the
context of systematic
assessment of other
relevant evidence
50
%
85% Research waste = over $85 Billion / year
50
%
Questions relevant
to clinicians &
patients?
Low priority questions
addressed
Important outcomes
not assessed
Clinicians and
patients not involved
in setting research
agendas
Appropriate design
and methods?
Accessible
full publication?
Unbiased and
usable report?
Over 50% studies
designed without
reference to
systematic reviews of
existing evidence
Over 50% of studies
never published in full
Over 30% of trial
interventions not
sufficiently described
Biased underreporting of studies
with disappointing
results
Over 50% of planned
study outcomes not
reported
Over 50% of studies
fail to take adequate
steps to reduce
biases, e.g.
unconcealed
treatment allocation
50
%
Most new research
not interpreted in the
context of systematic
assessment of other
relevant evidence
50
%
85% Research waste = over $85 Billion / year
50
%
Mismatch of patients’ and researchers’
priorities for osteoarthritis of the knee
Tallon, Chard and Dieppe. Lancet, 2000.
Priority treatment outcome from a
survey of patients with rheumatoid
arthritis was not pain
Priority treatment outcome from a
survey of patients with rheumatoid
arthritis was not pain
It was fatigue
New Healthtalkonline
interview study
A chance to share views/experiences of
patient and public involvement in
research
If you’d like to contribute, contact
louise.locock@phc.ox.ac.uk
01865 289303
How can patients and the
public help to reduce
waste in research?
Chalmers I. Lancet 2000;356:774
Conclusions of a 2002 study
commissioned by INVOLVE
“There was widespread support for the concept
of a publicly available resource containing
information about ongoing clinical trials
designed for use by potential participants.”
Campbell M, Entwistle V, Langston A, Skea Z (2002). Scoping study to
explore the most appropriate way to produce and disseminate information
on the quality of randomised controlled trials for potential participants.
http://www.invo.org.uk/pdfs/pact_reportfinal0305.pdf
INVOLVE 6TH National Conference
11 November 2008
The UK Clinical Trials Gateway:
should provide information
about ongoing clinical research
in the NHS
Iain Chalmers
James Lind Initiative
For every ongoing trial being conducted
within the NHS, the UK Clinical Trials
Gateway should aim to provide access to:




a lay summary
the patient information sheet
the WHO 20-item dataset
the protocol, with links to the systematic
review(s) showing why the trial is needed
 the trial website (if one exists)
Reliable, user-friendly
information about specific
ongoing clinical trials is
still NOT generally available
What is the James Lind
Initiative and what has it
done to try to address some
of these problems?
2003The James Lind Initiative
Funded by the National Institute of Health
Research and the Medical Research Council
“to promote acknowledgement
of uncertainties about the
effects of treatments, and
research to address them.”
Programme of work of
The James Lind Initiative
1.
Identifying and publishing uncertainties about the
effects of treatments: UK Database of
Uncertainties about the Effects of Treatments
2.
Identifying patients’ and clinicians’ shared
priorities for research about the effects of
treatments: James Lind Alliance
3.
Explaining and illustrating the development of fair
tests of treatments in health care: James Lind
Library and Testing Treatments
The UK Database of Uncertainties
about the Effects of Treatments
Established to publish uncertainties about
the effects of treatments which cannot
currently be answered by referring to
relevant and reliable, up-to-date
systematic reviews of existing
research evidence
The UK DUETs Development Team
Mark Fenton
Editor, DUETs
Hazim Timimi
Update Software
UK DUETs draws on three main sources
•Patients', carers' and clinicians' unanswered
questions about the effects of treatments
•Research recommendations in reports of
systematic reviews and clinical guidelines
•Ongoing research, both systematic reviews in
preparation and new 'primary' studies
Programme of work of
The James Lind Initiative
1.
Identifying and publishing uncertainties about the
effects of treatments: UK Database of
Uncertainties about the Effects of Treatments
2.
Identifying patients’ and clinicians’ shared
priorities for research about the effects of
treatments: James Lind Alliance
3.
Explaining and illustrating the development of fair
tests of treatments in health care: James Lind
Library and Testing Treatments
Established in 2004
to see if it would work
Iain Chalmers (James Lind Library)
John Scadding (Royal Society of Medicine)
Nick Partridge (INVOLVE)
To increase the focus of the therapeutic
research agenda on questions and
priorities shared by patients and clinicians.
To promote Priority Setting Partnerships
involving patients and clinicians to
identify and promote their shared
priorities for therapeutic research.
To increase general awareness and
understanding of the need to refocus the
therapeutic research agenda.
Lester Firkins
Chair, JLA Strategy
& Development Group
Sally Crowe
Chair, JLA Monitoring &
Implementation Group
Katherine Cowan
Editor, JLA Guidebook
Patricia Atkinson
Administrator,
JLA Secretariat
Involving patients, carers and
clinicians in research priority setting
The JLA’s principles
•Inclusive
•Balance of perspectives
•Accessible to all
•Supportive
•Recognising a range of capacities and skills
•Transparent and democratic
•Data sharing
•Agreed protocol
•Declaration of interests
•Neutral facilitation
•Communication and feedback
JLA Priority Setting Partnerships
Completed
Current
•Asthma
•Urinary incontinence
•Vitiligo
•Prostate cancer
•Schizophrenia
•Type 1 diabetes
•ENT aspects of balance
•Life after stroke
•Eczema
•Tinnitus
•Cleft lip and palate
•Acne
•Childhood disability
•Dementia
•Dialysis
•Head and neck cancer
•Inflammatory bowel disease
•Lyme disease
•Multiple sclerosis
•Pressure ulcers
•Pre-term birth
•Sight loss and vision
Identifying and prioritising uncertainties
about the effects of treatment
Harvest ‘raw uncertainties’
Stage 1
Derive ‘indicative
uncertainties’
Stage 2
Stage 3
Stage 4
Stage 5
19,551
2555
Publish ‘indicative uncertainties’ in UK
DUETs
Prioritize uncertainties,
James Lind Alliance
Priority Setting Partnerships
Agree shortlist of highest
priority uncertainties
Notify highest priority uncertainties
to research funders
1678
264
Research priority themes
[across asthma,
incontinence, vitiligo, eczema, stroke, prostate cancer,
schizophrenia, aspects of balance, and type 1 diabetes]
• Assessment of long-term effects (wanted and
unwanted) of treatments
• Assessment of safety and adverse effects of
treatments
• Assessment of complementary and non-prescribed
treatments
• Assessment of strategies to improve early diagnosis
and treatments, and harmonisation of practice
• Assessment of the effectiveness and safety of selfcare
For more information…
www.lindalliance.org – become an Affiliate
www.JLAguidebook.org – practical guidance
@LindAlliance
From 1 April 2013,
the JLA will be incorporated in the NIHR
Evaluation, Trials and Studies Coordinating
Centre (NETSCC)
Tom Kenny, Pamela Young, Sarah Fryett
Questions relevant
to clinicians &
patients?
Low priority questions
addressed
Important outcomes
not assessed
Clinicians and
patients not involved
in setting research
agendas
Appropriate design
and methods?
Accessible
full publication?
Unbiased and
usable report?
Over 50% studies
designed without
reference to
systematic reviews of
existing evidence
Over 50% of studies
never published in full
Over 30% of trial
interventions not
sufficiently described
Biased underreporting of studies
with disappointing
results
Over 50% of planned
study outcomes not
reported
Over 50% of studies
fail to take adequate
steps to reduce
biases, e.g.
unconcealed
treatment allocation
50
%
Most new research
not interpreted in the
context of systematic
assessment of other
relevant evidence
50
%
85% Research waste = over $85 Billion / year
50
%
Publication (2007) after registration (1999)
Country
Size
Phase
Funder
Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM (2009). Trial publication after
registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med 6(9): e1000144.
“Studies that report positive or significant results
are more likely to be published and outcomes
that are statistically significant have higher odds
of being fully reported.”
PLoS ONE, August 2008;3:e3081
Alessandro Liberati
Because research results have not
been made public…
…patients have suffered and died
unnecessarily and resources for health
care and health research have been
wasted.
TGN1412
TGN 1412
Why is medical academia content
to acquiesce in biased
under-reporting of research?
What is the position of the
Academy of Medical Sciences?
2006 letter to Prof John Bell,
President, Academy of Medical Sciences
What should be done?
The public needs to be made
aware of how the resources they
provide for research are being
wasted.
What should be done?
The public needs to be made
aware of how the resources they
provide for research are being
wasted.
The public needs to hold the
research community to account,
and be critically involved in
research, from agenda setting to
dissemination of results.
Programme of work of
The James Lind Initiative
1.
Identifying and publishing uncertainties about the
effects of treatments: UK Database of
Uncertainties about the Effects of Treatments
2.
Identifying patients’ and clinicians’ shared
priorities for research about the effects of
treatments: James Lind Alliance
3.
Explaining and illustrating the development of fair
tests of treatments in health care: James Lind
Library and Testing Treatments
www.jameslindlibrary.org
2006
2011
Write your suggestions
on the postcards in
your conference packs.
Pin them up on the
‘Burning Issues’
noticeboard.
Promote research on the effects of treatments…
Promote research on the effects of treatments…
…but only if it meets scientific and ethical principles
www.testingtreatments.org
“Bad Science introduces the basic scientific principles to
help everyone become a more effective bullshit detector.”