Evidence in action – moving from
guidance to review
Sophie Robinson, Toni Price, Caroline Miller
National Institute for Health and Clinical Excellence
(NICE)
Health Libraries Group conference July 2010
What we’re going to talk about
• Role of information specialists in producing and
reviewing NICE technology appraisals;
– how information specialists are involved in the
scoping and review processes
– information gathered by information specialists
– Skills & challenges involved in supporting technology
appraisals
What is NICE?
• The National Institute for Health and Clinical Excellence
(NICE) is the independent organisation set up in 1999 based
in England responsible for
– providing national guidance on the promotion of good
health and the prevention and treatment of ill health
– setting quality standards
– managing NHS Evidence
• NICE makes recommendations to the NHS on:
– new and existing medicines, treatments and procedures
– treating and caring for people with specific diseases and
conditions.
• NICE makes recommendations to the NHS, local authorities
and other organisations on:
– how to improve people’s health and prevent illness and
disease.
The information services team at NICE
• A team of 20 information professionals based in London
and Manchester
• Provide information support to meet the information
needs of NICE staff and to help produce NICE guidance
• Support topic selection, appraisals scoping & reviews,
short guidelines, interventional procedures, quality
standards, quality & outcomes framework, devices &
diagnostics, public health, NICE taxonomy etc
• Liaison leads for all teams in Institute
• Library resources – journals, databases, books, ILLs,
information skills training, current awareness service
What is a technology appraisal?
• Technology appraisals are recommendations on the
use of new and existing medicines and treatments
within the NHS, such as:
– medicines
– medical devices (for example, hearing aids or
inhalers)
– diagnostic techniques (tests used to identify
diseases)
– surgical procedures (such as repairing hernias)
– health promotion activities (for example, ways of
helping people with diabetes manage their
condition).
Types of appraisal
• Single technology appraisals (STA) specifically
designed for the appraisal of a single product, device or
other technology with a single indication where most of
the relevant evidence lies with one manufacturer
– eg Everolimus for the prevention of organ rejection in kidney
transplantation (in progress)
• Multiple technology appraisal (MTA) - includes more
than one product, device or technology or more than
one indication or more than one manufacturer or
sponsor
– eg Immunosuppressive therapy (basiliximab, daclizumab,
tacrolimus, mycophenolate (mofetil and sodium) and sirolimus)
for renal transplantation in children and adolescents TA99
Technology appraisals
• Technology appraisal recommendations are based on a
review of clinical and economic evidence.
• Clinical evidence measures how well the medicine or
treatment works.
• Economic evidence measures how well the medicine
or treatment works in relation to how much it costs the
NHS - does it represent value for money?
• Independent academic assessment groups carry out
the systematic review process for appraisals
• Obligation for NHS organisations to fund and resource
medicines and treatments recommended, usually within
three months of NICE issuing guidance
The technology appraisal process
Scoping & question setting,
consultation
Evidence review and independent
appraisal
Appraisal committees and open
consultation
Final conclusion, [opportunity for
appeal] and publication
Update of appraisal
The role of the information specialist
in the technology appraisal process
• Initial information support at topic selection stage –
support with identifying related guidance for briefing
notes for topic selection consideration panels
• Support for production of ‘draft scope’ – 2-4 page
document setting out parameters of topic in question,
used for consultation
• Collaborate with a technical analyst in production of
draft scope and provide other information support
needed throughout the production process
• Monitor topic right through to publication – licensing
updates and key new trials
• New topics and review topics are ‘scoped’
Contributing to the draft scope
• Collate information on
– Health condition/disease/behaviour in question
– Population affected including epidemiological
information, equality issues
– Technology in question including licensing
information
– Evidence based guidance on health condition /
technology
– Relevant key trials
• Not a systematic search at scoping stage
• Record information via NICE intranet on a ‘scoping
page’ which is updated
Information specialist skills used in
production of technology appraisal
• Retrieval, interpretation, synthesis, summarising &
presenting information
• Elements of critical appraisal to summarise key
evidence
• Collaborative with technical analyst
• Identifying issues for consideration - eg populations,
comparators, links with other appraisals or NICE
guidance
Challenges of contributing to the
scoping process
• Risk of working in advance of licensing and topics not
coming to fruition
• Uncertainty around licensing requires updates to
information
• Project management timelines
• Understanding diseases, stages, lines of treatment
• Identifying relevant key trials
• Identifying key issues eg whether an appraisal is
feasible (faller’s clinics where no definition or evidence
and topic did not progress)
• Knowing when to stop searching for information
Reviewing Technology Appraisals
• All published technology appraisal guidance is given a
date (between 1-5 years) at which the guidance will be
considered for review.
• These reviews are known as Review Proposal Projects
or RPPs.
• The aim of the review process is to decide whether or
not the guidance needs to be updated.
• Information services play a central role, by providing an
evidence base upon which to make the decision.
RPP process overview
A Review Proposal consultation
document is written by the
Information Specialist lead with
assistance of Technical Lead
Guidance executive agrees the paper and
the proposal is consulted on with the RPP
consultees and commentators
GE agrees the final decision and
consultees and commentators
are advised
Proposal options include:
- Appraisal should be updated
- Defer date for review
- Incorporate in a clinical guideline
- Transfer guidance to static list
- Appraisal combined with a new
appraisal that has recently been
referred
Consultation responses are
reviewed and incorporated into a
decision paper
Decision actioned
Information Services involvement
• IS staff carry out the bulk of the work for
development of RPPs, gathering and evaluating
information to see if there is significant evidence to
warrant a review.
• This involves literature searching and writing
recommendation papers for NICE’s Guidance
Executive (senior management team) to consider.
• The information specialist has to work to specific
timelines and processes developed and overseen by
a Project Manager from Technology Appraisals.
Information gathering
• The information specialist will familiarise themselves
thoroughly with the appraisal to be reviewed, with the
disease area concerned and with the drug or device.
• They will also search for new evidence including:
– New indications for drugs included in the original
guidance and related new products.
– Information on the progress of ongoing randomised
controlled trials.
– Update to the searches used in the original Health
Technology Assessment report for the appraisal.
Results
• The results of the searches are sifted by the information
specialist to identify any new evidence that could trigger
a review of the appraisal.
• For example, the IS may find a key clinical trial that
could lead to the deferral of the review of the appraisal
awaiting trial results.
• The published results of an RCT may contradict one of
the recommendations in the original appraisal.
• A new indication or new related drug may mean the
information in the appraisal is out of date.
Proposal paper for Guidance
Executive (GE)
• Aims to summarise and collate relevant information
found in a report template, which is presented to
NICE’s senior management team.
• Recommends a proposal for what should happen to
the appraisal (one from a list of standard options).
• A technical analyst will analyse and report on the
results of the searches and make the final decision on
the recommendation in conjunction with colleagues.
Possible recommendations
• Review of the appraisal should be:
– planned into the appraisal work programme;
– decision to review will be deferred until X (for
example, to await the results of a key clinical trial);
– combined with the review of a related technology
appraisal or with a related new topic referred to
NICE;
– incorporated or updated into an on-going clinical
guideline;
– transferred to static guidance list.
Proposal paper for Guidance
Executive
• The paper is agreed by an Associate Director from the
appraisals team.
• The paper is then submitted to the senior management
team (Guidance Executive) who consider the evidence
and make the final proposal for the recommendation.
• NICE then consults on the proposal with consultees
and commentators (manufacturers, patient groups,
NHS Trusts, Royal Colleges and other interested
parties).
• The proposal is posted on the NICE website.
Decision paper for Guidance
Executive
•
•
•
•
•
At the end of the consultation period a summary of
consultation responses is collated.
This is compiled by the information specialist with
input from the technical analyst and project manager
where necessary.
Comments from consultees are tabulated and
responses are given.
The paper is agreed by an Associate Director and
then goes to GE for consideration.
Consultees are advised of the final decision and this
is also posted on the NICE website.
Information Specialist work
•
•
•
•
•
•
Timelines
Reading and understanding the existing guidance
Identifying the challenging areas for search purposes
Searching
Understanding the findings and the implications
Communication with analyst in advance of presenting
the findings
• Identifying the best recommendation, according to the
findings
Challenges of TA99
• Sifting for relevant information - how relevant might data
on adults be?
• Choosing what to recommend as the way forward –
using and understanding the evidence base
• Information synthesis – making a robust case for review
• Time – not unlimited
• Transparency
RPP skills
• Process driven work
• Searching – finding the key information
• Judgement - understanding and interpreting the
findings (for example what’s pivotal, what will make a
material difference to our existing guidance)
• Decision making based on the evidence – make the
proposal
• Project management, and being an active team player
• Communications skills
Summary
• Information specialists use a wide range of skills when
supporting the scoping process and the reviews of
health technology appraisals
• Information services team do 3 RPPs and 7 scopes per
month in addition to work for other Directorates at NICE
• Any questions?
• caroline.miller@nice.org.uk; toni.price@nice.org.uk
sophie.robinson@nice.org.uk;
• www.nice.org.uk