HEALTH TECHNOLOGY
ASSESSMENT
Has the UK got it right?
Michael Drummond
Centre for Health Economics
University of York
United Kingdom
Disclosure of Interests
• Chair of the NICE Guidelines Review
Panel on Primary Care
• Centre for Health Economics is one of the
independent review groups for the NICE
Technology Appraisal process
• Recipient of grants from the NHS R and D
• Consultant for industry
Outline of Presentation
• Structure of health technology assessment
(HTA) in the United Kingdom (UK)
• Key features of the HTA process
• How do we do?
• Conclusions
Structure of HTA in the UK
• NCCHTA:
– Based in Wessex.
– Commissions a wide range of empirical and
theoretical projects.
– Administers contracts for NICE Technology
Assessment Reviews (TARs).
• NICE:
– Programmes in Technology Appraisals, Clinical
Guidelines and Public Health.
• SMC:
– Produces evaluations of all new medicines launched
in Scotland.
Other Players in HTA in the United
Kingdom
•
•
•
•
•
•
NHS methodology programme
Activities in Wales and Northern Ireland
MRC and ESRC projects/fellowships
Private research foundations
Manufacturers of drugs and devices
Health authorities
Key Features of the HTA Process
•
•
•
•
Selection of topics
Assessment procedures
Stakeholder involvement
Methodological development
Key Features of the HTA Process
(continued)
• Developing a critical mass of skilled
personnel
• Maintaining international links
• Implementation of HTA findings
• Transparency in decision-making
Cross-National Comparisons of
Technology Assessment Processes
• Comparison of VATAP (USA), NICE (UK), CCOHTA
(Canada) and AETS (Spain)
• Considered:
(i) the reasons for the choice of topics,
(ii) the types of technologies assessed,
(iii) the methods of assessment and
(iv) the outcomes of assessments
Garcia–Altés et al, Int. J. Tech. Assess. Health Care 2004
Health Policy Issues dealt with in
the Assessments
Topic
VATAP
NICE
CCOHTA
AETS
Is the assessment the result of a formal
prioritization process?




Is the assessment the result of a political
decision?




Did the assessment include economic
evaluation methods?




Did any stakeholders participate in the
assessment?




Did the assessment include additional
funding besides the own resources of
the organization?




Source: Garcia–Altés et al, Int. J. Tech. Assess. Health Care 2004
Selection of Topics
• In England the Department of Health sets
NICE’s agenda.
• In Scotland the SMC considers every new
drug.
• The NCCHTA and NHS Methodology
Programme consult widely on topics, but
then commission projects.
Selection of Topics:
Observations
• Unlikely to be cost-effective to assess
every new technology
• The mechanisms to select topics in
England are not totally transparent
• Sometimes the choices have been strange
(e.g. appraisals when a major trial will
report in 1-2 years)
Selection of Topics:
Observations (continued)
• Procedures are required to ensure
appropriate use of technologies that are
not appraised (e.g. drugs in the
Netherlands).
• The approach followed by NICE allows
groups of similar technologies to be
compared. (This has both advantages and
disadvantages.)
Assessing Drugs in the Same
Class Together
• Most common with NICE in the UK,
although there are a few examples from
elsewhere (e.g. Canada).
• Main problem is the lack of head-to-head
clinical studies.
• Puts the onus on manufacturers to
demonstrate extra benefits to justify a
premium price.
NICE’s Single Technology
Appraisals
• A new ‘fast track’ procedure introduced in
response to concerns over the time taken
by NICE’s standard approach.
• So far applies to drugs, in the main cancer
drugs.
• Will place more emphasis on analyses
submitted by the manufacturer and
incorporate less external review.
NICE’s Single Technology
Appraisals
• May suffice in situations where the number
of comparators is limited
• Raises further questions about the
methods for prioritising topics
• Raises issues about burden of proof and
responsibility for the results of the
appraisal
Assessment Procedures
• The majority of HTA agencies undertake
assessments in-house, although probably all
commission some work outside (e.g. in Canada,
CCOHTA spends 25% of its budget outside).
• In England, NICE places considerable emphasis
on independent review by academic groups
• By-and-large the independent review groups
apply ‘Cochrane-style’ methods.
Is Independent Review Cost-Effective?
• Re-affirms the ‘arms length’ nature of HTA
(assessment ≠ appraisal)
• More transparent and may help resolve disputes
when multiple products are being considered,
but:
• The Scots claim they reach the same decisions
at a fraction of the (assessment) cost
• Systematic reviews place emphasis on RCTs as
compared with other study designs
• Sometimes the economic model does not follow
from the systematic review (‘a game of 2 halves’)
Methods
• Sample of all TARs published between
January 2003 and July 2005.
• Data extracted on:
– the TAR topic;
– the clinical effectiveness measure;
– the approach to pooling;
– the measure of economic benefit;
– the use, or non-use, of the systematic review
in the model.
Results (1)
• 38 TARs published in the period studied,
all of which contained a systematic review.
• Most of the economic evaluations were
cost-utility analyses (32), reflecting NICE’s
appraisal guidelines.
• The other studies were cost-effectiveness
analyses (4) and cost-minimisation
analyses (2).
Results (2)
• In 15 cases, the clinical data were not
pooled in the systematic review, owing to:
(a) heterogeneity; or (b) the limited
number of studies.
• In 5 cases, a new model was not
constructed, because: (a) the cost
implications were thought to be trivial; (b)
lack of evidence; or (c) the existence of
satisfactory models provided by industry.
Results (3)
• In cases where there was both a pooled
estimate from the review and an economic
model:
– The review was always used when survival
was the endpoint (e.g. in the costeffectiveness studies);
– It was less often used when the endpoint was
QALYs (e.g. in the cost-utility analyses).
Reasons For Not Using The
Systematic Review In Cost-utility
Analyses
• Preference-based QoL measures are only
occasionally used in clinical studies
• The summary measure of clinical
effectiveness does not facilitate the
calculation of QALYs
Examples
• New drugs for epilepsy (adults):
– clinical effects assessed in terms of total or partial
reduction in seizures;
– These were classified as partial and total response
and a utility value to each state.
• Drug therapy for attention-deficit hyperactivity
disorder (ADHD):
– effectiveness measure was points change on the
Connors Hyperactivity Scale;
– economic evaluation used response/non-response
and assigned a utility to each state.
Stakeholder Involvement
• Stakeholders can include manufacturers,
professional organisations, health authorities,
academic groups and patient organisations.
• All HTA agencies have some stakeholder
involvement, but NICE is probably at the allinclusive end of the spectrum.
• Stakeholder involvement is resource-intensive
but may (i) lead to better assessments; (ii)
reduce the number of appeals and (iii) lead to
better implementation of HTAs.
Are Cochrane-Style Reviews
Always Helpful?
Respite care for frail older people: an appraisal of
effectiveness and cost-effectiveness (Mason et al, in
press)
Number of hits (in the literature review)
12,927
Papers retrieved
379
No. of potential studies
171
Potential economic evaluations
41
Actual number of economic evaluations (EEs)
22
Number of EEs of respite care meeting criteria for inclusion 5
Number of such studies undertaken in the UK
1
Methodological Development
• No doubt that HTA processes in the UK
have stimulated methodological
development; e.g.: mixed treatment
comparisons; probabilistic models
• Several methodology TARs
• The NHS Methodology Programme has a
good track record in funding projects
relevant to HTA
Developing A Critical Mass Of
Skilled Personnel
• The steady funding for NICE TARs has
enabled academic units to build a critical
mass of skilled personnel.
• Training fellowships are available from the
MRC, ESRC and the NHS R and D
• The UK is probably better placed than
most countries
Maintaining International Links
• We could probably learn more from
comparing and contrasting the different
approaches for drugs in England and
Scotland.
• On the international level, the UK seems
to be fairly well-connected. After Canada,
the UK has the most members of HTAi
(the international society).
Implementation of HTA Findings
• No health care system does this particularly well.
• Characteristics of the broader health care
system greatly influence the opportunities for
implementation, e.g. whether funding is largely
general, or tied to the use of particular
technologies.
• Restrictions in use (to most common guidance
from NICE) pose more implementation
challenges than simple yes/no decisions.
Evidence On The Implementation
Of NICE Guidance
• Sheldon et al, BMJ 2004; 329 (30
October)
• Audit Commission Report, September
2005
• Conn, F. Thesis, LSE, 2006
• Daily Mail – every issue!
NICE Outcomes
Hospital use of Paclitaxil & Docetaxel
Source: Sheldon et al, BMJ, April 2005
Orlistat & Alzheimer drugs
NICE Outcomes
•
•
NICE commissioned report on 28 guidance areas covering 33 (1/3 of all
recommendations to start of 2005) recommendations to assess impact
using IMS data and UK Hospital Pharmacy Audit data
30% recommended for 1st line use; 57% recommended for 2nd/3rd line
use or specific patient groups; 12% (#4) rejected (but one entered a
risk sharing agreement)
Source: NICE commissioned Report from Abacus International, 2005
NICE Outcomes
Source: NICE commissioned Report from Abacus International, 2005
Implementation Of NICE Guidance
• Variable, by technology and location
• The formal requirement, to implement TAs
within 3 months, probably has an impact
• Biggest problem is finding funding within a
resource-constrained system. (The new
tariffs will not reflect NICE guidance)
Implementation Of NICE Guidance
(continued)
• Local professional involvement and good
financial systems are important
• Almost half of GPs are welcoming of NICE
• NICE is perceived (by GPs) as being
independent of industry but not of
government
Implementation Of HTA Findings:
what can be done?
• Develop an implementation plan for each
HTA
• Produce more advice on what to
discontinue, as well as what to adopt
• Link funding streams more closely to
guidance (although not easy in the NHS)
• Increase the monitoring of the adoption of
guidance
Transparency in Decision-Making
• In general all HTA increases transparency
• NICE is among the most transparent of
HTA agencies
• The use of commercial-in-confidence data
in technology appraisals can reduce
transparency (Drummond, Applied Health
Economics and Health Policy, 2004)
Conclusions
• Overall we do not do too badly in the UK
• Although existing procedures for assessment,
involvement of stakeholders and production of
guidance could be streamlined, they are
basically sound
• Priorities for selection of topics and mechanisms
of implementation of findings need more
attention
• In resource-constrained systems, critics often
confuse the message with the messenger