NCEC
Economic
Evaluation
Training
Shelley O’Neill
Health Technology
Assessment
Directorate
Presentation outline
1) Introduction to Health Technology Assessment
(HTA)
2) NCEC guideline prioritisation criteria
3) Conducting a budget impact assessment
4) Reviewing economic literature
Health Technology Assessment
Objective:
To inform safe and effective health policies that are
patient focussed and achieve best value
HTA is a “decision support tool”
Health Technologies
Includes a wide range of interventions used in healthcare and health
promotion
−Pharmaceuticals (Drugs)
−Vaccines
−Medical Devices
−Diagnostics
−Medical and surgical procedures
−Public health activities
Includes the systems within which health is protected and maintained
Why do we need HTA?
• Introduce technologies speedily with proven significant health
•
benefits
Prevent the introduction of technologies which fail to meet the
requirements of evidence-based analysis
The best interests
of the individual
Fair & equitable
allocation
of resources
for society
Scarcity means that choices must be made !
Health Technology Assessment
Science
Patient
wishes
Industry
claims
Decision
making
“HTA is a decision support tool”
Multidisciplinary process, summarises information about
– Safety
– Clinical and cost-effectiveness
– Budget impact
– Organisational impact / resource implications
– Social and ethical issues
related to use of a health technology in a systematic,
transparent, unbiased and robust manner
HTA for consumers/patients
“rationing”
HTA for Clinicians…….
..a clinical purist?
..or a financial
realist?
Economic Evaluation
Cost effectiveness analysis:
A form of economic evaluation
which simultaneously assesses the
costs and consequences (benefits)
of an intervention
Compare the costs and
consequences (effects) of
technology A vs technology B.
HIQA: compare the costs per life
year gained (LYG) or quality
adjusted life years gained (QALYs)
Economic Evaluation
Questions addressed in an economic evaluation:
1. Can it work (efficacy)?
2. Does it work (effectiveness)?
3. Is it worth doing (efficiency)?
4. How much will it cost (affordability)?
Outcome measure
Life years gained:
Number of years a patient’s/individual’s life is prolonged as a result
of a particular intervention.
Quality-adjusted life years (QALY):
Number of years a patient’s/individual’s life is prolonged as a result
of a particular intervention, incorporating adjustments for the
quality of that life (morbidity)
• Universally applicable to all patients and diseases
• Allows comparisons between different health programmes
• Useful for decision makers
• QALYs allow for measurement of values or preferences for a
particular health state
Cost-effectiveness analysis
Choice
A
Intervention
• Compares costs and consequences of two technologies A
and B
Costs,
BenefitsA
Cost A – Cost B
B
Comparator /
routine practice
Effect of A – Effect of B
Costs,
BenefitsB
• Incremental cost-effectiveness ratio (ICER)
= Cost / QALY
Economic evaluation
Evaluation Type
Costs
Benefits
Cost-effectiveness
analysis (CEA)
€
Natural units
(e.g. life years gained)
Cost-utility analysis
(CUA)
€
Health Status (e.g. QALYs or
DALYs gained)
Cost-benefit analysis
(CBA)
€
€
(e.g. based on willingness-to-pay)
Cost-minimisation
analysis (CMA)
€
Assume benefits to be equivalent
Economic evaluations are usually in the form of CEA or CUA.
Cost-effectiveness plane
Cost (€)
Q4
Intervention less
effective and more
costly
Q1
Intervention more
effective and
more costly
Effect (QALY)
Intervention less
effective and less
costly
Q3
Intervention more
effective and less
costly
Q2
Cost-effectiveness plane
The incremental cost-effectiveness ratio is usually represented
graphically as a line passing through the origin on the cost
effectiveness plane
Cost (€)
Q4
Q1
Accept
(probably)
€10,000/QALY
Effect (QALY)
Q3
Q2
Cost-effectiveness plane
The incremental cost-effectiveness ratio is usually represented
graphically as a line passing through the origin on the cost
effectiveness plane
Cost (€)
Q4
€60,000/QALY
Q1
Reject
(probably)
Effect (QALY)
Q3
Q2
Cost-effectiveness plane
The line passing through the origin represents our ‘acceptable’
cost-effectiveness ratio. That is our maximum (or threshold)
willingness-to-pay for a unit of effect (life year or QALY).
Cost (€)
Q4
Q1
Effect (QALY)
Q3
Q2
Willingness to pay
Acceptability of a technology with an ICER value in
the region of the threshold will be influenced by:
• Degree of uncertainty in calculating the ICER
• The innovative nature of the technology
• Particular features of the condition and population receiving
the technology
• The wider societal costs and benefits
NCEC Guideline Screening and Prioritisation Criteria
Criteria 3 Economic Impact
Would implementing this guideline have a substantial
budget impact on the healthcare system ?
Are there potential cost savings to be realised if the
guideline is implemented?
Is there national or international cost-effectiveness
evidence to support implementing the guideline?
NCEC Guideline Screening and Prioritisation Criteria
Criteria 3 Economic Impact
Would implementing this guideline have a substantial budget
impact on the healthcare system?
• Have the resource implications of implementing the
guideline been considered?
• Have the resources required for any initial set up or roll
out phase been considered?
• Have the cost of these resources to the publicly-funded
system been estimated?
Budget impact
Would implementing this guideline have a substantial
budget impact on the healthcare system?
•
Comprehensive economic evaluation tends to be both time and
labour intensive
•
Budget impact analysis looks at the added financial impact of
implementing a new clinical guideline over a finite period (this may be
sufficient in most cases)
•
Typically only the direct costs to the publicly-funded health and social
care system (HSE) in Ireland are included
•
Indirect costs (absenteeism from work, disability, need for long term
care etc.) are important in instances where significant budget
implications for other publicly-funded services or transfer payments
are anticipated.
Budget impact
Budget impact matters – even when a technology
is deemed cost effective ?
Budget impact
Assessing the budget impact can be broken into three distinct
steps:
1. Identifying the resource use that may change
2. Estimating the size of these changes
3. Determining the relevant costs for these changes
1. Identifying the resource use that may change
It is important to determine the treatment or intervention pathway:
• How do the people receiving the intervention interact with the
health services?
• Who carries out the intervention and where?
• How is onward treatment managed and monitored?
1. Identifying the resource use that may change
• Are there any initial one-off resource requirements?
(capital investment, staff training)
• Will there be additional service utilisation arising from the
intervention (Additional testing (e.g. biopsies), treatment,
follow-up/monitoring, Primary/secondary/tertiary care)
• Technology (Equipment purchase, ICT upgrading, test
kits, Laboratory usage)
• Harms/side-effects: will there be additional treatment of
harms (medicines, surgical, alternative treatment
pathway)
Example from EWS
What are the ongoing material costs?
The early warning score chart is likely to replace currently used
charts. These vary across sites with some consisting of a single
sheet, however the change to the NEWS chart will have a
negligible cost implication.
Budget impact
Assessing the budget impact can be broken into three distinct
steps:
1. Identifying the resource use that may change
2. Estimating the size of these changes
3. Determining the relevant costs for these changes
2. Estimating the size of these changes
•
When evaluating the economic impact, it is the incremental impact
that should be considered, the total cost of implementing the
national guideline less what would have been spent on the current
standard of care.
•
The comparator used should be ‘routine care,’ that is, the current
or most widely used clinical practice in Ireland; in some cases this
may be a mix of a number of different practices
•
Who will receive the intervention? How many patients/
individuals?
•
This could be presented as a range, to incorporate variation in the
estimate or possible different scenarios.
2. Estimating the size of these changes
Who will receive the intervention?
The target population!
The target population characteristics
• age & sex
• socio-economic status
• life expectancy
• co-morbidities
• treatment response
• etc
will dictate how much of the intervention is required and will also
impact on the treatment response.
2. Estimating the size of these changes
Data we might be interested in and potential sources:
• Morbidity
oHospital In-Patient Enquiry Scheme (HIPE) www.hpo.ie
oPrimary Care Reimbursement Service (PCRS) www.hse.ie/eng/staff/PCRS/
oNational Cancer Registry www.ncri.ie/
• Mortality
oVital Statistics www.cso.ie
oNational Paediatric Mortality Register www.sidsireland.ie/
oNational Drug-Related Deaths Index www.hrb.ie/
oNational Perinatal Reporting System (NPRS) www.hpo.ie
31
2. Estimating the size of these changes
•Health service utilisation
oHIPE & PCRS
oBreastCheck www.breastcheck.ie & CervicalCheck www.cervicalcheck.ie/
oHealth Protection Surveillance Centre www.ndsc.ie/hpsc/
oNational Psychiatric Inpatient Reporting System www.hrb.ie/
• Demographic (Census) www.cso.ie
• Exposure
oNational Roads Authority www.nra.ie/
oOffice of Tobacco Control www.hse.ie/eng/about/Who/TobaccoControl
•HIQA catalogue of health information sources:
www.hiqa.ie/resource-centre/professionals/health-information-sources
32
Budget impact
Assessing the budget impact can be broken into three distinct
steps:
1. Identifying the resource use that may change
2. Estimating the size of these changes
3. Determining the relevant costs for these changes
3. Determining the relevant costs for these changes
•
Technology/intervention- (Data from the manufacturer)
•
Staff- include pay at mid-point of pay scale, employers’
PRSI, imputed pension cost, and overheads
www.hse.ie/eng/staff/Benefits_Services/pay/
•
Hospital in-patient and daycase procedure costs
www.casemix.ie/
•
Drugs covered on the community schemes (PCRS)
www.hse.ie/eng/staff/PCRS/
•
Hospital finance departments
3. Determining the relevant costs for these changes
•
Annual depreciation of any capital costs should be
included in the analysis.
•
Include any maintenance contracts
•
If using cost from literature:
o
Retrospective health costs must be inflated to current prices
using the Consumer Price Index (CPI) for Health.
o
Where costs are applied from other countries, all costs must be
converted to euro using Purchasing Power Parity indices (PPP).
o
If transferring costs from another currency, the inflation should
be calculated using the CPI for the local currency prior to
conversion to euro using PPP. www.oecd.org/std/prices-ppp/
Staff training example from EWS
What are the initial set up costs? Staff Training
It was estimated that in total 20,500 staff will require training,
training takes approx 8.5 hrs. The approximate cost for staff
time spent training is €7.3million.
A ‘train the trainer’ model was used for training with approx
300 staff delivering training, at 8hrs per sessions and each
trainer delivering 6 or 7 sessions. The staff time cost involved
to deliver training is an estimated €172,000.
These are opportunity costs, that is diverting staff members
from their usual activities to attend and provide training,
rather than an actual cash cost to the HSE.
NCEC Guideline Screening and Prioritisation Criteria
Criteria 3 Economic Impact
Are there potential cost savings to be realised if the guideline is
implemented?
• Are there any potential cost savings due to changes in the use
of resources?
• Have the benefits from improved outcomes been quantified
and the associated costs or savings been estimated?
Cost savings
Are there potential cost savings to be realised if the guideline is
implemented?
• Savings from treatment avoided due to improved health
outcomes (e.g. fewer bed days), a reduction in adverse events
or stopping or changing current practice (different material
costs).
• Particularly relevant for preventive care – use of intervention
may result in reduced health service use
• Although introduction of a new guideline may lead to a
reduction in staff requirements, it may be difficult to realise
any potential savings (e.g., redeployment of staff).
Cost savings example from NEWS
What are the cost savings from improved outcomes?
Introducing a NEWS will improve patient outcomes by reducing
the number of unplanned admissions to ICU and reducing the
number of cardiac-respiratory arrests.
ICU additional cost = €1,316 per day extra general acute ward.
Additional days on ward = 5
A saving of €6580 per patient not admitted to ICU.
In 2011, approx 3,750 inpatients were diagnosed with a cardiac
or respiratory arrest, where this was not the main reason for
their admittance to hospital.
Assuming a 26% reduction in those hospitals currently without
a EWS (based on literature) there could be €4.2million or 3,200
ICU bed days saved per year.
NCEC Guideline Screening and Prioritisation Criteria
Criteria 3 Economic Impact
Is there national or international cost-effectiveness
evidence to support implementing the guideline?
• Is a summary of the cost-effectiveness evidence
presented? Is this generalisable or relevant to the Irish
healthcare setting?
• Has this evidence been gathered using systematic
searching methods and are these methods documented?
Review of economic literature
Is there national or international cost-effectiveness
evidence to support implementing the guideline?
• Detailed guidance provided in the recently published
draft guidelines for the retrieval and interpretation of
economic evaluations of health technologies
• Currently out for public consultation
http://www.hiqa.ie/getting-involved/consultations/HTAguidelines
Review of economic literature
Is there national or international cost-effectiveness
evidence to support implementing the guideline?
• Has intervention/drug been assessed by HIQA or NCPE?
• Systematic, transparent, unbiased and robust, evidencebased review is needed
• Similar to clinical effectiveness/guideline review
• Same methods for defining search question (PICOs) and
reporting search strategy (inclusion/exclusion criteria,
flow chart etc.)
Search for economic evidence
•
Systematic literature search to identify cost effectiveness evidence
should be reproducible, thorough and transparent, consistent with
the search for clinical effectiveness/guidelines
•
Specialised databases (DARE, NHS EED, HTA Database
www.crd.york.ac.uk/CRDWeb, www.thecochranelibrary.com).
• Search should be performed using the major health search engines
such as MEDLINE and EMBASE using economic search filters
(available www.york.ac.uk/inst/crd/intertasc/econ.htm)
Identifying relevant studies
•
•
•
Is it current? older studies may include outdated
practices
How applicable is the study to the Irish context? Are
the populations, interventions and healthcare
system similar to the Irish setting?
Welte ‘knockout criteria’
o The relevant technology is not comparable to the one
that shall be used in the decision country
o The comparator is not comparable to the one that is
relevant to the decision country
o The study does not possess an acceptable quality
Appraising the evidence
Appraising the evidence:
• Is the study of adequate quality?
• Are there major limitations in the study design
or in the economic modelling?
Available tools in considering the quality of economic
studies (CHEC-list, BMJ, Philips)
Assessing Relevance and transferability
ISPOR
1.
2.
3.
4.
5.
6.
7.
8.
Relevance (population, interventions, outcomes, context)
Credibility (external, internal & face validity)
Model Design (type, assumptions, structure)
Data (values & how they were obtained)
Analysis (uncertainty)
Reporting (sufficient detail)
Interpretation (fair and balanced)
Conflict of interest (steps taken)
Synthesising and summarising the results
• Can any inconsistencies in the results be
explained? (e.g. different settings, perspectives
used, prevalence rates, input costs)
• What are the possible inferences that may be
drawn for the likely cost-effectiveness of the
intervention in the Irish setting?
• Summarise limitations and applicability of the
studies
Practical Exercise
From reading the selected Appendix XV for the Prevention and
Control MRSA – A National Clinical Guideline
Consider if each of the prioritisation criteria are met?
•Would implementing this guideline have a substantial budget
impact on the healthcare system ?
•Are there potential cost savings to be realised if the guideline
is implemented?
•Is there national or international cost-effectiveness evidence
to support implementing the guideline?
Appraisal of Clinical Guidelines
Relevant criteria
10. Systematic methods have been used to search for evidence
on effectiveness and cost-effectiveness to ensure that the
clinical guideline is based on best available evidence. The full
search strategy should be clearly outlined
14. The health benefits, side effects, risks, cost-effectiveness,
resource implications and health service delivery issues have
been considered in formulating the recommendations
23. The potential budget impact and resource implications
(equipment, staff, training etc.) of applying the
recommendations have been considered
Guidelines
Guidelines for the
Budget Impact
Analysis of Health
Technologies in
Ireland
2010
www.hiqa.ie
Useful websites
Ireland
–HIQA HTAs/Guidelines: www.hiqa.ie
–National Centre for Pharmacoeconomics: www.ncpe.ie
–NCEC Guideline Developers Manual:
http://www.patientsafetyfirst.ie/images/stories/docs/ncec_
guidedevmanual_v2.pdf
International Collaborations and Organisations
–http://www.eunethta.eu/
–http://www.inahta.org/
–http://www.htai.org/
–www.euroscan.org.uk
Databases
–http://www.crd.york.ac.uk/crdweb/
–http://www.hta.ac.uk/
–http://www.inahta.org/HTA/Database/
HTA explained
–http://www.nlm.nih.gov/nichsr/hta101/hta101.pdf
Further Reading
Silvia Evers et al. Criteria list for assessment of methodological quality of economic
evaluations: Consensus on Health Economic Criteria. International Journal of Technology
Assessment in Health Care, 21:2(2005), 240–245
J Caro et al. Questionnaire to Assess Relevance and Credibility of Modeling
Studies for Informing Health Care Decision Making: An ISPORAMCP-NPC Good Practice Task
Force Report. Value in health 17 (2014) 174-182
Z Philips et al. Review of guidelines for good practice in decision-analytic modelling in health
technology assessment. Health Technology Assessment 2004; Vol. 8: No. 36 (Appendix 3
Quality assessment in decision-analytic models:
a suggested checklist)
Robert Welte et al. A Decision Chart for Assessing and Improving the Transferability of
Economic Evaluation Results Between Countries. Pharmacoeconomics 2004; 22 (913) 857876.
Ron Goeree et al. Transferability of health technology assessments
and economic evaluations: a systematic review of approaches for assessment and
application. ClinicoEconomics and Outcomes Research 2011:3:89-104
MF Drummond et al. Guidelines for authors and peer reviewers of economic
submissions to the BMJ. BMJ 1996; 313:276-83
Thank You