1
4
5
9
Economic Evaluations –
What they are and what they
are not
Moira Napper
Health Economics Research
Unit
University of Aberdeen
HERU is supported by the Chief Scientist Office of the Scottish Executive Health Department.
The author accepts full responsibility for this talk.
Issues Relating to the Health
Economic Evaluation Literature
•
•
•
•
Methodological standards
Quality of reporting
Value-added information sources
Formulation of search strategies
Purpose of Economic Evaluation
Studies
• To ensure that the benefits from health care
programmes implemented are greater than
the the opportunity cost of such programmes
by addressing questions of
• allocative efficiency or
• technical efficiency
Economic Evaluation Studies
A FULL economic evaluation compares BOTH the
• costs AND consequences of
• two or more interventions
Requires
– identification
– measurement
– valuation
of both costs and benefits
Economic Evaluation Methodologies
•
•
•
•
Cost-benefit analysis (CBA)
Cost-utility analysis (CUA)
Cost-effectiveness analysis (CEA)
Cost-minimisation analysis (CMA)
Economic Evaluation Methodologies
Evaluative
technique
Evaluative
technique
Benefits
Benefits
Unit of
measurement
Unit of
measurement
Cost-effectiveness
Quantity of life Life years gained
analysis
OR
Cost-effectiveness
Quantity
of life Life years gained
Natural units
analysis
Quality ofOR
life
Natural units
Quality of life
Health years
Quantity +
eg. QALYs,
of life
Cost-utility analysis Quality
HealthHYEs
years
Quantity
+ Quality
eg. QALYs, HYEs
of life
Cost-benefit
Money
Quantity +
eg. human capital,
analysis
of
life
Cost-benefit analysis Quality
Quantity + Quality Money
Cost-utility analysis
to pay
(may include some non- willingness
eg.
human
capital,
of life
health aspects)
willingness to pay
(may include some nonhealth aspects)
Partial Economic Evaluation Studies
•
•
•
•
cost
cost
cost
cost
comparison/cost analysis study
of illness study
outcome description
description
Mislabelling of Economic Studies
Zarnke, KB; Levine, MAH; O’Brien, BJ. Cost-benefit
analyses in the health-care literature: don’t judge
a study by its label.
Journal of Clinical Epidemiology 1997;50:813-822
Of a set of economic studies labelled CBA:
• 68% did not use defined CBA methodology
• 53% were cost comparisons only
Quality of Health Economic
Evaluation Studies
• Poor study design
– poor data sources
– failure to define clearly the economic
evaluation method used
– use of an inappropriate method if
economic evaluation to address the
research question
– omission of sensitivity analysis to test
robustness of modelling
Quality of Health Economic
Evaluation Studies
• Inadequate reporting
– lack of transparency by authors
– journal editorial policy surrounding
economic submissions
• Publication bias
– publication bias in the clinical literature
– motivation for conducting an economic
evaluation
Guidelines for Economic Evaluation
•
Conduct
–
–
•
Reporting
–
•
Pharmaceutical Reimbursement e.g. Australian
“fourth hurdle”
Methodology e.g. NICE, Washington Panel*
Drummond MF; Jefferson TO. Guidelines for authors
and peer reviewers of economic submissions to the
BMJ. BMJ 1996;313:275-283
Appraisal
–
–
NHS Economic Evaluations Database (NHS EED)
guidelines for structured abstracts†
BMJ Guidelines
*Weinstein MC. et al. Recommendations of the Panel on Cost-effectiveness in Health and Medicine. JAMA
1996;276:1253-58
†Improving access to cost-effectiveness information for health care decision-making: the NHS Economic
Evaluation Database. NHSCRD Report no.6 (2nd ed). York. NHS Centre for Reviews
Searching Bibliographic Databases
for Economic Evaluations
• Indexing policy
– Availability of controlled vocabulary
– Specificity of indexing terms
• Indexing accuracy
– Clarity of description
– Accurate labelling
– consistency
Indexing - MeSH
Economics
Costs and Cost Analysis
Cost Allocation
Cost-Benefit analysis
Cost Control (+1)
Cost of Illness
Cost Sharing (+2)
Health Care Costs
Direct Service Costs
Drug Costs
Employer Health Costs
Hospital Costs
Health expenditures (+1)
Indexing - EMTREE
Health Economics
Economic Evaluation
Cost Benefit Analysis
Cost Control
Cost Effectiveness Analysis
Cost Minimization Analysis
Cost of Illness
Cost Utility Analysis
Fee (+5)
Health Care Cost
Drug Cost
Health Care Financing
Hospital Cost (+3)
Health Insurance (+8)
Pharmacoeconomics(+5)
Value-added Bibliographic Sources of
Health Economic Evaluations
Databases
• NHS Economic Evaluation Database
(NHS EED)
• Health Economic Evaluations Database (HEED)
• EURONHEED
• CEA Registry (formerly Harvard CUA Database)
• Journals
• Evidence Based Health Care
• ACP Journal Club
NHS Economic Evaluation Database
NHS EED
NHS Centre for Reviews & Dissemination
University of York, UK
• Economic evaluations
• critical appraisal detailed structured abstract
• Other types of health economic analysis with
bibliographic details
• MeSH searching
• 1995-
Health Economic Evaluations
Database HEED
Office of Health Economics, UK & International
Federation of Pharmaceutical Manufacturers
Associations (OHE-IFPMA). www.ohe-heed.com/
• Structured format
–
–
–
–
Bibliographic details
Study definitions
Analysis and results
Keywords
• Expert review
• 1992 -
The CEA Registry: Database of CostUtility Analysis Studies
•
•
•
•
•
Harvard Center for Risk Analysis
www.hsph.harvard.edu/cearegistry/
Reference list of studies that have used costs
per QALY to measure health benefits 1976 2001
Comprehensive league table of Cost-QALY Ratios
1976-2001
League table of cost-QALY ratios which adhere
to the Washington Panel 1976-1997
Catalogue of Preference Scores used for QALYs
Audit checklist
European Network of Health Economics
Evaluation Databases (EURONHEED)
Project
• Launched 2003 - 3 year development
– Country-specific databases set to be available in
2005
• 7 regional databases covering 17 European
countries
– 2 established viz. NHS EED (UK) and CODECS
(France)
– 5 to be created viz. Germany, The Netherlands,
Italy,Spain and Sweden
Health Economic Evaluation Studies
Summary Points
• Strict definition criteria
• Quality variation
• Methodology
• Reporting
• Differences in bibliographic database indexing
• Specificity and
• Scope
• Use of value-added sources
National Information Center on Health
Services Research & Health Care
Technology (NLM/NIH)
• Etext of Health Technology Assessment (HTA)
Information Resources (Chapter 11)
www.nlm.nih.gov/nichsr/ehta/ehta.html
• Health Economics Information Resources: a
self-study course
www.nlm.nih.gov/nichsr/edu/healthecon/