CER vs CEA – A decision analytic
perspective
How much difference should one letter make?
R. Scott Braithwaite, MD, MSc, FACP
Before Sept ‘09:
Associate Professor of Medicine
Yale University School of Medicine
After Sept ‘09:
Chief; Section of Comparative Effectiveness, Quality, and Value
NYU School of Medicine
CER vs CEA
 Some view cost-effectiveness research (CEA)
as one type of comparative effectiveness
research (CER)
 Allied tools for getting better performance from
our health care system
 Others view CEA as distinct from CER
 “Comparative effectiveness” does not have “cost”
in it
 Lighting rod for fears about rationing
Argument
 The whole motivation for CER is to better inform
decisions
 Therefore CER design and questions are
informed by decision analytic perspective
 Decision Analysis: Quantitative, systematic, objective
way to put harms and benefits on level playing field
 Questions:
 How much information will the study add?
 Is this information useful for decision making?
Decision-analytic perspective: CER
 Many implications, of which cost is only one:
 Prioritizing what to study
 Design decisions





Comparators
Health outcomes
Statistical criteria for hypothesis testing
Strength of evidence thresholds for applying results
Cost as possible outcome
 Measure cost?
 Perform CEA?
Roadmap of talk
 Overview of CER
 How decision-analytic perspective would impact
CER methods in general
 How decision-analytic perspective would impact
CER methods regarding cost
 Question: Should CER and CEA be completely
separate, completely overlapping, or
somewhere in the middle?
CER-Definition
(Institute of Medicine)
 The generation and synthesis of evidence that
compares the effectiveness of alternative methods to
prevent, diagnose, treat, monitor, and improve
delivery of care for a clinical condition.
 The purpose of CER is to assist patients, clinicians,
purchasers, and policy makers in making informed
health decisions.
CER – Some Examples
 For a patient who has severe coronary artery disease
and must choose between surgery and a less invasive
procedure, what is best?
 Is the answer the same if patient recently had heart attack?
 Should men be screened for prostate cancer with the
PSA blood test?
 Is the answer the same if sick from other conditions?
 How to choose the best high-blood-pressure drug?
 Is the answer the same for patients who are at high risk for
heart attacks and strokes?
CER-Study Designs
 All compatible with cost-estimation or lack thereof
 RCT
 Meta-analysis and systematic review
 Observational analyses
 Registries
 Administrative data and EMR
 Prospectively enrolled cohorts
 Indirect comparisons
 Bayesian approaches
 Mathematical modeling and computer simulations
Roadmap of talk
 Overview of CER
 How decision-analytic perspective would impact
CER methods in general
 How decision-analytic perspective would impact
CER methods regarding cost
 Question: Should CER and CEA be completely
separate, completely overlapping, or
somewhere in the middle?
What to study first?
 Preferentially study questions for which additional data or
analysis is likely to influence the decision
 Confers highest value of information (Claxton et al, 1986)
 Monetarized value of improvements in health outcomes resulting
from the better decision-making that is made possible by the study
 Focus on healthcare services
 In common practice
 Large aggregate expenditures
 Employ rapidly changing technologies for which multiple
alternative therapies exist for the same problem
 Clinical, preventive or treatment areas with substantial
uncertainty about what is the best decision
 Large penalty for making the wrong choice
Which comparators to use?
 One comparator should be the current standard of
care or usual care
 Comparators should include next best alternative
 e.g. COX-2 inhibitors (marketed for reduced GI side
effects) should have been more often compared against
dual-therapy with older NSAID + PPI (GI protective drug)
 Comparators might include systems of care as
well as specific interventions
 e.g. P4P, patient-centered medical home, VBID
Which health outcomes to use?
 Should be able to balance harms against benefits
 Note: harms may be unanticipated and therefore not reflected in disease-
specific outcome measures specified a priori
 e.g. If you are studying an analgesic and primary outcome measure is pain you
may not measure nausea that might offset the benefit of pain improvement
 Should include quality-of-life measure that is
 Generic (not disease-specific)
 Preference-weighted (more reflective of decision making)
 Gives results on a unidimensional, interval scale
 Allows explicit weighting of harms and benefits
 Limited group of instruments satisfy these criteria
 E.g., EQ-5D, HUI, QWB, SF-6D
 Disease-specific instruments are sometimes calibrated against these
What statistical criterion to use for
hypothesis testing?
 It is customary to mandate greater than a 19-in-20 chance that
an improvement is “real” rather than a statistical fluke
 p-value < 0.05
 This might be too strict of a standard when treatments are
compared against other treatments rather than placebo
 May not maximize benefits and minimize harms, particularly if
treatment is known to have a favorable toxicity profile from prior study.
 More research is needed to elucidate if and when less strict
statistical criteria are appropriate
 Should p < 0.10 ever be the criterion? How about others?
 Vary with budget constraints for gathering additional evidence?
What threshold of evidence to
require for decision making?
 It is difficult to specify a priori what threshold of evidence is
sufficient to influence a decision
 Will be based on how certitude of evidence compares with the
certitude of the decision maker’s beliefs
 If decision maker has certitude of belief >> certitude of
evidence, then his post-test distr ≈ pre-test distrib

Evidence will have little influence on his decision

E.g. World-expert reading a single institution case control study
 If the decision maker has certitude of belief << certitude of
evidence, then his post-test distr ≠ pre-test distr

Evidence should influence the decision greatly.
 E.g. world-expert reading 12 RCTs reaching consistent conclusions
What threshold of evidence to
require for decision making?
 CER should express strength of evidence using a
simple, unidimensional, and transparent scale
 e.g., United States Preventive Services Task Force
 Strength of evidence grades: A, B, C, I
 Separate grades for “this study alone” and “this study added
to existing literature”
 Would facilitate use of CER decision making
 Uncertain decisions with high evidence thresholds will
often have high value of information, and are
especially ripe for future study.
Roadmap of talk
 Overview of CER
 How decision-analytic perspective would impact
CER methods in general
 How decision-analytic perspective would impact
CER methods regarding cost
 Question: Should CER and CEA be completely
separate, completely overlapping, or
somewhere in the middle?
Should CER measure costs?
 CER is strengthened if it includes measures of costs
 Essential for estimating value and informing resource allocation
 Cost measures should
 Include assessments from societal perspective
 Not only expenses that accrue to any one party (e.g. the pharmacy,
the health plan), but also costs that accrue throughout the system,
including to the patient/family
 Encompass longest feasible time horizon
 May discount these costs as appropriate (e.g. 3%)
 Reflect the opportunity cost of resource allocation rather
than charges
CER more useful if costs measured
 P4P design
 Value-based insurance design (VBID)
 Clinical trial design
 Informing reimbursement schedules
 Estimating return on investment (ROI)
CER more useful if costs measured
 Influencing P4P design
 How large should be the monetary incentives should we
give to the highest-quality providers and health systems?
 Can we pay clinicians and health plans for improving
health (e.g., coordinating care), not just for doing
procedures and tests?
 Suppose Plan A has 80% compliance and Plan B has 60% compliance with
an evidence-based colorectal cancer screening program. Because
colorectal cancer screening confers 0.25 additional high-quality life-years,
on average plan A delivers 0.05 more high-quality life-years than Plan B.
Therefore, the value of this quality improvement is worth ≥$5,000,
assuming a value of health benefits of $100,000 per high-quality life-year.
CER more useful if costs measured
 Value-based insurance design (VBID)
 When should patients have copayments waived because
its value is so favorable?
 When should patients have copayments increased
because its value is so unfavorable?
 Should patients receive a “health dividend” for healthy
behaviors that save money for the healthcare system,
such as preventing kidney failure in diabetics?
 Designing clinical trials
 Ensuring we can detect improvements sufficiently large to
offer favorable value
CER more useful if costs measured
 Influencing reimbursements
 How much is a particular treatment “worth,” based on
the amount of benefit that it gives?
 How can we make sure that patients and health plans
do not pay more than this worth?
 If a health service is delivers 0.1 high quality lifeyears as a benefit, the maximum reimbursement for
that service would be $10,000 assuming a value of
health benefits of $100,000 per high-quality life-year
CER more useful if costs measured
 Estimating return-on-investment (ROI)
 Inappropriate to include only financial consequences
in an ROI calculation .
 Health benefits should be monetarized (e.g., an
intervention that delivers 1 high-quality year of
additional life delivers at least $100,000 worth of
health benefits) and included in ROI calculations.
 Imperative to consider health benefits when
estimating ROI else calculations just reduce to
budgetary-impact calculations, which ignore the
benefit of investing in health.
Roadmap of talk
 Overview of CER
 How decision-analytic perspective would impact
CER methods in general
 How decision-analytic perspective would impact
CER methods regarding cost
 Question: Should CER and CEA be completely
separate, completely overlapping, or
somewhere in the middle?
Should CER involve CEA?
 Not necessarily
 But it should provide all important inputs for
future CEA
 Outcome measures (quality-of-life, etc)
 Costs (perspective, etc)
 CEA using CER data could be funded using
alternative mechanisms
What to use as a threshold for
acceptable value?
 Value thresholds are subject to intense debate, and
vary from time to time and from society to society.
 That being said, it may be possible to make
inferences regarding plausible bounds
 For example, the appropriate value threshold in the US is
likely to be between $100,000 and $300,000 (Braithwaite
et al, 2008)
Conclusions: CER vs CEA
 CER will have limited use if costs aren’t
measured
 “Menu without prices” (Garber et al, 2007)
 It is unclear whether CER should encompass CEA
 But should measure all important CEA inputs so it can
facilitate CEAs
 CER should measure all outcomes necessary for
good decisions
 Health outcomes
 Costs