"Stated Preference Theory (Conjoint
Analysis) is the Best Way to Assess
Health-Related Quality of Life for
Economic Assessment of Drugs and
Medical Interventions: An Application
in Alzheimer Disease"
Joel Hay, PhD
Department of Pharmaceutical Economics and Policy
University of Southern California
April 21 2006
Presented at ISPOR Student Teleconference
Research supported by the State of California
Alzheimer Disease Research Centers Network
JHay@usc.edu
1
Overview
Measuring Health Related QOL
for Cost Effectiveness Analysis
Problems with CE ratios in
making decisions
Justifications for Net Monetary
Benefits based on willingness to
pay
JHay@usc.edu
2
Overview
Introduction to Conjoint Analysis
Alzheimer Conjoint Experiment
Design
Results for the Alzheimer CA
Experiment
Conclusions & Future Directions
JHay@usc.edu
3
Stated Preference Elicitation
Methods
Important to the design of successful
drugs and other interventions
Can reduce R&D costs enormously
by focusing on treatment and
disease symptoms that are most
important
Can be used to weigh utility of
outcomes in cost-effectiveness
analysis
JHay@usc.edu
4
Stated Preference Methods
Willingness to Pay/ Stated Value
Standard Gamble / Time Trade-off
Global Assessment / Visual Analog
Health Status Assessment / Multiattribute Scales
Conjoint Analysis/ Discrete Choice
Experiments
JHay@usc.edu
5
Stated Preference Methods
Stated preference is not revealed
preference
All stated preference methods ask the
subject to make hypothetical choices
All stated preference methods require the
subject (patient) to consider or imagine the
value of health states they don’t actually
experience
JHay@usc.edu
6
Denominator of CEA Ratio
[Cost(A) - Cost(B)] /
[QALY(A) - QALY(B)]
How are QALYs determined?
Patient or subject survey of health states
EQ-5D, HUI, QWB, SF-##
Results converted into QOL-weighted time
Uses someone’s utility weighting algorithm
Utility translation imprecision is generally
ignored
JHay@usc.edu
7
Problems with QALYs
Doesn’t accurately reflect individual
utilities
Implies that people have constant
preference for healthy years
Ignores risky behaviors
Ignores life expectancy constraints
Ignores alternative consumption
opportunties
Creates serious analytic problems in a
CE ratio
8
JHay@usc.edu
STOCHASTIC UNCERTAINTY /
VARIABILITY
2000
2000
1500
1500
1000
Difference in costs
Difference in costs
• Feiller’s Theorem Probability Ellipses
500
0
-500
-1000
-1500
-2000
-0.25
1000
500
0
-500
-1000
-1500
-0.15
-0.05
0.05
0.15
0.25
-2000
-0.25
Difference in effectiveness
-0.15
-0.05
0.05
0.15
0.25
Difference in effectiveness
JHay@usc.edu
9
Analyzing CEA Ratio Results
Computing 95% CI around mean ratio is plagued with many problems
ΔC
ΔE
JHay@usc.edu
10
Analyzing CEA Ratio Results
Computing 95% CI around mean ratio is plagued with many problems
ΔC
ΔE
• CI can include undefined
values
• ΔC / ΔE
• What if ΔE = 0?
• CI = (LL, ∞) U (∞, UL)
JHay@usc.edu
11
Analyzing CEA Ratio Results
Computing 95% CI around mean ratio is plagued with many problems
• CI can be undefined
JHay@usc.edu
12
Analyzing CEA Ratio Results
Computing 95% CI around mean ratio is plagued with many problems
ΔC
• The same value for a CER can
have 2 completely different
meanings
b1
a2
ΔE
a1
• a1 = a2 but many prefer a2
• b1 = b2 but in reality, b1 << b2
b2
JHay@usc.edu
13
Analyzing CEA Ratio Results
Computing 95% CI around mean ratio is plagued with many problems
ΔC
a
• Not properly ordered outside
trade-off quadrants
• b better than c
• c better than a
• a and b equal when in
ΔE
reality b >> a
c
b
JHay@usc.edu
14
Analyzing CEA Ratio Results
Acceptability Curves
Incremental costs positive
Quadrant 4
Inferior
Quadrant 1
Trade off
R
Incremental
effects
negative
Incremental
effects
positive
Quadrant 3
Trade off
Quadrant 2
Superior
Incremental costs negative
JHay@usc.edu
15
Acceptability Curves
Cost Effectiveness Acceptability Curves
Shows the probability that the new therapy will
be cost effective as a function of the societal
willingness-to pay (for a QALY) threshold
Gets around the problems of CEA confidence
intervals
Main advantage: Adopts the natural perspective /
interpretation of decision makers: “how likely is it
that the intervention will be cost effective”
JHay@usc.edu
16
Acceptability Curves
P (Intervention is Cost effective)
1
Curve tend to
1-“p” value (one-sided)
for effectiveness
difference
One-sided
“p” value
for cost (savings)
difference
Proportion
of cases
In which
intervention
is more
effective
(Quadrants
2+1)
Proportion
of cases in which
the intervention
is cost-savings
(Quadrants 2+3)
0
$0
$25,000
$50,000
$75,000
Cost-effectiveness Threshold R
JHay@usc.edu
$100,000
$
17
Acceptability Curves
Analysts have had the natural tendency to
interpret the results as the probability that the
cost-effectiveness ratio is lower or equal than a
specific threshold (given the data)
AC are not always monotonically increasing!
Their shape depends on the data – they can go
up and down!
95% intervals cannot always be defined
JHay@usc.edu
18
Using Net Monetary Benefits to Make
Decisions
The incr net benefit (INB) was introduced to
estimate whether a treatment is cost effective
INB = IC – λ *IE
λ is decisionmaker willingness to pay for QALY
A therapy, for which the INB is lower than zero, may be
considered cost-effective
However, even when acceptability curve indicates
that new treatment is cost-effective at a λ the
decision maker is willing to pay, there is still a
probability that the decision to reimburse the new
treatment is wrong
JHay@usc.edu
19
Using Net Monetary Benefits to Make
Decisions
Incr Net Ben = Incr Cost – λ * Incr Effect
λ reflects the decision maker’s willingness to pay for
new therapy
Why are we using subjects to generate HRQOL and
QALYs but not using subjects to generate λ
(willingness to pay for QALYs)?
The CEA researcher is throwing away the
opportunity to capture subject willingness to pay
FOR NO REASON!!
JHay@usc.edu
20
EXPECTED VALUE OF PERFECT
INFORMATION
Is quantifying uncertainty useful to decision
makers?
Focus should be on determining the value of collecting
new information (= cost of uncertainty)
New information valued in terms of expected reductions
in decision errors (NMB loss in money)
See Claxton K. The irrelevance of inference: a decision
making approach to the stochastic evaluation of health care
technologies Journal of Health Economics, 1999;18:341-64
JHay@usc.edu
21
Why not Capture Willingness to Pay
Directly?
Gives a more accurate and precise
measure of subject trade-off between
money and health states
Doesn’t force health utilities into an
artificial QOL framework
Allows direct assessment of Net
Benefits rather than artificial CEA
ratios
JHay@usc.edu
22
Why not Capture Willingness to Pay
Directly?
Even if data come from current
HRQOL measures it’s possible to
develop WTP conversion scales
Need to subject existing HRQOL
instruments to WTP assessment
Conjoint Analysis is the way to do
this
JHay@usc.edu
23
Advantages of Conjoint Analysis
Rigorous behavioral choice model:
Random Utility Theory
Actually estimates utility
parameters
Experimental design efficiently maps
choice preferences in multi-attribute
space
Subjects can’t “game” responses
JHay@usc.edu
24
Random Utility Maximization
Uiq = Viq + iq
Uiq > Ujq for all j  i element of A
Pi = Pr[(Viq - Vjq) > (jq -  iq )]
JHay@usc.edu
27
Stated Preference Elicitation in
Alzheimer Disease
AD treatments involve trade-offs
across different functional
domains
Important, since AD patients have
difficulty in making & expressing
choices
Relies on proxy responders,
usually AD caregivers
JHay@usc.edu
31
Stated Preference Elicitation in
Alzheimer Disease
Develop a CA experiment to elicit
preferences across function
domains for hypothetical AD
patients
Include treatment costs to
estimate willingness-to-pay for
improvements/decreases in daily
function
JHay@usc.edu
32
Stated Preference Elicitation in
Alzheimer Disease
Validate the CA design in a
sample of pharmacy students
Apply the CA experiment to a
sample of AD caregivers
Compare results
JHay@usc.edu
33
CA Experiment
Attributes & Levels from Health
Utilities Instrument-- Mark 3
This instrument is widely used
and validated
It was developed to map into a
standard gamble utility scale
valid for cost effectiveness
analysis
Utility scores from the HUI can
be compared to RUT scores
JHay@usc.edu
34
JHay@usc.edu
35
Survey
PILOT STUDY
The CA design was validated in a
sample of pharmacy students:
-HUI
functional domain attributes were
shown to be significant and
consistent predictors of choice
- Total HUI utility score is much better
predictor of choice than individual
components
JHay@usc.edu
36
Survey
•DEMOGRAPHICS
•Survey was administered to 74 AD
caregivers enrolled in the California AD
Research Centers.
•STATED PREFERENCE SCENARIOS
•HUI-3 for the AD PATIENT, as PROXIED
by the CAREGIVER
•HUI-3 for the CAREGIVER
•SF-36 for the CAREGIVER
JHay@usc.edu
37
Survey
STATED PREFERENCE SCENARIOS
Fractional Factorial Experimental Design
Attributes and Levels taken from the Health
Utilities Instrument - Mark 3
best – 3rd – worst
243 combinations of comparisons that
were not dominant or dominated choices
25 x 10 choice sets each
61 x 4 choice sets each
Cost of each choice
= $100 / $50 / $0
JHay@usc.edu
38
JHay@usc.edu
39
JHay@usc.edu
40
Speech
Ambulation
Dexterity
JHay@usc.edu
41
JHay@usc.edu
42
JHay@usc.edu
43
JHay@usc.edu
44
Scoring the HUI
• Patient HUI as proxied by the CG
• Caregiver HUI
• HUI of each choice set
Multi-attribute utility function:
u = 0.371(b1*b2*b3*b4*b5*b6*b7*b8)-0.371
JHay@usc.edu
45
Analysis Plan
• Demographics
• MNL for HUI, cost in predicting choice
• Strength of each attribute in determining
subject choice
• Subject WTP for various choice attributes –
determining strength of cost attribute
• Estimate subject utility
- Health utility of AD patients
- Mapping of the HUI-3 to the SF-36 for AD
JHay@usc.edu
46
Results:
Descriptives
Mean
Patient HUI
Caregiver HUI
SF-36 Physical Function
SF-36 Physical Role Function
SF-36 Bodily Pain
SF-36 General Health
SF-36 Vitality
SF-36 Social Functioning
SF-36 Emotional Role Function
SF-36 Mental Health
Caregiver Age
Caregiver Years of Education
Caregiver duration of caregiving
since disease onset (months)
Caregiver hours per week
Caregiver is Spouse of Pt
Caregiver is Daughter of Pt
Caregiver is Son of Pt
Working Full Time
Retired
Primary Caregiver
Married
White
Hispanic
Asian
Black
Gender (1=male)
JHay@usc.edu
0.35
0.88
82.00
76.47
74.09
76.32
63.34
80.69
73.49
76.68
60.79
15.31
82.70
Std.
Deviation
0.38
0.13
20.68
33.84
22.59
16.54
18.76
20.87
34.78
15.37
14.45
3.68
136.53
23.76
0.40
0.28
0.17
0.42
0.30
0.76
0.73
0.45
0.31
0.11
0.09
0.35
32.21
0.49
0.45
0.38
0.49
0.46
0.43
0.44
0.50
0.46
0.32
0.29
0.48
47
Results: Correlations
Patient HUI
Patient HUI
Pearson
Correlation
Caregiver
HUI
1
Sig. (2-tailed)
Caregiver HUI
Pearson
Correlation
Sig. (2-tailed)
Caregiver SF-36
Physical Function
Pearson
Correlation
Sig. (2-tailed)
Caregiver SF-36 General
Health
Pearson
Correlation
Sig. (2-tailed)
Caregiver SF-36 Mental
Health
Pearson
Correlation
Sig. (2-tailed)
0.303
Caregiver
SF-36
Physical
Function
Caregiver SF36 General
Health
0.303
0.097
-0.003
0.115
0.00
0.07
0.96
0.03
1
0.528
0.356
0.335
0.00
0.00
0.00
1
0.401
0.216
0.00
0.00
1
0.511
0.00
0.097
0.528
0.07
0.00
-0.003
0.356
0.401
0.96
0.00
0.00
0.115
0.335
0.216
0.511
0.03
0.00
0.00
0.00
JHay@usc.edu
Caregiver
SF-36
Mental
Health
0.00
1
48
Dependent Variable: Patient HUI
Results:
Regression
(Constant)
Caregiver SF-36 Physical Function
Caregiver SF-36 Physical Role
Function
Caregiver SF-36 Bodily Pain
Caregiver SF-36 General Health
Caregiver SF-36 Vitality
Caregiver SF-36 Social Functioning
Caregiver SF-36 Emotional Role
Function
Caregiver SF-36 Mental Health
Age
Years of Education
CG duration of caregiving since
disease onset (months)
CG hours per week
Caregiver is Spouse of Pt
Caregiver is Daughter of Pt
Caregiver is Son of Pt
Working Full Time
Retired
Primary_CG
Married
White
Hispanic
Asian
Black
Gender
Caregiver
HUI
JHay@usc.edu
Unstandardized
Coefficients
B
Std.
Error
0.226
0.197
-0.005
0.001
-0.004
0.001
Sig.
0.253
0.000
0.000
0.005
-0.005
0.003
0.003
0.001
0.001
0.001
0.002
0.001
0.001
0.000
0.000
0.070
0.015
0.056
-0.005
-0.002
-0.004
0.000
0.002
0.002
0.005
0.000
0.002
0.344
0.504
0.528
-0.003
0.176
0.033
0.197
0.012
0.129
0.095
-0.234
-0.119
-0.395
-0.004
-0.395
-0.078
1.261
0.001
0.083
0.064
0.086
0.050
0.059
0.054
0.057
0.090
0.090
0.101
0.111
0.058
0.180
0.000
0.034
0.605
0.022
0.812
0.029
0.078
0.000
0.185
0.000
0.966
0.000
0.181
0.000
49
Results
•In a multinomial logistic regression, treatment
choice was positively related to HUI score for
the chosen intervention and negatively related
to treatment costs (P < 0.01).
•At the margin, caregivers would be willing to
spend an additional $5-$7 per month for any
AD intervention that increased patient HUI
utility scores by 1%.
•The strongest rankings include improvements
in ambulation, emotion and cognition.
JHay@usc.edu
50
Results
•While HUI scores are related to SF-36 scores,
the magnitude of response is fairly small.
•Neither HUI score is related to SF-36 Vitality
and the Caregiver HUI score is not
significantly related to Physical Role Function.
•Regression based on the HUI multi-attribute
score was superior to any single-attribute
model and non-inferior to models allowing
unconstrained parameter weights for each HUI
attribute domain.
JHay@usc.edu
51
Conclusions
•Conjoint Analysis is a useful method for
benchmarking the potential values for
AD treatments trade-offs in terms of their
costs and impacts on patient
functioning.
•As found in prior studies, HUI is a useful
scale for characterizing proxy patient
utility levels.
JHay@usc.edu
52
Conclusions
•Consistent with utility theory, results show a
methodologically independent and innovative
validation of the HUI utility scale as a strong
predictor of subject health state preferences.
•Demonstrates that we can convert HUI, SF-36
and other HRQOL scales directly into
willingness to pay values for alternative health
states
•Do not need to increase imprecision and
difficulty by using Cost Effectiveness Ratios
JHay@usc.edu
53
Conclusions
•These methods will become even more
relevant as we increasingly utilize brain
scanning methods to map utility of
health states and utility of money
directly
•Functional Magnetic Resonance
Imaging
•Positron Emission Tomography
JHay@usc.edu
54
SUGGESTED READINGS
Handling Variability and Uncertainty (1)
Briggs AH, Fenn P. Confidence intervals or sufaces? Uncertainty on the costeffectiveness plane. Health Econ 1998; 7:723-740.
Briggs AH. Handling uncertainty in cost-effectiveness models.
Pharmacoeconomics 2000; 17(5):479-500.
Clark DE. Computational methods for probabilistic decision trees. Comput
Biomed Res 1997; 30(1):19-33.
Critchfield GC, Willard KE, Connelly DP. Probabilistic sensitivity analysis
methods for general decision models. Comput Biomed Res 1986; 19(3):254265.
Dittus RS, Roberts SD, Wilson JR. Quantifying uncertainty in medical
decisions. J Am Coll Cardiol 1989; 14(3 Suppl A):23A-28A.
Doubilet P, Begg CB, Weinstein MC, Braun P, McNeil BJ. Probabilistic
sensitivity analysis using Monte Carlo simulation. A practical approach. Med
Decis Making 1985; 5(2):157-177.
Glick HA, Briggs AH, Polsky D. Quantifying stochastic uncertainty and
presenting results of cost-effectiveness analyses. Expert Rev
Pharmacoeconomics Outcomes Res 2001; 1(1):25-36.
55
JHay@usc.edu
SUGGESTED READINGS
Handling Variability and Uncertainty (2)
Halpern EF, Weinstein MC, Hunink MG, Gazelle GS. Representing both first- and second-order uncertainties
by Monte Carlo simulation for groups of patients. Med Decis Making 2000; 20(3):314-322.
Shaw JW, Zachry WM. Application of probabilistic sensitivity analysis in decision analytic modeling. Fomulary
2002; 37:32-40.
Stinnett AA, Paltiel AD. Estimating CE ratios under second-order uncertainty: the mean ratio versus the ratio of
means. Med Decis Making 1997; 17(4):483-489.
Stinnett AA, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in costeffectiveness analysis. Med Decis Making 1998; 18(2 Suppl):S68-S80.
Whang W, Sisk JE, Heitjan DF, Moskowitz AJ. Probabilistic sensitivity analysis in cost-effectiveness. An
application from a study of vaccination against pneumococcal bacteremia in the elderly. Int J Technol Assess
Health Care 1999; 15(3):563-572.
Fenwick E, O'Brien BJ, Briggs A. Cost-effectiveness acceptability curves - facts, fallacies and frequently asked
questions. Health Econ. 2004 May;13(5):405-15.
Recommended Books
Hunink M, Glasziou P, Siegel J, et al (2001). Decision Making in Health and Medicine. Integrating Evidence
and Values. Cambridge, UK: Cambridge University Press.
Drumond M, McGuire A (Eds) (2001). Economic Evaluation in Health Care: Merging Theory with Practice. New
York, NY: Oxford University Press.
56
JHay@usc.edu
SUGGESTED READINGS
Expected Value of Perfect Information
Claxton, K., Sculpher, M. and Drummond, M. (2002) A rational framework for decision making by the
National Institute For Clinical Excellence (NICE) . Lancet 360, 711-716.
Claxton, K.C., Neuman, P.J., Araki, S.S. and Weinstein, M.C. (2001) The value of information: an
application to a policy model of Alzheimers disease. International Journal of Technology Assessment in
Health Care 17 (1): 38-55.
Claxton, K. (1999a) Bayesian approaches to the value of information: Implications for the regulation of new
pharmaceuticals. Health Economics 8, 269-274.
Conjoint Analysis
Hay JW: Conjoint Analysis in Pharmaceutical Research. J Managed Care Pharmacy 8:206-9, 2002.
Chiou C-F; Hay J; Wallace J, Bloom B; Neumann P, Sullivan S, Yu H-T, Keeler E, Henning J, Ofman J.
Development and Validation of A Grading System for the Quality of Cost-Effectiveness Studies. Medical
Care 2003; 41:32–44.
Dwight-Johnson M, Largomarsino I, Aisenberg E, Hay J. “Understanding Depression Treatment
Preferences among Low-Income Latinos using Conjoint Analysis”. Psychiatric Services 2004;55(8):934-37.
Johnson FR. Einstein on Willingness to Pay per QALY:Is There a Better Way? Med Decis Mak 2005; 607-8.
Recommended CA Book
Louviere JJ, Hensher DA, JD Swait: Stated choice methods : analysis and applications. Cambridge, UK,
New York, Cambridge University Press, 2000.
JHay@usc.edu
57