Introduction to decision analysis modeling
Alice Zwerling, Postdoctoral fellow, JHSPH
azwerli1@jhu.edu
McGill TB Research Methods Course
July 7, 2015
Overview of Decision Analysis models
• Understand why we use decision analysis models & what
these models can and cannot do
• Examples of decision analysis and simple decision trees
• Understand key decision analysis concepts and terminology
2
Decision Analysis Models
• Some decisions are
easier than others
3
Decision Analysis
• Provides a systematic approach to decision-making under
conditions of uncertainty
• Is employed across many diverse fields: healthcare, business,
economics, law, engineering…..
• Helps decision makers think clearly about many elements of
complex decisions
4
Decision Analysis Modeling
• Requires defining events in terms of their logical and temporal
significance
• Disaggregating a complex problem into smaller steps or
events simpler to understand
5
What is a model?
• Representation of real world phenomena, object or
behaviours
• Theoretical construct
• Simplification of real world
• Model needs to be simple enough to be understood but
complex enough to capture essential elements of problem or
question to be addressed
• Many simplifying assumptions are needed…
6
Decision Trees
• Branching structure used to represent
different kinds of events including decisions
and uncertainties, and the
associated outcomes
or alternatives
7
Decision Trees
Chance Node
A
Decision Node
Branches
(alternative pathways)
B
Time
Time (sequence of events) moves from left to right
8
Terminal Node
(Outcomes or
pay-offs)
Decision Trees
Pathway probabilities
(probability of going down
each branch)
p =0.1
p =0.9
•
9
User defined probabilities are entered at each chance node, need to add to 1
Decision Trees
Outcomes: Costs
& Effectiveness
(DALYs/QALYs or
cases 0/1)
10
Decide what sequences/events are important to include/ model
11
12
Vassall et al PlosMed, 2011
Steps in Decision Analysis
• Define the problem
• Define alternatives to be assessed
• Identify clinical outcomes of interest
•
•
•
•
Build the tree structure
Assign probabilities to all chance events
Assign costs and outcomes
Estimate expected average values of each strategy (Folding
back)
• Assess uncertainty and variability (sensitivity analyses)
13
A simple problem…
Evaluate a new treatment
Cure
Standard Tx
Failure
Patients with TB
diagnosis
Cure
New Tx
Failure
• Create structure
14
A simple problem…
Evaluate a new treatment
Cure p=0.6
Standard Tx
Failure p=0.4
Patients with TB
diagnosis
Cure p=0.8
New Tx
Failure p= 0.2
• Assign probabilities
15
A simple problem…
Evaluate a new treatment
Cure p=0.6
$100
Saves 10 life years
Standard Tx
Failure p=0.4
$100
Saves 1 life year
Cure p=0.8
$500
Saves 10 life years
Patients with TB
diagnosis
New Tx
Failure p= 0.2
• Assign values and outcomes
16
$500
Saves 1 life year
Calculations
Cure p=0.6
$100
Saves 10 life years
Standard Tx
Failure p=0.4
$200
Saves 1 life year
Cure p=0.8
$500
Saves 10 life years
Patients with TB
diagnosis
New Tx
Failure p= 0.2
$600
Saves 1 life year
Outcome
Standard Treatment
New Treatment
Expected cost
(0.6 x $100) + (0.4 x $200)
=$60 + $80 = $140
(0.8 x $500) + (0.2 x $600)
=$400 + $120 = $520
Expected life years saved
(0.6 x 10) + (0.4 x 1) = 6.4
(0.8 x 10) + (0.2 x 1 ) = 8.1
17
Calculations
Expected Outcomes
Standard Treatment
New Treatment
Expected cost
(0.6 x $100) + (0.4 x $200)
=$60 + $80 = $140
(0.8 x $500) + (0.2 x $600)
=$400 + $120 = $520
Expected life years saved
(0.6 x 10) + (0.4 x 1) = 6.4
(0.8 x 10) + (0.2 x 1 ) = 8.1
• Incremental cost: $520 - $140 = $380
• Incremental life years saved: 8.1 – 6.4 = 1.7
• Cost per life year saved = $380/1.7
= $223.53
18
Decision analysis
• Final model outcomes are calculated based on the
probability of entering into a particular node and the price
tag or effectiveness measure associated with that node
• Individuals move through the decision trees for a
specified amount of time
• Costs and rewards accrue over the simulation
• At end of simulation get a tally of specified outcomes (eg.
TB related costs per person, number of TB cases,
number of TB deaths, etc for each intervention
considered (outcomes)
Comparing scenarios!!!
20
21
22
Brief note on sensitivity analyses
• Systematically asking how would decision results change if
parameters/probabilities were different
• Determines how robust model result may be
• Vary probabilities and/or outcomes across possible ranges
• Can produce tornado diagrams
23
Decision Analysis Models
• Not designed to capture transmission
• Static, not dynamic models
• Typically deterministic (can be stochastic)
24
Decision Analysis Models
• STATIC models:
(as opposed to dynamic transmission models)
• The annual risk of infection (ARI) is not sensitive to the
changing number of infectious cases in the population
• Does not account for ongoing transmission in a population
25
Population vs. Individual models
Population based
Individual based
• Follows populations
• Divides population into mutually
exclusive groups
• Homogeneity within groups, but
can be subdivided
• Individual level factors are
averaged together, model shows
changes in average
characteristics of whole
population
• Follows individuals
• Characteristics of each individual
are tracked through time
• Can explore complex
relationships, social/spatial
interactions, heterogeneity
• May include approaches such as
agent-based models, or queue
model
Decision analysis are most commonly population based, but can track individuals
26
Population based
Individual based
• 25% of population has TB HIV
co-infection
• Each individual tracked
• 45% of cohort is female
• Pt 1 of 10,000 simulated pts
Is a female 45 years old, HIV+
• Cohort comprised of persons on
average 45 years old
27
Deterministic models
Stochastic models
• All parameters are fixed, no
random element
• Incorporate chance into the
model
• Model predictions are fixed,
same answer with every run
• Results vary with every run
• Describes what happens on
average to whole population
• More frequent in literature
• Important in small populations or
where chance might play a role
• Require many simulations (more
computing power)
Decision analysis can use either approach, but are most commonly deterministic models
28
Decision Analysis: Advantages
•
•
•
•
•
Easy to learn & user friendly
Intuitive, visual representation of problem
Can take advantage of average data (e.g. meta-analyses)
Low cost and faster (relative to empirical studies)
Can consider hypothetical situations or specific
populations/scenarios for whom a trial is not ethical or feasible
• Can capture more complex pathways
• Can be used to generalize/extrapolate trial findings
(time/pop’n)
• Integrated costing capability and can be easily modified for
cost-effectiveness (common form of modeling methods in
economic evaluation)
Decision analysis: Disadvantages
• Decision analysis describes what happens to a cohort of
selected individuals
– By design does not incorporate pop’n outside of cohort
• Population level impact not accounted for
• Transmission effects not typically incorporated in decision
analysis models
• The annual risk of infection is not sensitive to changing number of infectious cases in the
population
• Can be partially addressed using Markov models and
introducing assumptions around transmission parameters
30
Decision Analysis Models CAN….
• Compare relative impact and cost of two different well defined
interventions
• Understand problems in a logical and transparent fashion
• Identify weakness in our conceptualization of problem
• Make assumptions explicit
31
Summary: Models are not so good for…
•
•
•
•
Predicting the future
Giving precise estimates
Working magic with bad/limited data
Can only work to level of complexity that we understand/ have
data to support
• Models rely on assumptions, may be limited by poor data (can
test these…)
• Struggle to capture heterogeneity/phenomenon that we are
not aware of or do not understand
Summary…
• Different types of models are available
Choice depends on :
• The research question
• The data that we have to work with
• The assumptions that we are willing to make
• How quickly we need the results
• The expertise of the modelling “team”
Reproduced from xkcd.com
34