Introduction to Health Care Decision-Making
1
Course Introduction
• This includes a set of core slides that can be incorporated into an existing Intro to Health
Economics or Intro to Healthcare Decision-Making course.
• The aims of this primer are to:
• Introduce key concepts related to decision analysis in healthcare
• Give an understanding of the design, conduct and analysis of economic evaluation
in healthcare
2
Module 1: Decision Trees
3
Module 1: Decision Trees
What is decision analysis?
• Decision Analysis - A quantitative method for choosing from a set of alternatives under
conditions of uncertainty
• Process allows decision-makers to think clearly through elements of complex decisions:
• Range of possible actions (or inaction) and their consequences
• Impact of complex, unpredictable systems and processes (e.g., markets, health)
• Actions of others (e.g., competitors, regulators, patients)
• Incorporates what is known and also what is uncertain
4
Module 1: Decision Trees
Decision Trees
Decision Trees:
• A decision tree is a decision support tool that uses a tree-like diagram or branching
structure to represent the decision, competing strategies and their consequences.
• Decision trees utilize various nodes to represent different elements, including decisions
and uncontrollable events, resulting in a set of possible pathways.
• Analyzing the decision tree considers all the pathways and allows you to choose your
optimal strategy for each decision.
• It is one way to display an algorithm of a decision problem.
5
Module 1: Decision Trees
Decisions and Strategies
Decisions and Strategies:
• Decision trees will typically have one or more points where a choice needs to be made
from a set of competing strategies.
• For each strategy, there will be a set of consequences that will result from that choice.
• The goal of the analysis is to determine the optimal strategy.
What are the
consequences
of this choice?
6
Module 1: Decision Trees
Decisions and Strategies
• In this example, we have one decision with two strategies:
• Decision: I want to buy a lottery ticket but there are two different lotteries I can
enter.
• Strategy 1: Should I buy a ticket for Lottery 1?
OR
• Strategy 2: Should I buy a ticket for Lottery 2?
7
Module 1: Decision Trees
Pathways
• For each of these strategies, there are two possible outcomes:
• I could win
OR
• I could lose (not win)
• In this example decision tree, there are 4 potential pathways, given the different
outcomes in each scenario:
Pathway 1
Pathway 2
Pathway 3
Pathway 4
8
Module 1: Decision Trees
Pathways
• Choosing a strategy does not guarantee a result.
• Some outcomes or events will be outside our control.
• In addition, the competing outcomes must be mutually exclusive.
• For example if there is an event that may or may not occur, you need to have two
outcomes:
• One outcome where the event occurs AND
• Another outcome where the event does not occur.
• There may be a number of events/outcomes that could occur within a strategy – these
constitute decision tree pathways.
• Each model pathway is a unique path from the decision node to a final outcome
where no further events are possible.
• For a decision tree to be realistic, it must contain all possible outcomes or events, which
could result in a large number of pathways.
• Downstream outcomes could be dependent on earlier events occurring.
9
Module 1: Decision Trees
Outcomes and probabilities
• Each of these outcomes has a probability associated with how likely it is to occur.
• Probability is the likelihood of a given outcome's occurrence, which is expressed as a
number between 1 (absolute 100% certainty) and 0 (absolute impossibility – 0%
certainty).
• An outcome with a probability of 0.5 can be considered to have equal odds of occurring or
not occurring: for example, the probability of a coin toss resulting in "heads" is 0.5, because
the toss is equally as likely to result in "tails."
10
Module 1: Decision Trees
Outcomes and probabilities
• If I buy a ticket for Lottery 1, the chance of winning is 10%. Consequently the chance of
losing is 90%.
• If I buy a ticket for Lottery 2, the chance of winning is 40%. Consequently, the chance
of losing is 60%.
• There is no conditional probability in this example.
11
Module 1: Decision Trees
Payoffs (Rewards)
• Payoffs or Rewards are the values gained or lost as a result of each pathway.
• These payoffs could be measured as different kinds of outcomes:
•
•
•
•
Revenue
Costs
Life Expectancy
Death
• Typically in decision trees you enter payoffs at the endpoint for each pathway.
12
Module 1: Decision Trees
Payoffs (Rewards)
• In our example, there are two strategies, each strategy has 2 pathways, resulting in 4
total pathways or scenarios.
• Payoff values need to be entered for each of those 4 pathway endpoints.
• In this example:
•
•
•
•
If I win Lottery 1 I win $1,000
If I lose Lottery 1 I win $0
If I win Lottery 2 I win $300
If I lose Lottery 2 I win $0
13
Module 1: Decision Trees
Decision Tree
• The model is now complete.
• In Lottery 1 I have a 10% chance of winning $1,000.
• In Lottery 2 I have a 40% chance of winning $300.
• We can analyze the model to choose the optimal strategy.
14
Module 1: Decision Trees
Analyzing the Model – Expected Values
• Expected Value - a predicted value calculated as the sum of all possible values each
multiplied by the probability of its occurrence.
• Expected values are calculated for each strategy and outcome measure
• In a decision tree, we can calculate the overall expected value for each strategy by
multiplying the reward for each pathway by the probability of that pathway.
• Note there may be multiple probabilities within each pathway and multiple
outcome measured in the model, but not in this example.
• Expected values help us determine the optimal strategy.
15
Module 1: Decision Trees
Expected Values – Lottery Example
• Strategy 1: Lottery ticket 1 offers us a 10% chance of winning 1,000. Therefore:
• The expected value of playing Lottery 1 is: (0.10*1,000) + (0.90*0) = $100
• Lottery 1 gives us an expected value of $100.
• Strategy 2: Lottery ticket 2 offers us a 40% chance of winning $300. Therefore:
• The expected value of playing Lottery 2 is: (0.40*300) + (0.60*0) = $120
• Lottery 2 gives us an expected value of $120.
16
Module 1: Decision Trees
Analyzing the model – Choosing the optimal strategy
• Models will be configured to choose the method by which the optimal strategy is
identified, based on the expected values. This could entail:
• Maximizing expected value (e.g., revenue, gains in health, etc.)
• Minimizing expected value (e.g., reducing deaths, costs, etc.)
• In this example we are trying to maximize our expected value, by choosing the strategy
that has the highest expected value.
• In healthcare models, there may be a balance between multiple payoffs.
• For example, minimize cost AND maximize life expectancy.
17
Module 1: Decision Trees
Optimal Strategy – Lottery Example
• Optimal strategy – the decision that chooses the strategy with the “best” expected
value.
• In this example, buying a ticket for Lottery 2 maximizes our expected reward, therefore
it is the optimal strategy in this decision analysis.
18
Module 1: Decision Trees
Decision Trees – Key points
• Decisions and strategies
• Pathways
• Probabilities
• Payoffs
• Expected Value
• Optimal strategy selection
19
Module 1: Decision Trees
Homework assignment
Die roll game:
In this game, you have three choices of how to play. Each has its own goal and reward.
1. Roll a die once and win $100 only if you roll a “6”.
2. Roll a die twice and win $125 if both rolls are “5” or greater.
3. Roll a die three times and win $110 if all three rolls are “4” or greater.
Build a model to identify the choice above that maximizes your reward.
20
Module 2: Decision Analysis in Healthcare
21
Module 2: Decision Analysis Healthcare
Measurement and Valuation of Health – What is decision analysis in
healthcare?
• Health care systems are created by societies to maintain health and well-being.
• Systems are expected to run efficiently, however we tend to find gaps in quality, safety,
equity and access in our health care systems.
• Efficiency relates to maximizing the quality of a comparable unit of health care
delivered or unit of health benefit achieved for a given unit of health care resources
used.
• Health care cost increases continue to outpace the rise in wages, inflation, and
economic growth. Resources are finite. Wants are infinite.
• Decision analysis helps decision-makers reconcile spending with maximizing optimal
health outcomes for individuals or populations, and serves an important role in
healthcare decision-making.
22
Module 2: Decision Analysis Healthcare
Model Perspective
Model perspective - The point of view from which the costs and benefits are recorded and
assessed. The choice of perspective must be derived logically from the research question.
Perspective affects the measurements that get included in our decision analysis model.
• Healthcare Provider
• Patient
• Society
• Government
• Employer
23
Module 2: Decision Analysis Healthcare
Types of economic evaluation - Single outcome
• Always looking to choose the optimal strategy
• What is the evaluation method by which you will make your decision or choice?
• If there a single primary outcome, you will choose the strategy with either the highest
expected value or the lowest value.
• Highest – Life Expectancy, QALYs, etc.
• Lowest – Costs
• Typically, however, we are trying to balance multiple competing outcomes:
• For example, a new intervention improves health benefit or efficacy, but comes at
an increased cost.
• We want to minimize cost AND maximize life expectancy.
24
Module 2: Decision Analysis Healthcare
Putting economic evaluation into consideration - Heart attacks
• Imagine there is a $10 Million magic pill that prevents heart attacks in people forever.
• Does it make sense to pay for this pill?
• Given the benefit it provides, is it worth paying for this magic pill?
• Consider that there are other causes of death.
25
Module 2: Decision Analysis Healthcare
Types of economic evaluation – Cost-effectiveness analysis
• Cost-effectiveness: In the context of health and medicine, cost-effectiveness analysis
(CEA) is a method for evaluating tradeoffs between health benefits and costs resulting
from alternative courses of action.
• Cost-effectiveness analysis measures the incremental cost of achieving an incremental
health benefit.
• For example, a new treatment might cost
an additional $10K to increase life expectancy
by one year.
Incremental
Life expectancy
Incremental
Cost
• Cost-effectiveness analysis requires
measurements for cost and effectiveness.
26
Module 2: Decision Analysis Healthcare
Outcome measures – Examples of Costs
There may be many different types of costs that comprise the overall cost of a treatment
strategy.
Direct Costs – all consumption of resources (including direct medical and non-medical) resulting
from a treatment or therapy
• Treatment costs arise directly from the treatment (e.g. diagnosis, drug therapy, medical
care, office visits, in-patient treatment, devices, surgery, etc.).
• Other costs arise from the consequences of the disease or treatment (e.g.
hospitalizations, ER visits, adverse events, etc.).
Indirect Costs - represent changes in resources that occur not directly in relation to the
treatment of the disease (e.g., productivity lost, caregiver time and resources, transportation,
etc.)
• Typically not included in healthcare models
27
Module 2: Decision Analysis Healthcare
Outcome measures – Examples Clinical Effectiveness
There are also different health outcomes that can be used to measure effectiveness.
Quality Adjusted Life Year (QALY)* – The most common clinical/health outcome measure
used in healthcare decision-making to assess the value for money of medical
interventions. It is a generic measure of disease burden, including both the quality and
the quantity of life lived.
Utility* – (quality measure) the preference or value that an individual or society gives a
particular health state. It is generally a number between 0 (representing death) and 1
(perfect health).
Life Years (quantity measure) – measures quantity of years lived, but does not incorporate
the quality of a person’s life
A treatment will result in higher QALYs the longer you live and the healthier you are.
28
*As defined by the National Institute for Health and Care Excellence (NICE)
Module 2: Decision Analysis Healthcare
Outcome measures - QALYs
The QALY can be calculated using the following formula:
Years of Life x Utility Value = #QALYs
For example:
• If a person lives in imperfect health (utility is 0.8) over 5 years, that person will
accumulate 4 QALYs.
(5 Years of Life x 0.8 Utility Value = 4 QALYs)
• Note that utilities can change over time, resulting in more complex calculations.
29
Module 2: Decision Analysis Healthcare
Outcome measures – Other Clinical/Health
Alternative health outcome measures:
• Disability-Adjusted Life Years (DALYs)
• Infections
• Hospitalizations
• ER visits
• Successful transplants
• Deaths
30
Module 2: Decision Analysis Healthcare
Cost-effectiveness for comparing strategies
Competing
strategy is
dominated
Incr. Cost
More costly
What if an Intervention is less effective and more costly than existing treatments? There
is no question that this is not something we should fund.
Less costly
Incr. Effectiveness
Less Effective
More Effective
Reference strategy is at
the center of the graph
and represented by the
blue circle
31
Module 2: Decision Analysis Healthcare
Cost-effectiveness for comparing strategies
Incr. Cost
Cost
increasing
What if a new, competing intervention is more effective and less costly than existing
treatments? There is no question that this is the best option for us to fund.
Incr. Effectiveness
Cost
saving
Competing
strategy is
dominant
Less
Effective
More
Effective
Reference strategy is at
the center of the graph
and represented by the
blue circle
32
Module 2: Decision Analysis Healthcare
Cost-effectiveness for comparing strategies
Incr. Cost
Cost
increasing
The question we are most often dealing with is - What if an intervention is more
expensive and more effective than current treatments? Is it worth it for us to pay for this
new treatment? Does the additional effectiveness justify the additional cost?
Optimal
strategy is
determined
by ICER
Cost
saving
Incr. Effectiveness
Optimal
strategy is
determined
by ICER
Less
Effective
More
Effective
Reference strategy is at
the center of the graph
and represented by the
33
blue circle
Module 2: Decision Analysis Healthcare
Incremental cost-effectiveness plane
• Cost-effectiveness analysis provides a mathematical basis to measure the relative value
of the increased cost vs. improved health outcomes between competing strategies.
Incr. Cost
Cost
increasing
• In these cases, the estimation of value is based upon calculation of an Incremental CE
Ratio (ICER).
Cost
saving
Incr. Effectiveness
Reference strategy is at
the center of the graph
and represented by the
blue circle
Less
Effective
More
Effective
34
Module 2: Decision Analysis Healthcare
Cost effectiveness - Incremental costs vs. incremental effectiveness
We can only rationally justify higher cost strategies if they provide sufficient increases in
health outcomes.
• In cost-effectiveness analysis, this is the incremental cost-effectiveness ratio (ICER) and
it is defined as the difference in cost (C) divided by the difference in effectiveness (E).
CNew – CReference
ICER =
_____________
ENew – EReference
• The ICER measures the additional cost per additional unit of effectiveness.
*Note – incremental values are always calculated by comparing a more expensive strategy to a less expensive strategy.
35
Module 2: Decision Analysis Healthcare
Willingness to Pay
• The ICER can be used as a decision rule in resource allocation.
• If a decision-maker is able to establish a willingness to pay value for the outcome of
interest, it is possible to adopt this value as a threshold.
• The Willingness to Pay (WTP) represents the additional amount a decision-maker is
willing to pay per additional unit of effectiveness, thereby setting a limit on the ICER.
36
Module 2: Decision Analysis Healthcare
Willingness to Pay
• If for a given intervention the ICER is above this WTP threshold it will be deemed too
expensive and thus should not be funded, whereas if the ICER lies below the WTP
threshold the intervention can be judged as cost-effective.
• On a CEA graph, the slope of the ICER and WTP lines provide this information.
ICER from less expensive and more expensive strategy
is < WTP, so we can fund the new intervention
ICER from less expensive and more expensive strategy
is > WTP, so we should not fund the new intervention
37
Module 2: Decision Analysis Healthcare
Multiple strategies
• There may be more than two strategies that need to be compared.
• When there are multiple strategies, you will need to consider comparisons in pairs,
starting with the least costly strategy.
Drug 2 vs. Drug 1:
ICER > WTP = Drug 2
not preferred
Drug 1 vs. no
treatment:
ICER < WTP = Drug 1
preferred
38
Module 2: Decision Analysis Healthcare
Multiple Strategies - Dominance
• Dominated strategies are not on the cost-effectiveness frontier, and are therefore
“dominated” by other, more cost-effective strategies.
39
Module 2: Decision Analysis Healthcare
Other types of economic evaluation
• Cost-effectiveness is the most commonly used economic analysis in healthcare, but there are
other approaches including:
• Cost-minimization - Cost-minimization analysis is a method of calculating treatment costs to
project the least costly drug or therapeutic modality.
• For example, budget impact models (BIMs)
• Comparative effectiveness - evaluating and comparing health outcomes and the clinical
effectiveness, risks and benefits of 2 or more medical treatments, services or items
• Cost-utility - Cost-utility analysis (CUA), a subset of CEA, is used to determine cost relative to
changes in utilities, especially quality of life. CUA is generally applied when multiple patientrelevant clinical outcome parameters are expressed in different units.
• Cost-benefit - Cost-benefit analysis is used to value both incremental costs and outcomes in
monetary terms and therefore allows a direct calculation of the net monetary cost of
achieving a health outcome.
40
Module 2: Decision Analysis Healthcare
Homework assignment
• There are 2 strategies for treating a disease:
• Standard of Care (SOC) – it costs $10K per year, which provides an average Life
Expectancy of 10 years at a constant utility of 0.75
• Novel treatment – it costs $15K per year, which provides an average Life
Expectancy of 11 years at a constant utility of 0.85
• Assignment:
• Calculate the ICER between the two strategies (without a model).
• Determine the optimal strategy based on a WTP threshold of $50K and explain
why.
41
Module 3: Building a Healthcare Decision Tree Model
(Medicine vs. Surgery)
42
Module 3: Building a Decision Tree Model
Defining the healthcare problem
Before you build any model, you need to have a firm understanding of the goals and
measurable outcomes within the model.
To establish this understanding, answer the following 3 questions:
• What is the underlying problem?
• What are our strategies/options?
• How do we measure the outcomes associated with these strategies?
43
Module 3: Building a Decision Tree Model
Defining the healthcare problem
Establishing decision points and strategies:
• What is the underlying problem?
Patients with Disease X have historically only had one treatment option. A new treatment
has become available. We want to identify the most cost-effective treatment for patients
with Disease X.
44
Module 3: Building a Decision Tree Model
Developing your model – defining strategies
Establishing decision points and strategies:
• What are our alternatives?
The current SOC for treating this disease is surgery.
There is a new medicine available that can also be used to treat patients for this disease.
This medicine is more expensive than the SOC but has the potential to be more effective
than the SOC.
Thus, we have 2 options for treating patients that we want to consider in our decision
analysis:
1. Medicine (New)
2. Surgery (SOC)
45
Module 3: Building a Decision Tree Model
Create decisions and strategies
• In this healthcare decision problem, we have two strategies to consider in treating
patients with Disease X.
• In a model, this is represented by a decision node with a branch for each strategy.
46
Module 3: Building a Decision Tree Model
Developing your model – defining outcomes
• How do we measure the outcomes associated with these strategies?
• For this model, we will use a traditional cost-effectiveness analysis
• Cost is measured in US$
• Effectiveness is measured in QALYs
• The model must be configured for cost-effectiveness.
47
Module 3: Building a Decision Tree Model
Developing your model – define patient pathways for each strategy
Defining pathways:
• Surgery:
• It costs $50K one time for the procedure.
• There is a 75% probability of success with surgery.
• Medicine:
• It costs $7K per year for the treatment over the course of a patient’s lifetime.
• There is a 80% probability of success with Medicine.
• We will know if a patient fails on medicine in Year 1, therefore in the model if a
patient fails on Medicine, treatment is stopped after Year 1.
• With either Surgery or Medicine, if treatment is successful, patients have an average
Life Expectancy of 12 years at a constant utility of 0.85
• If either treatment fails, patients have an average Life Expectancy of 8 years at a
constant utility of 0.70
48
Module 3: Building a Decision Tree Model
Developing your model – create parameters to build the model
Defining parameters:
• Creating variables for your parameters provides transparency, consistency and flexibility.
• Parameter variables should be defined once numerically at the beginning of your
model.
Parameter
Variable Name
Value
Probability of success with Surgery
pSuccessSurg
0.75
Probability of success with Medicine
pSuccessMed
0.80
Cost of Surgery
cSurgery
50K
Cost of Medicine
cMedAnnual
7K
Life Expectancy Post Treatment Success
lePostSuccess
12
Life Expectancy Post Treatment Failure
lePostFailure
8
Utility Success Post Treatment
uPostSuccess
0.85
Utility Failure Post Treatment
uPostFailure
0.70
49
Module 3: Building a Decision Tree Model
Developing your model – define patient pathways for each strategy
• Each strategy consists of a chance node (represented by a ), with branches to the right
to represent possible outcomes – in this example there are two outcomes for each
strategy:
• Success
• Failure of treatment
• Most models will have a more complex structure.
50
Module 3: Building a Decision Tree Model
Developing your model – adding probabilities
• For each outcome, we need to define the probability of that outcome occurring.
• We know the probability of success with Medicine and the probability of success with
Surgery.
• The parameter variables ‘pSuccessMed’ and ‘pSuccessSurg’ represent the probability of
success for each strategy.
• For each chance node, probabilities must sum to 1. In TreeAge Pro, the complement to
each known probability [1-pSuccessMed or 1-pSuccessSurg] is represented by a “#”.
51
Module 3: Building a Decision Tree Model
Adding outcome measures
• Each terminal node ( ) terminates and represents a full patient pathway and now we
need to add outcome measures associated with each pathway.
• Let’s consider the first pathway: Success from Medicine
52
Module 3: Building a Decision Tree Model
Adding outcome measures
Success from Medicine pathway
Costs:
• Medicine costs $7K per year over the course of the full life expectancy of 12 years.
• cMedAnnual * lePostSuccess = $7K * 12 = $84K
Effectiveness:
• Utility with successful treatment is 0.85 over the course of the full life expectancy of 12
years.
• uPostSuccess * lePostSuccess = 0.85 * 12 = 10.2 QALYs
• In your model enter the formula referencing your parameters, NOT the calculated value.
• Again, using variables as demonstrated above keeps the model flexible by allowing
you to change your parameter values at any time.
53
Module 3: Building a Decision Tree Model
Adding outcome measures
• Let’s consider the next pathway: Failure from Medicine
54
Module 3: Building a Decision Tree Model
Adding outcome measures
Failure from Medicine pathway
Costs:
• Medicine costs $7K per year of treatment, but Medicine is stopped after 1 year in this
scenario.
• cMedAnnual * 1 Year = $7K * 1 = $7K
Effectiveness:
• Utility with treatment failure is 0.7 over the course of the full life expectancy of 8 years.
• uPostFailure * lePostFailure = 0.7 * 8 = 5.6 QALYs
55
Module 3: Building a Decision Tree Model
Adding outcome measures
• The next two slides will walk through Success and Failure pathways associated with the
Surgery strategy.
56
Module 3: Building a Decision Tree Model
Adding outcome measures
Success from Surgery pathway
Costs:
• Surgery costs $50K only one time over the course of the full life expectancy of 12 years.
• cSurgery = $50K
Effectiveness:
• Utility with successful treatment is 0.85 over the course of the full life expectancy of 12
years.
• uPostSuccess * lePostSuccess = 0.85 * 12 = 10.2 QALYs
57
Module 3: Building a Decision Tree Model
Adding outcome measures
Failure from Surgery pathway
Costs:
• Surgery costs $50K one time over the course of the full life expectancy of 8 years.
• cSurgery = $50K
Effectiveness:
• Utility with treatment failure is 0.7 over the course of the full life expectancy of 8 years.
• uPostFailure * lePostFailure = 0.7 * 8 = 5.6 QALYs
58
Module 3: Building a Decision Tree Model
Completed Model
• Our model is now complete and ready for analysis.
59
Module 4: Analyzing a Healthcare Decision Tree
Model
(Medicine vs. Surgery)
60
Module 4: Analyze the Tree
Analyze the Tree
• We want to compare Surgery against Medicine to make sure we are utilizing our
healthcare dollars wisely. Ultimately, we want to know which is more cost-effective.
• Evaluate the strategies independently and then compare them.
• How do we evaluate each strategy?
• How do we compare the strategies?
61
Module 4: Analyze the Tree
Analyze the Tree – Evaluate each strategy
• Let’s start with the Medicine strategy.
• We have outcomes associated with our patient pathways:
• Success = $84K and 10.2 QALYs
• Failure = $7K and 5.6 QALYs
• We need a weighted average of those pathways to determine an overall expected value
for the Medicine strategy. This is where probabilities are utilized.
• Cost:
$84K * 0.8 + $7K * 0.2 = $68.6K
• Effectiveness:
10.2 * 0.8 + 5.6 * 0.2 = 9.28 QALYs
• Average values weighted by the probabilities ensure more likely pathways have a
bigger impact on our overall strategy than less likely pathways.
62
Module 4: Analyze the Tree
Analyze the Tree – Evaluate each strategy
• Let us now evaluate the Surgery strategy.
• We have outcomes associated with our patient pathways:
• Success = $50K and 10.2 QALYs
• Failure = $50K and 5.6 QALYs
• We need a weighted average of those pathways to determine an overall expected value
for the Surgery strategy.
• Cost:
$50K * 0.75 + $50K * 0.25 = $50K
• Effectiveness:
10.2 * 0.75 + 5.6 * 0.25 = 9.05 QALYs
• Each terminal node has a “P =“ expression which represents the likelihood of that
63
pathway relative to all pathways in the strategy (Product of all probabilities along path).
Module 4: Analyze the Tree
Analyze the Tree – Compare Strategies
• The model is ready to run Cost-Effectiveness Analysis.
• Let us do it first by hand, and then show how TreeAge Pro can do it for you.
• As a reminder, cost-effectiveness analysis provides a mathematical basis to measure the
relative value of the increased cost vs. improved health outcomes between competing
strategies (i.e., increased effectiveness).
• In cost-effectiveness analysis, this is the incremental cost-effectiveness ratio (ICER) and
it is defined as the difference in cost (C) divided by the difference in effectiveness (E).
CNew – Creference
ICER = -----------------------ENew – Ereference
=
68600 – 50000
-----------------------
9.28 – 9.05
=
18600
---------------- = 80870
0.230
64
Module 4: Analyze the Tree
Analyze the Tree – Compare Strategies
We can run the Rankings Report in TreeAge Pro to analyze the model and generate an
ICER.
CNew – Creference
ICER = -----------------------ENew – Ereference
=
68600 – 50000
----------------------9.28 – 9.05
=
18600
---------------- = 80870
0.230
65
Module 4: Analyze the Tree
Analyze the Tree – Willingness to Pay (WTP)
We can now compare the ICER to a willingness-to-pay (WTP) threshold:
• ICER = the amount we have to pay for each additional unit of effectiveness
• WTP = the amount we are willing to pay for each additional unit of effectiveness
The analysis can then tell us is the ICER too high relative to the WTP?
• ICER <= WTP, choose the more expensive/effective option as we can justify the higher
cost
• ICER > WTP, choose the less expensive/effective option, as we cannot justify the higher
cost
66
Module 4: Analyze the Tree
Analyze the Tree – Choose the Optimal Strategy
• The ICER calculated is $80.87K. If our Willingness to Pay Threshold is $50K, then this
ICER value is higher than our WTP, which means we cannot recommend funding the
new treatment, Medicine.
• Therefore, Surgery is the optimal strategy.
• If there were dominated strategies, they would appear in the second grouping and not
67
rd
th
the first. You will never use the 3 and 4 groupings to determine the optimal strategy.
Module 4: Analyze the Tree
Analyze the Tree – Choose the Optimal Strategy
• The TreeAgo Pro CEA graph function also compares strategies and identifies the optimal
strategy.
• A WTP line can be added to your graph as was done below. The WTP line will always
pass through the optimal strategy.
68
Module 4: Analyze the Tree
Analyze the Tree – Choose the Optimal Strategy via Net Benefits
• Net Benefits Calculations combine cost, effectiveness and WTP into a single value.
• The optimal strategy will have the highest Net Benefit value.
• Net Monetary Benefits (NMB) = Effectiveness * WTP – Cost
• Surgery:
9.05 * 50000 – 50000 = 402500
• Medicine: 9.28 * 50000 – 68600 = 395400
• Net Health Benefits (NHB) = Effectiveness – Cost / WTP
• Both will identify the optimal strategy.
• The NMB calculation in the Rankings Report also confirms Surgery as the optimal
strategy.
69
Module 4: Analyze the Tree
Homework Assignment
• For Disease X, there exists surgery and generic medicine as treatment options. A New
Medicine is entering the market and we need to determine which of the 3 treatments is
the most cost-effective.
• Surgery is a one-time cost of $50K. There is a 1% chance of dying from surgery.
• Generic medicine costs $7K per year over the course of a lifetime. If a patient fails on
generic medicine, treatment ends after one year.
• New medicine costs $12K per year over the course of a lifetime. This treatment is also
discontinued after one year if a patient fails on the New medicine.
• Probability of success with surgery is 75%, success with Generic medicine is 70% and
success with New medicine is 80%
• Your WTP threshold is $50K.
• Build a model to identify the optimal strategy and describe how you came to that
conclusion.
• Are there any dominated strategies? If so, which one(s) and how do you know?
70
Module 5: Considering Uncertainty
71
Module 5: Considering Uncertainty
Completed Model
Our model is now completed and has been analyzed using a single value for each
parameter based on our best research and estimates.
72
Module 5: Considering Uncertainty
Confidence in Parameters - Sensitivity Analysis
But, how confident are we about those parameters?
For example:
• In the model, we assume the price of a new medicine is $7K/year. What if the drug is
actually priced less at $6K, or more at $8K when it launches?
• In the model, we assume the probability of success with surgery is 75%. What if that
probability is closer to 70% in reality?
73
Module 5: Considering Uncertainty
Confidence in our Conclusions - Sensitivity Analysis
Given the uncertainty in our parameters, how confident are we of our conclusions?
The application of Sensitivity Analysis to our model assesses the extent to which a model’s
calculations and recommendations are affected by uncertainty.
• Specific questions about the model sensitivity analysis can help answer are:
• Is a model sensitive to a particular uncertainty? E.g., does varying a parameter’s
value result in changes in optimal strategy?
• How does uncertainty related to multiple parameters affect our overall confidence
in the conclusions/recommendations?
74
Module 5: Considering Uncertainty
One-Way Sensitivity Analysis
One-Way Sensitivity Analysis:
• Consider a parameter over a range of values around the base case value.
• Re-evaluate the model multiple times across that range.
• Is there a change in strategy within that uncertainty range?
• Example: pSuccessSurg base case is 0.75 in the Medicine vs. Surgery model. Let us
consider a range of values reflecting our uncertainty in this parameter by running 1-way
sensitivity analysis over the range of 0.7 to 0.78.
75
Module 5: Considering Uncertainty
One-Way Sensitivity Analysis
• If we run our analysis using the range of pSuccessSurg values, we find that at the lower
values within the uncertainty range (0.7, 0.71), Medicine becomes the optimal strategy,
with ICERs below our WTP threshold of $50K.
Medicine is
the optimal
strategy
Surgery is
the optimal
strategy
76
Note: Not pasted here - Report includes outputs up to pSuccessSurg values of 0.78
Module 5: Considering Uncertainty
One-Way Sensitivity Analysis
• The Net Benefits (CE Thresholds) graph illustrates the pSuccessSurg threshold where the
optimal strategy (highest NMB) shifts from Medicine to Surgery.
77
Module 5: Considering Uncertainty
Tornado Diagrams
• A Tornado Diagram is a set of one-way sensitivity analyses brought together in a single
graph. It can include any number of parameters defined in the tree.
• The horizontal bars depict how much the uncertainty of each parameter affects the
ICER.
• Red indicates an increase in the value from the base case, whereas blue indicates a
decrease in the value from base case.
ICER increases as
parameter value
increases
CE threshold line
ICER decreases as
parameter value
increases
Base case ICER
line
78
Module 5: Considering Uncertainty
Probabilistic Sensitivity Analysis (PSA)
• Uncertainty always exists in multiple parameters in a model.
• Studying parameters independently does not fully represent the overall uncertainty of
the model.
• Different parameter combinations could impact optimal strategy. It may take outliers of
multiple parameters to cause a change in optimal strategy.
• Probabilistic Sensitivity Analysis (PSA) enables us to study the combined uncertainty
across multiple parameters.
• PSA results estimate the total impact of uncertainty on the model, or the confidence
that can be placed in the analysis results.
79
Module 5: Considering Uncertainty
Probabilistic Sensitivity Analysis (PSA) - Distributions
• PSA requires a distribution for each
parameter included in the analysis.
• In Example model Medicine vs.
Surgery_PSA, we have set up distributions
for 4 parameters in the model.
• This distribution represents the
uncertainty around probability of success
with surgery (Dist_pSuccessSurg).
• Beta distribution type
• Mean – 0.75
• Standard deviation – .03
• Sample per EV (once per model calc.)
80
Module 5: Considering Uncertainty
Probabilistic Sensitivity Analysis (PSA) - Distributions
• Use the “Graph It” tool to sample each distribution and validate that it appropriately
reflects your uncertainty.
• The histogram below represents the range of samples generated for Dist_pSuccessSurg.
• Note that the distribution mean is 0.75, so these values look reasonable.
81
Module 5: Considering Uncertainty
Probabilistic Sensitivity Analysis (PSA) - Distributions
• Distributions must be referenced in the model (4 total in this model).
• Variable pSuccessSurg is now set equal to the distribution.
• When you run CEA, the variable uses the mean from the distribution.
• When you run PSA, the variable uses the appropriate sample from the distribution.
82
Module 5: Considering Uncertainty
Probabilistic Sensitivity Analysis (PSA) – Running Analysis
• PSA calculates the model many times with different sets of sampled parameter data from
your distributions.
• Sample distributions, calculate the model, and then repeat.
• To run the analysis in TreeAge Pro:
• Choose the Root Node of the model.
• From the menu, choose Analysis > Monte Carlo Simulation > Sampling
• You will be presented with aggregated output;
however, our focus is on the PSA outputs.
• We are going to highlight the
Acceptability Curve and the
ICE Scatterplot outputs.
83
Module 5: Considering Uncertainty
Probabilistic Sensitivity Analysis (PSA) – CE Acceptability Curve
• The CE acceptability curve shows the percentage of model recalculations that favor
each strategy across a range of WTP.
• As WTP increases, more of the recalculations will favor the more effective strategy.
• At a WTP of $50K, it shows Surgery is the optimal strategy approximately 70% of the
time.
84
Module 5: Considering Uncertainty
Probabilistic Sensitivity Analysis (PSA) – ICE Scatterplot
• The ICE scatterplot also shows us that 70% favored the less expensive strategy, Surgery,
as depicted by the dots above and to the left of the WTP line on the graph.
• In almost all cases, Medicine was more expensive (IC > 0).
• In a solid majority of cases, Medicine was more effective (IE > 0).
• Given our PSA results, how confident are we that Surgery is the more cost-effective
strategy?
85
Module 5: Considering Uncertainty
Homework Assignment
Run PSA on the Classroom Medicine vs. Surgery Model built in Modules 3 and 4.
• Run PSA using the following three parameter distributions:
o Add a Beta Distribution for pSuccessNewMed (Mean: 0.80; Std Dev .03)
o Add a Gamma Distribution for cSurgery (Mean: 50000; Std Dev 5000)
o Add a Gamma Distribution for cNewMedicine (Mean: 12000; Std Dev 3000)
• Reference the distributions in the model.
• Run the PSA and interpret the results.
86
Section 6: More Advanced Modelling Approaches and
When to Apply Them
87
Module 6: Advanced Modelling Approaches
Markov Models
• Markov modelling is a technique that allows presentation and analysis of disease
progression over time.
• It is particularly suitable for diseases that are chronic and recurrent in nature.
• Markov models take a long disease progression and break it down into shorter time
cycles that repeat to cover the entire model time horizon.
• Markov model structure consists of health states and events/transitions.
• Markov cohort analysis sends a cohort of patients through disease progression
pathways consisting of health states and events/transitions, cycle by cycle until the
end of the total time horizon.
• In each cycle, across the time horizon, the analysis will accumulate health utility and
cost, which can be attached to any health state and/or event.
88
Module 6: Advanced Modelling Approaches
Decision Trees Vs. Markov Models
Decision Trees:
• Used for “one off” decisions
• Particularly suited to:
• Acute care problems (“kill or cure”)
• Once-only diseases/conditions
• Short-term diagnostic/screening decisions
Markov Models:
• Represent disease processes which evolve over time
• Suited to modelling the progression of chronic disease
• Can handle recurrence
• Estimate long term costs and life years gained/QALYs
89
Module 6: Advanced Modelling Approaches
Markov Models – Example
Events/Transitions
Health States
Terminal Nodes
(return cohort to
health states for
next cycle)
Markov Node
(start of cycle)
90
Module 6: Advanced Modelling Approaches
Markov Models – Cohort Analysis
Cohort flow
through
events/transitions
Accumulation of
costs and
effectiveness
91
Module 6: Advanced Modelling Approaches
Individual vs. Cohort
• All of the models we have considered thus far have been cohort models, which analyze
data for populations.
• Cohort analysis does not allow for patient-level data.
• If we run individual patients through the model via microsimulation, we can then
associate data with each patient which enhances our model capabilities in several ways:
• Heterogeneity – each patient can have his/her own set of patient characteristics
• Event tracking – events that occur to each patient can be recorded
• Heterogeneity and event tracking allow patients to take their own individual paths
based on their unique characteristics and prior events.
• In many cases, this creates a more robust, realistic model.
92
Module 6: Advanced Modelling Approaches
Discrete Event Simulation
Discrete Event Simulation (DES) models are similar in structure to Markov models, but
instead of patients moving from cycle to cycle in fixed time increments , patients move
from event to event based on the sampled (variable) timing of those events happening.
• Similar to Markov models, patients will accumulate cost and effectiveness at any time
point in the patient pathway.
• DES models are computationally more efficient than Markov models in that patients
move from event to event vs. moving through cycles for events to occur.
93
Module 6: Advanced Modelling Approaches
Discrete Event Simulation - Example
• Time-to-event vs. probabilities
• At a basic level: as a probability of an event increases, the time to event decreases
• If we take the example below, the DES node (D) is similar to the Markov node as it starts
with the health states to the right.
• Transitions are calculated based on the time to that event occurring as opposed to
probability of that event occurring.
• Time to event is derived from distributions, and re-sampled after event, therefore DES
models require microsimulation.
94
Other complex models
Interactions among individuals in the model [parallel trials]
• Patient pathways may be affected by other patients in the model
• Infectious Disease models where the likelihood of infection may depend on the
percentage of the cohort that is currently infected
• Resource constraints/queues where patients may compete for a resource like a hospital
bed
Dynamic Cohort
• Studying a patient population as a whole rather than looking for average values per
patient
• Could be used for budget impact
95