presentation to:
2012 Midwest Biopharmaceutical
Statistics Workshop
May 22, 2012
by Harry J. Smolen, President and CEO
Medical Decision Modeling Inc.
Indianapolis, IN, USA
Discussion Agenda
 General background on
pharmacoeconomic (PE) models
 Types of models commonly used in
healthcare technology assessment
 Methods for selecting modeling
method
 Cost-effectiveness analysis Overview
General PE Modeling Background
What is a model?
A model is a hypothetical description of
a system
General PE Modeling Background
Why not experiment with the actual
system?
 If it is possible to experiment with the
actual system, do it!
 However, especially in healthcare, it
is frequently too expensive, too
disruptive, too slow, and/or unethical
to experiment with the actual system
General PE Modeling Background
Example “experiment”: How many life years
would be saved if every other person in the US
>65 years had a one-time colonoscopy, with
polypectomy and surveillance every three to
five years for positive findings?
 Expensive – (1/2) x 40.3M persons x $1,714
≈$34,537,100,000 + follow-up costs
 Disruptive – a day of inconvenience per
patient for prep and exam; also disruption to
gastroenterologists, is there enough capacity?
 Slow – Many years to get results
 Possibly Unethical – though uncommon,
perforations occur (≈0.9/1000 colonoscopies)
with unknown benefit to patients
Computational vs. Physical Models
 When most people think of models they
envision physical models, e.g., clay cars in
wind tunnels or life-size layouts of an
emergency room for workflow analysis
 The types of models used to evaluate the
effectiveness and cost-effectiveness of
healthcare technologies are computational
models
 Computational models represent a system in
terms of logical and quantitative relationships
that are manipulated to examine how the
model reacts, and thus how the system
would react if the model is valid1
1. Law and Kelton, Simulation Modeling and Analysis, 2nd edition, 1991.
Simulation vs. Analytical Solution
A simple computational model:
d = rt, where
d = distance traveled
r = rate of travel
t = time spent traveling
 In this simple model it is possible to get an
exact, analytical solution
 This equation may apply to a single car on a
test track, but …
 Is of little use to determining distance traveled
on a busy highway with varying speeds and
levels of congestion
Simulation vs. Analytical Solution
In this simple d = rt model it is possible to get an
exact, analytical solution
 Some analytical solutions can become
extraordinarily complex, e.g., inverting a nonsparse matrix, and require substantial computing
power
 If an analytical solution to a computational model is
available and computationally efficient, study the
model in this manner
 However, many systems are highly complex so that
valid computational models of them are themselves
complex, precluding the possibility of an analytical
solution
 Such models must be analyzed by simulation, i.e.,
numerically varying the relevant model inputs to
determine how they affect the output measures of
interest1
1. Law and Kelton, Simulation Modeling and Analysis, 2nd edition, 1991.
Evidence from RCTs
 Evidence from randomized controlled
trials (RCTs) remain the highestquality data source for evaluating the
efficacy of health care interventions
 However, evidence from RCTs alone
can be uninformative if the RCT
endpoints are not translated into
measures that are valued by patients,
providers, insurers, and policy
makers1
1. Weinstein MC, et al. Value Health. 2003 Jan-Feb;6(1):9-17.
Evidence from RCTs
 For example, in patients with
osteoporosis, fracture risk evolves over
a lifetime from the development of peak
bone mass to subsequent bone loss and
the age-related increase in the likelihood
of falling, among other factors1
 It may takes decades to determine the
effects of osteoporosis interventions
 Hence, the most informative
osteoporosis intervention RCTs would
potentially last for decades2
1. NIH Osteoporosis Prevention, Diagnosis and Therapy Consensus Statement 2000.
2. Vanness DJ. Osteoporos Int. 2005 Apr;16(4):353-8.
Evidence from RCTs
 A similar situation exists for many
other chronic diseases such as
diabetes, colorectal and prostate
cancer, and Alzheimer's disease
 The lengthy clinical trials needed to
properly evaluate interventions for
these chronic diseases would require
financial and time opportunity costs
that would make them infeasible1
1. Vanness DJ. Osteoporos Int. 2005 Apr;16(4):353-8.
Evidence from RCTs
 Very few epidemiological studies or
clinical trials are able to measure
disease progression and the impact of
interventions on costs, quality of life,
and health outcomes over a lifetime1
 In the absence of such information, the
most practical method to evaluate the
health outcomes and costs of
interventions is to develop models that
integrate relevant data and extrapolate
to long-term time horizons
1. Brändle M, et al. Curr Med Res Opin. 2004 Aug;20 Suppl 1:S1-3.
Evidence from Models vs. RCTs
 Models that evaluate health care
interventions synthesize evidence on
health consequences and costs from
many different sources, including data
from clinical trials, observational
studies, insurance claim databases, case
registries, public health statistics, and
preference surveys1
1. Weinstein MC, et al. Value Health. 2003 Jan-Feb;6(1):9-17.
Evidence from Models vs. RCTs
 Even when the disease does not require a longterm evaluation period, models can prove
valuable by:
 Using results from indirect comparisons of
individual RCTs to compare treatments not
studied head-to-head and estimate
outcomes not consistently measured
 Allowing the extrapolation of effects to
populations not studied in a particular RCT
 Allowing sensitivity analysis of assumptions
addressing treatment efficacy, health state
utilities, costs, etc.
1. Weinstein MC, et al. Value Health. 2003 Jan-Feb;6(1):9-17.
Decision Model Defined
A decision model is a structured
representation of a decision process that
allows a person to perform a decision
analysis
“… decision analysis is just the systematic articulation of
common sense: Any decent doctor [decision maker]
reflects on alternatives, is aware of uncertainties,
modifies judgments on the basis of accumulated
evidence, balances risks of various kinds, considers the
potential consequences of his or her diagnoses and
treatments, and synthesizes all of this in making a
reasoned decision that he or she decrees right for the
patient. All that decision analysis is asking the doctor
[decision maker] to do is to do this a lot more
systematically and in such a way that others can see
what is going on and can contribute to the decision
process.” 1
1. Raiffa H. Clinical Decision Analysis. Philadelphia:WB Saunders; 1980: ix-x.
Types of Modeling Methods
Types of modeling methods frequently used in
health technology assessment:
 Decision trees
 Markov
 Cohort
 Monte Carlo
 Microsimulation
 Fixed-time advance
 Discrete-event, Time-to-event (without and
with queuing for resources)
 Agent-based
Decision Trees
 A decision tree is a diagrammatic
representation of the possible outcomes and
events used in decision analysis
 The questions to be asked in an analysis of a
question are arranged as a series of decision or
chance nodes, each node with resulting
branches, creating a tree effect
 The sequential steps proceed with each step
depending on the decision or probability
outcome from the preceding step1
1. http://medical-dictionary.thefreedictionary.com/decision+tree
Decision Tree Example
Simple tree fragment modeling complications of
anticoagulant therapy1
1. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Decision Tree Components
Simple tree decision trees embody the essential
paradigm of decision analysis. Specifically, all
decisions may be decomposed into three
broadly-defined components:
1. Decision node – point in time when a choice is
made among competing strategies
2. Decision strategy – set of actions or events
consequent to a decision
3. Outcome nodes – terminal branches of tree
that represent outcomes of a strategy.1
Multiple outcomes (payoffs) may be assigned.
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Calculating Expected Value of a Decision Tree




The expected value of a decision tree is calculated
by “averaging out” or “folding back the branches of
the tree
The value(s) of each strategy is path probability to
the terminal node multiplied by the payoff(s) at
the terminal node
Branching probabilities can be deterministic
represented by point values (e.g., 0.6) or
stochastic represented by probability distributions
(e.g., normal, exponential)
Uncertainty around branching probabilities and
terminal node values is examined with sensitivity
analysis1
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Advantages of Decision Trees



Graphical – can diagrammatically represent
decision alternatives, chance events, and possible
outcomes; visual approach assists with
comprehending decision sequences and
dependencies
Efficient - can quickly express complex alternatives
clearly, and easily modify as new information
becomes available
Complementary – can use in conjunction with
other methodologies, e.g., append recursive
methods to terminal nodes1
1. Olivas R. http://www.stylusandslate.com/decision_trees/download/files/decisiontree_v5_1b.pdf
Disadvantages of Decision Trees
 Must assume population being examined
can be modeled in the aggregate; if being
applied to an individual, assumption is
made that aggregate probabilities are
relevant to the individual1
 Does not specify when events occur
 Assumes that each event can occur only
once

Can address previous two disadvantages with a
recursive tree (see next slide)2
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
2. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Recursive Decision Tree1
Previous terminal nodes of
POST-BLEED, POSTEMBOLUS, AND NO EVENT
are replaced by the chance
node ANTICOAG which
appears at the root of the
tree
Note that with only two time
periods, there are 17
terminal nodes; five periods
would have hundreds of
branches
1. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Markov Models
 Described as “partially cyclic directed
graphs”
 Particularly useful when a decision
problem involves:
 exposure to risks or events over time
 ongoing exposures or situations where the
specific timing of an event is regarded as
important or uncertain
 or where describing the timing of events is
necessary for face validity1
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Markov Models
 Assumes that the patient is always in
one of a finite number of health states
called Markov states
 All events of interest are modeled as
transitions
 Each state is assigned a utility (and
possibly a cost) and the contribution of
this utility depends on the length of time
in the state1
1. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Markov Models
 Common representation of a
simple Markov process called a
state-transition diagram
 Each state represented by a
circle
 Arrows connecting different
states indicate allowed
transitions
 States with arrows to itself
indicate that patient may remain
in that state in consecutive cycles
 Note no transition from
“DISABLED” to “WELL”, nor
“DEAD” to any other state
Markov Models – Time Cycles
 Time horizon of the analysis
divided into equal increments of
time called Markov cycles
 Assumed that a patient can only
make a single state transition
during a cycle
 Length of cycle chosen to
represent a clinically meaningful
time interval
 If time horizon is patient
lifetime, then cycle is usually
one year
 If events occur more
frequently, cycle can be a
month or even a week1
1. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Markov Models – Incremental Utility
 Evaluation of a Markov process yields the
average amount of time spent in each state,
patient is “given credit” for time spent in
each state (e.g., life years)
 Optionally, each state can be associated with
a numerical factor representing the quality of
life in that state relative to perfect health
(e.g., WELL=1.0, DISABLED=0.7,
DEAD=0.0)
 Utility associated with spending one cycle in
a particular state is referred to as the
incremental utility1
1. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Markov Models – Costs
 Analogous to utilities assigned to particular
states, a cost may be specified for each state
representing the financial cost of residing in
that state for one cycle
Markov Models – Types of
 Markov processes are categorized by
whether or not state-transition
probabilities are constant over time
 For example, transition probability from
WELL to DEAD may consist of two
components:
 Probability of dying unrelated to disease in
question – changes over time as patient gets
older
 Probability from disease (e.g., fatal
hemorrhage or embolus during cycle) may or
may not be constant over time1
1. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Markov Models – Transition Probabilities
 For a Markov model of n states, there will
be n2 transition probabilities
 When these probabilities are constant
with respect to time (Markov chains) they
can be represented by n x n matrix1
1. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Markov Models – Types of
 Most Markov models used in health
care are semi-Markov
 Unlike Markov chains, in semiMarkov models state transitions may
be allowed to vary or be timevariant and usually need to be
solved numerically via simulation1
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Markov Models – The Markov Property
 In Markov processes, the behavior of the process
subsequent to any cycle depends only on its
description in that cycle, i.e., the process has no
memory for earlier cycles
 Because of this assumption, the prognosis of a
patient cannot depend on events prior to arriving in
that state1
 For example, patient is WELL after recovering
from an osteoporotic forearm fracture
 Though WELL, the probability of future fracture is
likely higher than for patients in WELL without
history of fracture
 Build additional states to accurately represent
these patients?
1. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Markov Model-Decision Tree Combination1
Markov process used
to represent all
processes
1. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Markov Cohort Simulation1

In the table above, column 5, sum of
number of patients in each state is
multiplied by the incremental utility
of that state

First row does not contribute to sums

Patients will spend on average 1.5
cycles in the WELL state and 1.25 in
the DISABLED state, for a net
unadjusted life expectancy of 2.75
cycles
1. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Markov Monte Carlo Simulation1

As an alternative to cohort
simulation, analysis performed by
individually simulating large numbers
of individual patients (e.g., 104)

At the end of each cycle, a random
number generator is used together
with transition probabilities to
determine to which state the patient
will transition

Outputs from a large number of
individual patients will constitute a
distribution of survival values, the
mean of which should be similar to
expected utility of cohort simulation

Variance measures can also be
determined from Monte Carlo
simulations
1. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Discounting Utilities and Costs
 Costs and benefits occurring immediately are
valued more highly than in the future
 Discounting formula:
Ut = U0 / (1 + d)t
where
Ut is the incremental utility at time t
U0 is the initial incremental utility
d is discount rate
 Analogous discounting is done for costs
Advantage of Markov Models
 Primary advantage over decision
trees is the increased face validity by
capturing extended time horizons via
Markov cycles1,2
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
2. Sonnenberg FA, et al. Med Decis Making. 1993 Oct-Dec;13(4):322-38.
Disadvantages of Markov Models
 State transitions can only occur at
the start/middle/end (whichever is
determined) of a cycle, creating
potential biases
 Cycle time may force simplifying
assumptions regarding transition
probabilities1
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Microsimulations
 In microsimulations, patients are
simulated as individuals (as opposed
to homogenous cohorts) and
represented by software objects
called entities
 One of the primary advantages of
microsimulations is that the patient
entities can be assigned attributes
and the occurrence and timing of
patient events can be determined
from these attributes
Microsimulations
 Microsimulations have the ability to ascribe a
potentially vast number of combinations of
characteristics to individually simulated
patients
 An individual’s health state is defined by its
combination of characteristics, not by its
assignment among limited pre-defined health
states as in Markov cohort simulations
 Eliminates the need for excessive number of
defined health states, as patient transitions to
subsequent health states are dependent upon
patient characteristics, not just their current
health state
Microsimulations, Types of
 Fixed-time advance
 Patients can only transition to
different health states at fixed time
intervals defined by the cycle length
 Time-to-event
 The occurrence (yes/no) and timing of
an event is determined by random
sampling of a probability distribution
Microsimulations, Types of
 Discrete-event simulation (DES)1
 Entities (patients) may interact or compete
with each other for system resources (e.g.,
physicians, hospital beds, etc.)
 Key elements of DES are entities, attributes,
resources, and queues
 When a resource is not available for an
entity, the entity is relegated to a queue
until a resource is available
 Agent-based simulations1
 Independent multi-agent DES
 Agent entities contain information about
their state and decision rules on how to
communicate and interact with other agents
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Choosing Most Appropriate Modeling
Method1
Criteria to be evaluated:
 Project Type
 Population Resolution
 Interdependencies/Feedback
 Treatment of Time
 Treatment of Space
 Resource Constraints
 Autonomy/Freedom of Action of
Entities/Populations
 Embedding of Knowledge
 Availability of Evidential Data
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Choosing Most Appropriate Modeling
Method1
 Project Type
 Model to be used to answer a single
question? KISS, model should only be
complex enough to answer the single
question
 Model to be used programmatically (i.e.,
long term)? Methods that can handle
more complexity usually required
 Population Resolution
 In aggregate? Most simulation methods
can handle
 Individual level? Microsimulation
required
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Choosing Most Appropriate Modeling
Method1
 Interdependencies/Feedback
 Are entity interdependencies important,
e.g., an infectious epidemic involving
exposed, unexposed, infected groups?
DES or agent-based simulations required
 Treatment of Time
 If time treated cumulatively or
instantaneously, then trees can be used
 If changes over time are important, e.g.,
time of therapy, surveillance interval,
then Markov or microsimulations needed
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Choosing Most Appropriate Modeling
Method1
 Treatment of Space
 Is location important to the study
question, e.g., optimal distribution of
mental health professionals in a state,
then microsimulations or agent-based
simulations needed
 Resource Constraints
 If it is important to model limited
resources and waiting lists, then DES or
agent-based simulations required
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Choosing Most Appropriate Modeling
Method1
 Autonomy/Freedom of Action of
Entities/Populations
 Are all potential paths through the model
predefined, e.g., idealized RCT?
 Are absorbing states required, e.g., Markovs?
 Are multiple paths open to the populations being
modeled?, then DES or agent-based simulations
 Embedding of Knowledge Treatment of
Space
 Is the knowledge embedded in the structure of
the model, e.g., decision trees, Markovs?
 Or is the knowledge embedded in the entities,
e.g., microsimulations?
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Algorithm for Choosing a Simulation Method1
CRC = colorectal cancer; AAA = abdominal aortic aneurysm
1. Pickhardt PJ, et al. AJR Am J Roentgenol. 2009 May;192(5):1332-40.
DEEPP = Describe, Evaluate, Explore, Predict, and Persuade; DES = discrete event
simulation; KISS – keep it simple stupid
1. Stahl JE. Pharmacoeconomics. 2008;26(2):131-48.
Cost-Effectiveness Analysis (CEA)
 CEA is a form of analysis that compares the
relative costs and outcomes of competing courses
of action
 Central measure used in CEA is the costeffectiveness (CE) ratio
 Implicit in the CE ratio is a comparison between
alternatives
 The CE ratio for comparing these alternatives is
the difference in their costs divided by the
difference in their effectiveness:
CE ratio = (costA - costB) / (effectivenessA - effectivenessB)
Cost-Effectiveness Analysis (CEA)
 The CE ratio is essentially the incremental
cost of obtaining a unit of health effect
 Health effects (captured in the CE ratio
denominator) can be intermediate (e.g.,
changes in A1c or bone mineral density) or
long term (e.g., lives saved, life years gained,
or quality-adjusted life years [QALYS])
 QALYS are the most comprehensive outcome
measure in CEA in that it incorporates both
quality of life and survival information1
1. Gold et al. Cost-effectiveness in health and medicine. Oxford University Press. 1996
Cost-Effectiveness Analysis (CEA)
 CE ratios expressed as “cost per QALY
gained” provide a standardization which
allows comparisons of incremental value
across different outcomes in a particular
disease or across different diseases
altogether
 different outcomes in a particular disease: e.g.,
diabetes – comparing value in preventing renal
failure vs. lower extremity amputation
 different diseases – e.g., stroke outcomes to
schizophrenia outcomes
CE Plane1
1. Black WC. Med Decis Making. 1990 Jul-Sep;10(3):212-4.
CE Plane1
 Quadrant IV: e>0, c<0, dominant (costsaving)
 Quadrant II: e<0, c>0, dominated
1. Black WC. Med Decis Making. 1990 Jul-Sep;10(3):212-4.
Contact Info
Harry J. Smolen
Medical Decision Modeling Inc.
8909 Purdue Road, Suite #550
Indianapolis, IN 46268
smolen@mdm-inc.com
(317) 704-3800 office
(317) 716-6650 mobile