Decision Analysis and Markov Modeling in Urology
Michael H. Hsieh* and Maxwell V. Meng
From the Department of Urology (MHS, MVM) and Program in Urologic Oncology (MVM), Urologic Outcomes Research Group, University
of California-San Francisco Comprehensive Cancer Center, University of California-San Francisco, San Francisco, California
Purpose: The process of decision making in medicine has become increasingly complex. This has developed as the result of
increasing amounts of data, often without direct information or answers regarding a specific clinical problem. The use of
mathematical models has grown and they are commonly used in all areas. We describe and discuss the application of decision
analysis and Markov modeling in urology.
Materials and Methods: We define decision analysis and Markov models, providing a background and primer to educate
the urologist. In addition, we performed a complete MEDLINE® database search for all decision analyses in all disciplines
of urology, serving as a reference summarizing the current status of the literature.
Results: The review provides urologists with the ability to critically evaluate studies involving decision analysis and Markov
models. We identified 107 publications using decision analysis or Markov modeling in urology. A total of 36 studies used
Markov models, whereas the remainder used standard decision analytical models. All areas of urology, including oncology,
pediatrics, andrology, endourology, reconstruction, transplantation and erectile dysfunction, were represented.
Conclusions: Decision analysis and Markov modeling are widely used approaches in the urological literature. Understanding the fundamentals of these tools is critical to the practicing urologist.
Key Words: urology, decision support techniques, Markov chains
ecision making in medicine has become increasingly
complex for patients and practitioners. This has resulted from factors such as the shift away from physician authority toward shared decision making, unfiltered
information on the Internet, new technology providing additional data, numerous treatment options with associated
risks and benefits, and results from new clinical studies.
Despite the plethora of information, synthesizing this into
practical knowledge can be difficult. In addition, while well
designed trials are performed in many clinical scenarios,
often these studies are difficult to compare with each other
and they ultimately do not directly answer basic questions.
Consequently medical care has evolved toward an evidence based approach. Decision making tools, such as nomograms, and computer based models and algorithms, have
been developed. In urology the most popular applications
have been predicting outcomes after treatment, eg the Kattan model for prostate cancer. We discuss DA and MM, and
their application to urology, serving as a primer and reference for the urological community. We believe that these are
powerful methods of assessing treatment choices and they
will have an increasingly important role in medicine.
D
HISTORICAL AND
CONTEMPORARY PERSPECTIVE
DA is a systematic, quantitative approach to decision making, in which the relative values of different options are
Submitted for publication December 14, 2006.
* Correspondence: 400 Parnassus Ave., UFP, 6th Floor Crede Ambulatory Care Center, Box 0738, San Francisco, California 94143
(telephone: 415-476-6843; e-mail: mhsieh@urology.ucsf.edu).
0022-5347/07/1785-1867/0
THE JOURNAL OF UROLOGY®
Copyright © 2007 by AMERICAN UROLOGICAL ASSOCIATION
compared and uncertainty exists within the system. It has
origins in game theory and it was initially applied to economics and negotiations, as illustrated by the Prisoner’s
Dilemma game and the Cold War politics of nuclear strategy. It has been disseminated in diverse fields, such as oil
exploration, law and engineering. The goals in these situations
as well as in medicine are to use available information to
maximize outcomes. The process is designed to help decision
makers think clearly about the numerous elements of complex
choices, such as the range of possible consequences of action or
inaction, preferences among different consequences and the
impact of unpredictable processes. DA can be used in clinical
scenarios in which clinical trials are unethical or difficult to
perform. When trials are available, DA can extend their findings to issues such as cost-effectiveness.
The first published application of DA to a clinical problem
addressed the role of radical neck dissection in patients with
oral cancer without palpable neck metastases.1 In urology
early studies examined issues of screening and treatment for
prostate cancer, and management of benign prostatic disease.2,3 Information from DA can be used to determine how
to treat an individual patient and formulate policy recommendations regarding a group of patients, and as an aid for
patients making decisions about therapies.
THE DA PROCESS
Medical DA consists of 5 basic steps (Appendix 1). 1) The
clinical problem is identified. 2) The problem is disaggregated
and structured as a decision tree. This graphic representation
depicts the components of the problem and relates action to
consequences. 3) Information needed to populate the decision
tree is gathered from published literature, primary data collec-
1867
Vol. 178, 1867-1874, November 2007
Printed in U.S.A.
DOI:10.1016/j.juro.2007.07.006
1868
DECISION ANALYSIS AND MARKOV MODELING IN UROLOGY
tion and/or expert opinion. 4) The model is analyzed using
baseline information by calculating EV, that is the net values
of the series of actions and events. 5) Sensitivity analysis is
performed, in which parameter values are varied across ranges
to determine their effect on the model.
DECISION TREE DESIGN
Several conventions are used for decision trees (Appendix 2).
The tree is organized from left to right and it consists of
nodes, branches and outcomes (fig. 1, A). A decision node
(square) is a branch point where several options are available to the decision maker. A chance node (circle) is a branch
point where several outcomes are possible but not controlled
by the decision maker. Probabilities are associated with the
events depicted at chance nodes. At any given chance node
the sum of the various probabilities typically equals 100% to
reflect all possible outcomes. Branches connect nodes to
nodes or nodes to outcomes. Outcomes are the consequences
of the final events depicted in the tree. Every outcome state
is associated with payoffs, which are used to calculate the
EV of each branch of the decision tree. Payoffs can be assessed using costs, specific clinical end points, QOL measures and life expectancy.
ANALYZING THE DECISION TREE
The determination of the best treatment choice is based on
Bayes’ decision rule, in which the action that maximizes EV is
selected. Calculating the EV of each treatment, ie options at
the decision node, is called folding back or rolling back the
decision tree. For each branch at the decision node an EV
represents the weighted average of the payoffs of the possible
outcomes. These values for each choice are compared with a
greater value reflecting a better outcome. Figure 1, B shows 2
treatment options at the decision node, including OP and RAP.
In this example the relevant outcome is surgical margin status
with negative and positive assigned payoffs of 1.0 and 0, re-
spectively. Folding back the decision tree shows that the EV for
the OP arm is 0.8 ⫻ 1 (probability of a negative surgical margin
multiplied by the payoff value of a negative surgical margin) ⫹
0.2 ⫻ 0 (probability of a positive surgical margin multiplied by
the payoff value of a positive surgical margin) ⫽ 0.8. The EV for
the RAP arm is 0.7 ⫻ 1 (probability of a negative surgical
margin multiplied by the payoff value of a negative surgical
margin) ⫹ 0.3 ⫻ 0 (probability of a positive surgical margin
multiplied by the payoff value of a positive surgical margin) ⫽
0.7. Hence, the EV for OP is higher.
The calculations involved in analyzing decision trees are
simple arithmetic operations that can be performed manually or through spreadsheet programs. DA software is available (TreeAge, Williamstown and Syncopation Software,
Concord, Massachusetts) and it can simplify the process.
However, folding back models manually can facilitate understanding of the mechanics of DA and how one derives a
seemingly arbitrary EV.
SENSITIVITY ANALYSIS
Sensitivity analysis is a crucial element of DA. It tests the
stability of the conclusions of the base case analysis as well as
the validity of model assumptions. Thus, an explicit statement
of assumptions is important in model design. Frequently the
probabilities of events and the values assigned to outcome
states are uncertain or variable. In addition, studies may show
conflicting findings with various values available for imputation into the model. During sensitivity analysis a specific parameter, eg probability at a chance node, is varied and the
impact on the EV of treatment choices is determined. On 1-way
sensitivity analysis a single parameter is varied across a realistic range, while all other values are kept constant. If the
relative EV of the choices is unchanged, the analysis is insensitive to that variable. However, if the relative EV changes at
a certain value of the parameter being tested, the point at
which the optimal decision shifts from 1 alternative to another
is referred to as the threshold value.
FIG. 1. Decision tree comparing open and robotic assisted laparoscopic prostatectomy. A, end point of interest is surgical margin status with
utility values of 1.0 and 0 for negative and positive margins, respectively. Chance nodes (circles) represent possible outcomes with respect
to margin status and listed fractions represent probability of each outcome state. B, folding back decision tree. Calculated EV for open
prostatectomy is 0.8, representing weighted average of outcome states in that branch. Similarly calculated EV for robotic assisted
prostatectomy is 0.7, representing lower result, reflecting greater chance of positive surgical margins.
DECISION ANALYSIS AND MARKOV MODELING IN UROLOGY
One-way sensitivity analyses can be graphically represented (fig. 2, A). The effect of varying the probability of a
negative surgical margin after OP (x axis) is reflected in the
EV (y axis). The threshold value for the probability of a
negative surgical margin after OP is 70% (dotted line). Multiple parameters can be varied concurrently (fig. 2, B). The
effects of simultaneously varying 2 parameters, that is the
probabilities of a negative surgical margin after OP (x axis)
and a negative surgical margin after RAP (y axis), are reflected in the shaded regions of the graph. At 80% probability of a negative surgical margin after RAP and 20% probability of a negative surgical margin after OP (marked by
asterisk) RAP is the optimal choice.
MMs
A MM is a form of DA suited to represent processes in which
there are transitions in and out of mutually exclusive outcome
1869
states and the rates of transitions vary with time. This is
particularly appropriate for medical DA since patients proceed
among various states of health and health altering events may
occur more than once or with uncertain timing. Similar to DA,
MMs are represented by decision trees after the health states
have been identified, and transition probabilities and outcome
values are applied. Hence, these models also feature decision
and chance nodes but they use absorbing (final) or nonabsorbing (temporary) health states rather than outcome states.
Health states for a MM examining urethral strictures could
include urethral stricture leading to treatment, temporarily
stricture-free and permanently stricture-free with the first 2
being nonabsorbing states and the third being an absorbing
state (fig. 3). In MM it is possible to designate all patients to
start in specified health states. For example, a cohort of patients would start in the urethral stricture leading to treatment health state. Possible outcomes after starting the simu-
FIG. 2. A, 1-way sensitivity analysis in which open prostatectomy is preferred when probability of negative surgical margins is greater than
70%. Point at which EVs for treatment choices are equivalent is threshold value. Note that EV for robotic assisted prostatectomy does not
change since only 1 value, ie probability of negative margin after open prostatectomy, is varied, while keeping other model parameters
constant. B, values subjected to 2-way sensitivity analysis are probabilities of negative surgical margins for open (x axis) and robotic assisted
(y axis) prostatectomy. Other values remain constant. Each shaded region represents combinations of probabilities where 1 treatment is
preferred over another. Boundary between 2 regions represents threshold frontier.
1870
DECISION ANALYSIS AND MARKOV MODELING IN UROLOGY
FIG. 3. Markov model for urethral stricture disease. Result in patient treated for urethral stricture may be 1 of 3 health states, including
recurrent stricture, or temporarily or permanently stricture-free. In first 2 states patient is cycled again, so that patient who is temporarily
stricture-free may remain stricture-free for another cycle or have recurrent stricture. If patient is cured, patient is in permanently
stricture-free, absorbing health state and no longer proceeds through further cycles.
lation in this state could be recurrent stricture, temporarily
stricture-free and permanently stricture-free, which would cycle a hypothetical individual patient to the relevant health
state. In contrast, patient(s) in the temporarily stricture-free
state may only have recurrent stricture and temporarily stricture-free as possible outcomes. Patients who become permanently stricture-free after treatment would remain so for the
rest of the simulation. Cycle length determines when transitions into and out of the various states are allowed and it
reflects the specific clinical scenario. For example, the cycle
may be relatively short, eg months, for a model examining
testis cancer, while the cycle would likely be longer for prostate
cancer, eg years.
Similar to DA, MMs can be evaluated by calculating EV.
A common end point in MMs is maximizing life expectancy.
Life expectancy can be calculated through matrix algebra,
cohort simulation and Monte Carlo simulation. For a cohort
simulation the simplest approach (transition probabilities)
is used at each cycle to distribute and redistribute a hypothetical cohort of patients among the health states as a
group, usually until all members of the cohort have died.
Life expectancy is the average amount of time that all patients are alive. In a Monte Carlo simulation each hypothetical patient is individually simulated and transitioned
among the health states. The advantages of this method are
the ability to 1) estimate the variability of the outcome
measure, 2) determine events occurring for each patient and
3) repeat the process for a large number of patients to reduce
simulation variability. Finally, like standard DA models,
sensitivity analysis should be applied to any MM.
EVALUATING OUTCOMES
An important aspect of DA is choosing which clinical and/or
economic outcomes to study. These outcomes can be classified into the categories of QOL or utility, other clinical end
points and costs. Outcome assessments combining costs and
QOL considerations can be made through cost-effectiveness
analysis or cost-utility analysis.
QOL has become a crucial component of many DAs because most choices result in incremental trade-offs in health
status, rather than simply life or death. The preference that
society or individuals have for a given health outcome is
referred to as utility.4 When used as a direct end point in
DA, a utility of 1.0 typically represents perfect health, while
0 represents the worst health state, eg death. QALYs combine life expectancy with expected QOL into a single parameter.5 The acceptance of QALY as a measure of utility high-
lights the willingness of individuals to assume the risk of
poor outcomes to attain certain health states. Other integrated metrics used for utility are years of healthy life and
healthy-year equivalents.6
Assessment of individual and group preferences for
health states is a key issue for DA models. Common techniques are the standard gamble7 and time trade-off.8 With
the former approach raters (the population used to derive
utility) must choose between a certain but imperfect health
outcome, eg persistent prostate cancer, vs an uncertain optimal health outcome, eg healthy and cancer-free after prostatectomy, of which the alternative outcome is death. The
health state being evaluated is the one with an uncertain
outcome (cancer-free after surgery). Raters are asked at
what probabilities of the optimal health outcome the rater
would still be willing to choose the uncertain path. The point
of indifference between choices is used to calculate utility for
the tested health state.
In the time trade-off technique raters are asked how many
years of life they would give up to be in a healthier state rather
than a less healthy one and utility values are derived based on
the number of years of life sacrificed. Generic measures of
health related QOL include the EuroQol,9 Quality of WellBeing Scale10 and SF-36™.11 These instruments, sampled
from the general population, have been adapted for use in
utility analysis. Alternatively utility values for various health
states can be estimated by expert opinion, which is a more
tenuous approach to preference assessment.
Although utility metrics such as QALY can be informative in DA, they can be difficult to obtain for certain health
states or unnecessary for specific clinical problems. Using a
specific clinical end point may be more suitable in these
situations. For example, a DA for infertile couples seeking
pregnancy may be better served using pregnancy rates as
the outcome rather than QALY.12 In this form of DA a
particular clinical outcome is the metric for effectiveness. A
strength of DA is the option to choose the outcomes for
investigation.
ECONOMIC ANALYSES
Many DAs seek to answer questions by exclusively examining medical outcomes. Given the pressures in the contemporary health care system, economic issues cannot be ignored
and they have an increasing role in medical decision making. Economic DA uses the same quantitative methods to
compare alternative strategies with respect to resource use
and expected outcomes, although economic evaluations
DECISION ANALYSIS AND MARKOV MODELING IN UROLOGY
merely seek to inform resource allocation decisions, not
make them. Guidelines for reading economic evaluations
have been previously outlined.13
When evaluating DA based on costs, it is important to
consider various perspectives, such as patient, health care
organization and society. There are 4 main forms of economic evaluation (Appendix 3). In addition, cost of illness
analyses, in which total societal costs of caring for individuals with an illness are calculated vs individuals without the
illness, can be considered another form of economic evaluation. Given that most interventions vary in cost as well as in
efficacy, cost minimization analyses, which assume that
benefits of interventions are equivalent, are often of limited
use. Cost-benefit studies, which assign monetary values to
lives and morbidity, are pragmatically and ethically difficult
to perform.
In contrast, cost-effectiveness and cost-utility studies
have become prevalent in DA because of the recognition that
society has limited resources and various management approaches for illness have trade-offs in costs and efficacy. In
these studies the outcome metrics are cost-effectiveness and
cost-utility ratios, which use cost as the numerator and
effectiveness or utility as the denominator. Hence, it is incorrect to declare that 1 alternative is more cost-effective
than another choice based on cost considerations alone.
Rather, an alternative is considered more cost-effective than
another choice if it is 1) less costly and at least as effective,
2) more effective and more costly with additional effectiveness worth the added cost or 3) less effective and less costly
with the additional effectiveness of the more costly alternative not worth the added cost.
DA IN UROLOGY
Our search of urological studies using DA or MMs yielded
a total of 107 publications in English (see table). Search
terms included combinations of decision analysis and
Markov with all urological diseases and specialties, adult
and pediatric as well as the genitourinary organs (kidney,
prostate, bladder, testis, urethra, penis, ureter and adrenal). A detailed discussion of all publications is beyond the
scope of this review.
One of the earliest decision analyses compared immediate transurethral resection of the prostate with watchful
waiting in men with lower urinary tract symptoms.14 The
model used data from the literature and Medicare claims as
well as patient interviews to assess preferences. In the base
case analysis immediate surgery led to the loss of 1 month of
life expectancy. However, when QOL was considered, immediate surgery was favored and it yielded a benefit of 2.94
quality adjusted life-months. Sensitivity analysis demonstrated that the model was sensitive to the degree of disutility attributed to the urinary symptoms. This example illustrates the importance of incorporating QOL considerations
into decision models as well as the dependence of these
models on the accurate assessment of individual patient
preferences. As highlighted by others,15 most DA models
inherently describe cohort level outcomes and they are better suited for informing policy makers rather than individual patients. Typically for a single patient the outcome is all
or nothing, such as whether a patient is alive or dead from
disease, but DA models are not designed to predict individual patient outcomes. However, we contend that appropri-
1871
ately planned models and sensitivity analyses can account
for distinct clinical contexts and assist clinicians and patients in decision making.
A semi-MM was used to examine various treatments for
men with clinically localized prostate cancer.16 This study
was controversial because it concluded that “in most cases
the potential benefits of therapy [radical prostatectomy, external beam radiation therapy] are small enough that the
choice of therapy is sensitive to the patient’s preferences for
various outcomes and discounting” and “watchful waiting is
a reasonable alternative to invasive treatment for many
men with localized prostatic carcinoma.”16 Reexamination of
the study showed that the model was structurally valid but
it was driven by the annual rate of metastatic progression.17
Indeed, using potentially more accurate estimates of metastatic rates demonstrated a benefit of radical prostatectomy.
For example, even with well differentiated tumors the benefit of radical prostatectomy improved from ⫺0.34 to 1.81
years. Other criticisms of the initial analysis include the
determination of utilities by a consensus panel using unclear methodology and the use of out-of-date surgical outcome data. This example shows the ability of DA to address
a crucial clinical question that remains unanswered while
awaiting outcomes from ongoing clinical trials and the critical dependence on the data entered into the model. In
addition, the resulting discussion highlighted areas requiring further study as well as which clinical factors may be
most important in the decision making process.
COMPARISON WITH OTHER TOOLS
In urology other well developed, clinically useful computational tools are available, including predictive nomograms
and ANNs. Each is based on multivariable logistic regression, is developed based on data sets with a known outcome
and helps predict the outcome for a specific situation or
patient. The nomogram is merely the graphic representation of the calculating device, while in ANN a computer
system simulates human synaptic physiology and attempts to recognize patterns and learn from experience.
Each method can be validated by testing various patient
populations but they are limited by the retrospective data
used to develop the models. A touted advantage of ANN is
the ability to resolve complex nonlinear relationships
among variables that may be unknown. However, the
black box nature of the hidden layers obscure how the
results are derived. Decision models may be more difficult
to apply for making individual decisions and accuracy is
ultimately difficult to confirm but the transparent methodology permits critique and modification, and sensitivity
analysis extends the potential applicability.
CONTROVERSIES IN DA AND MMs
Although the use of DA models in medical decision making can
be informative and it is increasingly common, limitations of the
technique must be recognized and the practicing clinician must
be aware of common caveats. All decision models necessitate
oversimplification of complex biological processes. Thus, one
must be aware of explicit and unstated assumptions and omissions inherent within the model. Despite the reduction of complex clinical questions into decision models the structure may
still be valid, provided that it accounts for all desired treatment
1872
DECISION ANALYSIS AND MARKOV MODELING IN UROLOGY
Published urological decision analyses
Topic
Oncology:
Prostate Ca screening
Total No. Studies
(% using MM)
11 (36)
Prostate Ca prevention
3 (33)
Prostate Ca staging
5 (20)
Prostate Ca treatment
22 (59)
Bladder Ca
11 (27)
Testis Ca
6 (33)
Renal Ca
Adrenal indicentaloma
2 (0)
3 (0)
Benign prostatic disease
management
11 (45)
Benign bladder disease
5 (20)
Erectile dysfunction
Male infertility
1 (0)
4 (25)
Urethral stricture
Urinary stone disease
2 (0)
5 (20)
Donor nephrectomy
Orchiopexy
Ureteropelvic junction
obstruction
Vesicoureteral reflux
1 (0)
1 (0)
2 (0)
7 (14)
Circumcision
5 (60)
Totals
References
Chang and Friedland2; Mold et al, J Fam Pract, 34: 561, 1992; Optenberg and Thompson, Urol
Clin North Am, 17: 719, 1990; Krahn et al, JAMA, 272: 773, 1994; Cantor et al, J Fam Pract,
41: 33, 1995; Gottlieb et al, Invest Radiol, 31: 84, 1996; Perez-Niddam et al, Crit Rev Oncol
Hematol, 32: 167, 1999; Ross et al, JAMA, 284: 1399, 2000; Ellison et al, Urology, 60: 42,
2002; Ross et al, Urology, 66: 1038, 2005; Stone et al, Aust N Z J Public Health, 29: 349, 2005
Svatek et al, Cancer Epidemiol Biomarkers Prev, 15: 1485, 2006; Lotan et al, J Clin Oncol, 23:
1911, 2005; Zeliadt et al, Am J Med, 118: 850, 2005
Wolf et al, Urology, 42: 680, 1993; Wolf et al, J Urol, 153: 993, 1995; Meng et al, J Urol, 164:
1235, 2000; Jager et al, Radiology, 215: 445, 2000; Calvert et al, Br J Cancer, 88: 31, 2003
Fleming et al16; Elkin et al18; Cowen et al, J Clin Epidemiol, 47: 3, 1994; Goodnough et al,
Urology, 44: 226, 1994; Etchason et al, N Engl J Med, 332: 719, 1995; Kattan et al, J Gen
Intern Med, 12: 299, 1997; Yoshimura et al, Jpn J Cancer Res, 89: 681, 1998; Cowen et al,
Med Decis Making, 18: 376, 1998; Seidenfeld et al, Evid Rep Technol Assess (Summ), i, 1999;
Grossfeld et al, J Urol, 164: 93, 2000; Knight et al, Clin Prostate Cancer, 1: 105, 2002; Alibhai
et al, J Clin Oncol, 21: 3318, 2003; Penson et al, J Urol, 174: 547, 2005; Konski et al, Int J
Radiat Oncol Biol Phys, 66: 408, 2006; Konski et al, Cancer, 106: 51, 2006; Bayoumi et al,
J Natl Cancer Inst, 92: 1731, 2000; Ramsey et al, Urology, 66: 835, 2005; Konski, Int J Radiat
Oncol Biol Phys, 60: 1373, 2004; Bhatnagar et al, Urology, 63: 103, 2004; Krahn et al,
Urology, 60: 258, 2002; Saranchuk et al, J Clin Oncol, 23: 4146, 2005; Konski et al, Int J
Radiat Oncol Biol Phys, 63: 788, 2005; Hillner et al, Urology, 45: 633, 1995
Ellwein and Farrow, Med Decis Making, 8: 110, 1988; Corwin and Silverstein, J Urol, 139:
1002, 1988; Kent et al, Med Decis Making, 9: 26, 1989; Kiemeney et al, Br J Urol, 73: 164,
1994; Nam et al, J Urol, 163: 752, 2000; Lachaine et al, Can J Urol, 7: 974, 2000; Lotan et al,
J Urol, 167: 75, 2002; Svatek et al, Urol Oncol, 24: 338, 2006; Lotan et al, Cancer, 107: 982,
2006; Kent et al, Med Decis Making, 11: 240, 1991
Langenstroer et al, J Urol, 168: 1396, 2002; Link et al, J Clin Oncol, 23: 5762, 2005; Spermon
et al, Eur Urol, 48: 258, 2005; Debono et al, J Clin Oncol, 15: 1455, 1997; Munro and Warde,
Med Decis Making, 11: 131, 1991; Ogan et al, J Urol, 168: 1945, 2002
Link et al, J Endourol, 20: 782, 2006; Lotan et al, J Urol, 170: 752, 2003
Brunaud et al, Med Sci Monit, 12: CR355, 2006; Kievit and Haak, Endocrinol Metab Clin
North Am, 29: 69, 2000; Dwamena et al, J Nucl Med, 39: 707, 1998
Barry et al14; Krumins et al, Med Decis Making, 8: 1, 1988; Lowe et al, Urology, 46: 477, 1995;
Baladi et al, Pharmacoeconomics, 9: 443, 1996; Cher and Lenert, J Am Med Inform Assoc, 4:
301, 1997; Ackerman et al, Urology, 56: 972, 2000; Blute et al, Urology, 56: 981, 2000;
Ohsfeldt et al, J Manag Care Pharm, 10: 412, 2004; McDonald et al, Can J Urol, 11: 2327,
2004; Manyak et al, J Endourol, 16: 51, 2002; DiSantostefano et al, Pharmacoeconomics, 24:
171, 2006
Weber and Walters, Obstet Gynecol, 96: 867, 2000; Noe et al, J Manag Care Pharm, 8: 343,
2002; Weber et al, BJU Int, 89: 356, 2002; Oremus et al, Can J Urol, 10: 1934, 2003; Cody
et al, Health Technol Assess, 7: iii, 2003
Tan, Pharmacoeconomics, 17: 77, 2000
Hsieh et al12; Schlegel, Urology, 49: 83, 1997; Pavlovich and Schlegel, Fertil Steril, 67: 133,
1997; Meng et al, J Urol, 174: 1926, 2005
Rourke and Jordan, J Urol, 173: 1206, 2005; Wright et al, Urology, 67: 889, 2006
Lotan et al, J Urol, 172: 2275, 2004; Lotan et al, J Urol, 167: 1621, 2002; May and Chandhoke,
J Urol, 159: 24, 1998; Wolf et al, J Endourol, 9: 243, 1995; Lotan et al, Urol Res, 33: 223,
2005
Pace et al, J Endourol, 16: 495, 2002
Lorenzo et al, J Urol, 172: 712, 2004
Gettman et al, J Urol, 169: 228, 2003; Hsieh et al, unpublished data
Benoit et al, J Urol, 168: 2173, 2002; Mathews et al, J Urol, 163: 561, 2000; Kobelt et al,
J Urol, 169: 1480, 2003; Thompson et al, Pediatrics, 115: 426, 2005; Benoit et al, J Urol, 176:
2649, 2006; Hsieh et al, J Urol, 177: 703, 2007; Benoit et al, J Urol, 176: 1588, 2006
Lawler et al, Fam Med, 23: 587, 1991; Chessare, Clin Pediatr (Phila), 31: 100, 1992; Ganiats
et al, Med Decis Making, 11: 282, 1991; Van Howe, Med Decis Making, 24: 584, 2004; Schoen
et al, J Urol, 175: 1111, 2006
107 (34)
options and their various potential outcomes. Limited understanding of underlying social forces and mechanisms of disease
can lead to erroneous model construction.
The quality of the information imputed into a model is
also vital. As illustrated by the discussion surrounding the
study by Fleming et al study,16 the most up-to-date and
relevant data must be applied. This is particularly pertinent
when considering novel, rapidly evolving technologies. A
knowledge of the relevant contemporary literature assists in
determining whether the applied probabilities and costs are
reasonable or whether the poor quality of the data results in
the garbage in, garbage out phenomenon. Many investigators have proposed hierarchical systems to grade the level of
evidence in the medical literature and the data imputed into
decision analyses should be assessed according to these criteria. Some of these limitations and/or the lack of available
studies can be addressed by sensitivity analysis, as discussed. However, in models sensitive to a critical parameter
that specific element should be scrutinized carefully. Furthermore, readers should be cautious of DA models in which
relatively small changes in key probabilities result in a
reversal of the treatment preference. It is possible that the
limitations of even a valid model using the best available
information can amplify small errors and lead to a potentially false conclusion.
Defining and measuring outcome values is particularly
challenging. Although health states such as alive and dead
are easy to evaluate, determining utility values for other
DECISION ANALYSIS AND MARKOV MODELING IN UROLOGY
health states can be difficult. There is growing interest in and
study of QOL measures in all urological disciplines and they
may ultimately be able to be incorporated into decision models.
The method of assessing preference may alter the result of a
DA, as reported by Elkin et al, who examined treatment recommendations for prostate cancer.18 Similarly different populations, eg age, gender and ethnicity, may have divergent utility values for the same health outcome states.
Issues of perspective are pertinent to all DA models. For
instance, while a DA examining the global financial impact
of a screening test for bladder cancer can be informative to
American taxpayers, an indigent patient with hematuria
may care only about out-of-pocket costs and whether the test
will identify cancer earlier. These concerns relate to group vs
individual utility and are particularly germane to cost-effectiveness analysis. Furthermore, outcomes are usually expressed as differences between mean outcomes for a group of
patients. Thus, if a treatment causes a dramatic change in
outcome for only a small subset of the group, the difference
may appear insignificant and it may be overlooked. This
should be considered in analyses examining screening tests
and novel cancer therapies, and it may point to specific use
of the test or treatment in a population enriched for effect.
Similarly base case analyses of DA models do not provide
CIs, unlike relative risk and confidence bar meta-analyses or
concordance index and area under the curve for nomograms.
However, Monte Carlo simulation of MMs can provide CIs.
Careful examination of sensitivity analyses can indicate
whether the EV occurs within a realistic or unrealistic range
of specified variable(s) and, thus, indirectly provide information about the confidence in the outcome.
1873
Unfortunately to our knowledge formal methods to evaluate DA models do not exist. Beyond the confirmation of
construct and content validity there are no established
means of proving the quality and reliability of decision analyses, which limits the credibility of the method. In contrast,
conclusions can be tested for other quantitative methods.
Cost-effective analysis has been aided by the formalization
of evaluation criteria and results can be compared with
actual medico-economic data. Nevertheless, few reports adhere to all fundamental study principles.
The ultimate question is how we can incorporate DA/MMs
into clinical practice. We are currently adapting a published
MM of vasectomy associated male infertility12 for real-time
World Wide Web use by patients. Specifically patients will be
able to input clinical and financial parameters specific to their
circumstances, namely the obstructive interval, female partner
age and willingness to pay, and receive immediate feedback
regarding the most cost-effective approach to infertility treatments. We hope that the interactive website format will serve
as a model for patient specific DA.
The ability of DA to examine population level outcomes is
conducive to providing data for health policy recommendations. Cost-effectiveness analysis is especially useful for making choices regarding resource allocation. Pay for performance
and a single payer national health insurance program are
potential, upcoming major changes in American medicine.
These trends toward health care consolidation would further
emphasize the importance of techniques such as DA, which can
help optimize medical outcomes and expenditures. It is vital
that urologists should participate in these forms of analysis to
maintain leadership in medicine.19
APPENDIX 1
Fundamental Steps in Designing a Decision Analysis
Step
1. Identification of the clinical problem to be addressed
2. Building a decision tree by disaggregating the problem into choices and
consequences
3. Populating the decision tree with data
4. Comparison of treatment choices by calculating payoffs
5. Testing the validity and robustness of the model using sensitivity analysis
Example
What are the costs and outcomes of RAP compared to OP?
Brainstorming by investigators to define all relevant possible outcomes
of RAP vs OP (the decision node), followed by creation of computer
based model
MEDLINE searches of costs and probabilities of specific outcomes of
RAP and OP are performed, relevant costs and probability values are
placed into model
Overall costs and outcomes for the RAP and OP treatment arms are
calculated (also known as base-case analysis)
Values for costs and outcome probabilities are varied to determine
their impact on the model
APPENDIX 2
Glossary of Basic Terms in Decision and Markov Analyses
Base case analysis—the findings of a model using the initial (baseline) values assigned to the probabilities and outcomes. Typically performed prior to
sensitivity analysis.
Chance node—branch point in the decision tree where multiple outcomes are possible that are not under the control of the decision maker.
Decision node—branch point in the decision tree where alternative actions are under the control of the decision maker.
Decision tree—graphic representation of the clinical question with treatment choices and their possible consequences.
Expected value—for a treatment choice, the weighted average of the values of the possible outcomes; a higher expected value, depending on the payoff,
typically denotes a more favorable outcome.
Markov model—form of decision analysis incorporating changes in states over time; applicable to clinical situations where events can occur more than
once over time, the timing of events is relevant, and health states are mutually exclusive.
Outcome states—the consequences of the final events in the decision tree, which typically are associated with specific payoffs.
Payoff—the numerical reward associated with achieving an outcome state. Assigned by the model designer, can be costs, profits, quality adjusted life
years, etc.
Sensitivity analysis—the process of altering value(s) within the model, keeping other values constant, and assessing the impact on the conclusion; all
elements can be subjected to sensitivity analysis, such as probability and utility values.
Threshold value—the value of the variable being tested in 1-way sensitivity analysis where the treatment alternatives have equal outcomes; also known
as the “break even” point.
Utility value—the assigned desirability or preference of a given outcome state; typically 1.0 represents perfect health while 0 represents the worst health
state (eg death).
1874
DECISION ANALYSIS AND MARKOV MODELING IN UROLOGY
APPENDIX 3
Forms of Economic Evaluation for Decision Analysis
Study Type
Cost minimization analysis
Cost effectiveness analysis
Cost-utility analysis
Cost-benefit analysis
Measurement of Benefits
Question Posed
Benefits found to be equivalent
Which is the most efficient way of achieving a given goal (or objective)?
Natural units (eg life-years gained)
What is the most efficient way of spending a given budget?
Healthy years (eg QALY, healthy years equivalents) What is the comparative cost per unit of outcome?
Monetary terms
What is the economic trade-off between interventions when all of the
costs and benefits of the intervention and its outcome are measured
in monetary terms?
Adapted from Drummond and Jefferson,13 and Petitti.19
9.
Abbreviations and Acronyms
ANN
DA
EV
MM
OP
QALY
QOL
RAP
⫽
⫽
⫽
⫽
⫽
⫽
⫽
⫽
artificial neural networks
decision analysis
expected value
Markov model
open prostatectomy
quality-adjusted life years
quality of life
robotic assisted prostatectomy
REFERENCES
Henschke UK and Flehinger BJ: Decision theory in cancer
therapy. Cancer 1967; 20: 1819.
2. Chang P and Friedland GW: The role of imaging in screening
for prostate cancer. A decision analysis perspective. Invest
Radiol 1990; 25: 591.
3. Mold JW, Holtgrave DR, Bisonni RS, Marley DS, Wright RA
and Spann SJ: The evaluation and treatment of men with
asymptomatic prostate nodules in primary care: a decision
analysis. J Fam Pract 1992; 34: 561.
4. Torrance GW: Utility approach to measuring health-related
quality of life. J Chronic Dis 1987; 40: 593.
5. La Puma J and Lawlor EF: Quality-adjusted life-years. Ethical
implications for physicians and policymakers. JAMA 1990;
263: 2917.
6. Gold MR, Siegel JE, Russell LB and Weinstein MC: CostEffectiveness in Health and Medicine. New York: Oxford
University Press 1996.
7. Llewellyn-Thomas H, Sutherland HJ, Tibshirani R, Ciampi A,
Till JE and Boyd NF: Describing health states. Methodologic issues in obtaining values for health states. Med Care
1984; 22: 543.
8. Torrance GW, Thomas WH and Sackett DL: A utility maximization model for evaluation of health care programs. Health
Serv Res 1972; 7: 118.
10.
11.
12.
13.
1.
14.
15.
16.
17.
18.
19.
EuroQol—a new facility for the measurement of health-related
quality of life. The EuroQol Group. Health Policy 1990; 16:
199.
Kaplan RM and Anderson JP: A general health policy model:
update and applications. Health Serv Res 1988; 23: 203.
Ware JE Jr and Sherbourne CD: The MOS 36-item short-form
health survey (SF-36). I. Conceptual framework and item
selection. Med Care 1992; 30: 473.
Hsieh MH, Meng MV and Turek PJ: Markov modeling of
vasectomy reversal and ART for infertility: how do obstructive interval and female partner age influence cost effectiveness? Fertil Steril 2007, Epub ahead of print.
Drummond MF and Jefferson TO: Guidelines for authors and
peer reviewers of economic submissions to the BMJ. The
BMJ Economic Evaluation Working Party. BMJ 1996; 313:
275.
Barry MJ, Mulley AG Jr, Fowler FJ and Wennberg JW: Watchful waiting vs immediate transurethral resection for symptomatic prostatism. The importance of patients’ preferences. JAMA 1988; 259: 3010.
Elkin EB, Vickers AJ and Kattan MW: Primer: using decision
analysis to improve clinical decision making in urology. Nat
Clin Pract Urol 2006; 3: 439.
Fleming C, Wasson JH, Albertsen PC, Barry MJ and Wennberg JE: A decision analysis of alternative treatment strategies for clinically localized prostate cancer. Prostate Patient Outcomes Research Team. JAMA 1993; 269: 2650.
Beck JR, Kattan MW and Miles BJ: A critique of the decision
analysis for clinically localized prostate cancer. J Urol 1994;
152: 1894.
Elkin EB, Cowen ME, Cahill D, Steffel M and Kattan MW:
Preference assessment method affects decision-analytic
recommendations: a prostate cancer treatment example.
Med Decis Making 2004; 24: 504.
Petitti DB: Meta-Analysis, Decision Analysis and Cost-Effectiveness Analysis, 2nd ed. New York: Oxford University
Press 2000; p 306.