Dynamic microsimulation to model multiple outcomes in cohorts of
critically ill patients
Gilles Clermont*, MD, MSc
Vladimir Kaplan*, MD
Rui Moreno†, MD
Jean-Louis Vincent‡, MD, PhD
Walter T. Linde-Zwirble§
Ben Van Hout¢, PhD
Derek C. Angus*¶, MD, MPH
* Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA
† Intensive Care Medicine, Hospital de St. Antonio dos Capuchos, Lisbon, Portugal
‡ Department of Intensive Care, Erasmus University Hospital, Brussels, Belgium
§ Health Process Management, Inc, Doylestown, PA
¢ Department of Health Care Policy and Management, Erasmus University, Rotterdam,
Netherlands
¶ Center for Research on Health Care and Graduate School of Public Health, University of
Pittsburgh, Pittsburgh, PA
Running head
Microsimulation of organ dysfunction
Word count
3,239
Financial support
Partially supported by Eli Lilly & Company (Gilles Clermont and Derek C. Angus) and by the
Stiefel-Zangger Foundation, University of Zurich, Switzerland (Vladimir Kaplan)
Address for correspondence
Gilles Clermont, MD, MSc
Room 606B, Scaife Hall
Critical Care Medicine
University of Pittsburgh
3550 Terrace Street
Pittsburgh, PA 15261
Tel: (412) 647 7980
Fax: (412) 647 3791
E-mail: clermontg@ccm.upmc.edu
Microsimulation of organ dysfunction
Abstract
Background: Existing intensive care unit (ICU) prediction tools forecast single
outcomes, (e.g., risk of death) and do not provide information on timing.
Objective: To build a model that predicts the temporal patterns of multiple outcomes,
such as survival, organ dysfunction, and ICU length of stay, from the profile of organ
dysfunction observed on admission.
Design: Dynamic microsimulation of a cohort of ICU patients.
Setting: 49 ICUs in 11 countries.
Patients: 1,449 patients admitted to the ICU in May 1995.
Interventions: None.
Model Construction: We developed the model on all patients (n=989) from 37
randomly-selected ICUs using daily Sequential Organ Function Assessment (SOFA) scores. We
validated the model on all patients (n=460) from the remaining 12 ICUs, comparing predictedto-actual ICU mortality, SOFA scores, and ICU length of stay (LOS).
Main Results: In the validation cohort, the predicted and actual mortality were 20.1%
(95%CI: 16.2%-24.0%) and 19.9% at 30 days. The predicted and actual mean ICU LOS were 7.7
(7.0-8.3) and 8.1 (7.4-8.8) days, leading to a 5.5% underestimation of total ICU bed-days. The
predicted and actual cumulative SOFA scores per patient were 45.2 (39.8-50.6) and 48.2 (41.654.8). Predicted and actual mean daily SOFA scores were close (5.1 vs 5.5, P=0.32). Several
organ-organ interactions were significant. Cardiovascular dysfunction was most, and
neurological dysfunction was least, linked to scores in other organ systems.
Conclusions: Dynamic microsimulation can predict the time course of multiple shortterm outcomes in cohorts of critical illness from the profile of organ dysfunction observed on
admission. Such a technique may prove practical as a prediction tool that evaluates ICU
performance on additional dimensions besides the risk of death.
2
Microsimulation of organ dysfunction
Descriptor
Severity-of-disease scoring systems
Keywords
Intensive care, multiple organ failure, mortality, resource use, computer simulation,
microsimulation
3
Microsimulation of organ dysfunction
Progress in acute care medicine and new resuscitation techniques have led to significant
improvement in the immediate survival of victims of severe trauma, burns, and infections.
However, early resuscitation is often followed by progressive dysfunction of multiple organ
systems (MODS) [1, 2] that may lead to prolonged morbidity and death [3, 4]. Indeed, MODS
accounts for most late-onset deaths in critical illness [5] and consumes large amounts of
healthcare resources [6].
Several investigators have developed schemes to quantify organ dysfunction and have
consistently demonstrated a close relationship between the presence and intensity of MODS
and hospital mortality [7-9]. Specifically, the cumulative burden of organ failure in terms of
both, the number of organs failing [8, 10] and the degree of organ dysfunction within each
organ system [11] was the strongest predictor of death [12]. However, these prediction tools
typically predict single outcomes (e.g., risk of death) at fixed time points. Other statistical
techniques provide prediction the timing of events [13, 14], but only simulations provide
simultaneous predictions for the incidence and timing of multiple outcomes.
Our first objective was to build a single model that predicts, in a cohort of critically ill
patients, the temporal patterns of multiple outcomes, such as survival, organ dysfunction, and
intensive care unit (ICU) length of stay, from demographic variables and the profile of organ
dysfunction assessed on admission by the sequential organ dysfunction score (SOFA) [15]. Our
second objective was to use the model to explore organ-organ interactions. Because such
predictions cannot typically be constructed using standard analytic methods[16], we developed
a microsimulation model, a technique suited to predict multiple events over time in systems
where characteristics change in a time dependent fashion. We validated its predictive
performance in a separate group of patients using basic demographic data and the first ICU day
SOFA score only.
4
Microsimulation of organ dysfunction
Such simulations have a range of potential applications in the ICU such as predicting
the rate and timing of various events and outcomes, and the potential impact of interventions
aimed at modifying the predictors of these outcomes.
5
Microsimulation of organ dysfunction
Materials and methods
Patient population
We used an international database of 1449 critically ill patients from 40 institutions (49
ICUs) in 11 countries. The database included all adult patients admitted to the ICUs in May
1995, except for those who stayed in the ICU for less than 48 hours after uncomplicated surgery.
The data were prospectively collected by the European Society of Intensive Care Medicine
(ESCIM) to evaluate and validate the usefulness of the SOFA score [8, 17]. SOFA scores were
collected daily until ICU discharge or a maximum of 33 days. Details regarding data collection
were described previously [8].
Missing values
Scores missing on days prior to the first recorded value were attributed the first
available score. Scores missing between the last recorded score and ICU discharge were
attributed the last recorded score. Other missing scores were assigned according to the
following rules: linear interpolation was used for organ systems with a slowly evolving
physiology (hematologic, renal, and hepatic) and last available scores were carried forward for
the other organ systems (cardiovascular, pulmonary, and neurologic). We chose not to
implement a priori stochastic rules[18] of imputing missing data a because of the expected nonrandomness and high predictability of missing values [19]. To assess the sensitivity of
predictions to different imputation rules for missing data, we provide predictions using last
value carried forward and next value carried backwards as alternatives imputation rules for
missing intercurrent values.
Dynamic microsimulation
Dynamic microsimulation is a method particularly suitable for modeling multiple
6
Microsimulation of organ dysfunction
events over time that occur in systems where the interactions within the system are complex,
the characteristics of the systems change in a time-dependent fashion, and the analysis of the
system is intractable by conventional analytic methods [16]. Such models allow probabilistic
projections forward in time on cohorts with known baseline characteristics. In our model, the
patient represents the complex system, defined by his/her profile of organ dysfunction, which
evolves over time and dictates the occurrence of multiple outcomes (i.e., death, organ failure, or
ICU discharge). Microsimulation is well-suited to describe cohort behavior, and not the time
course of individual patients.
Model development
We used a subset of 989 patients from 37 randomly selected ICUs to develop the
microsimulation model. We built the model in three steps. First, using the SOFA scores of the
current day, we constructed trinomial logistic regression equations to generate daily
probabilities of “discharge from the ICU on the current day”, “ICU death on the current day”,
or “remain in the ICU until the next day” (the ternary outcome sub-model). Because we
anticipated that the predictive probabilities of a given pattern of SOFA scores would change
over time, we also included an explicit time factor as independent predictor (periods A, B and C
corresponding to ICU day 1, ICU days 2 to 9, and ICU days 10 to 30). We ignored data beyond
30 days because of a paucity of data points. We also included sex, type of patient
(emergent/scheduled surgery, trauma, medical/cardiac/others), age (<45, 45-64, >65) as
predictors. Second, to update SOFA scores in individual patients remaining in the ICU, we
constructed multinomial logistic regression equations to generate SOFA scores for the next day
based on SOFA scores of the current day as well as the same demographic predictors described
above and ICU day (the SOFA sub-models). We developed 6 SOFA sub-models (one for each
organ systems). Third, we integrated all sub-models in a global model, a dynamic
7
Microsimulation of organ dysfunction
microsimulation, and propelled each patient in daily steps until ICU discharge or a maximum
of 30 days.
To verify the ability of the microsimulation model to reproduce the outcome and the
level of organ dysfunction, we simulated the time course of the ICU stay in the development
cohorts (Figure 1). A specific example on how the microsimulation decides of a patient’s
outcome given a set of independent predictors on day 1 is provided as supplementary material
(Tables E1-E4). If the patient remained in the ICU to the next day, the simulation generated
SOFA scores for day 2 using the appropriate SOFA sub-models (Tables E5-E22). The
probability of being discharged alive from the ICU on day 2, remaining in the ICU to the next
day, and dying in the ICU on day 2 was recalculated and the fate of the patient determined
(Figure E1). This process was iterated until the patient was ICU discharge or ICU day 30.
To assess model accuracy we calculated mean ICU mortality, mean ICU length of stay,
average daily organ-specific and global (sum of organ-specific scores on any given day) SOFA
scores, and cumulative (over the entire ICU stay) organ-specific and global SOFA scores for the
entire simulated cohort. The probability distribution of the predictions was derived from
running the microsimulation 500 times. We generated standardized ratios (SR) and 95%
confidence intervals (CI) for all evaluated outcomes and organ dysfunction scores using
prediction from the model as the numerator and actual observations in the development cohort
as the denominator.
Model validation
We validated the model in 460 patients from the remaining 12 ICUs using the day 1
predictors to initiate the microsimulation. We used a random sample of ICUs to increase the
external validity of the model. Indeed, prediction models are typically applied in situations
where both patients and therapeutic approaches to those patients vary from the development
8
Microsimulation of organ dysfunction
environment. Again, we simulated the ICU time course of 460 patients selected with
replacement from the validation cohort and predicted the mean values for ICU mortality, ICU
length of stay, and daily and cumulative organ-specific and global SOFA scores. We compared
the mean values for each outcome predicted by the model to those observed in the validation
cohort and calculated SR and 95% CI.
Organ-organ interaction
To investigate organ-organ interaction we constructed standard linear regression
equations for SOFA scores for the entire cohort irrespective of time period and examined the
magnitude and significance of the regression coefficients for predicting single organ SOFA
scores of the next day based on SOFA scores of the current day. The strength of interaction is
conveyed by the magnitude of the regression coefficients.
Statistical procedures
We compared proportions using Chi-square statistics. We compared lengths of stay and
organ failure scores using the Mann-Whitney U test. We assumed a significance level of p<0.05
for all comparisons. We used the backwards stepwise procedure to select significant predictors
for the multinomial sub-models (p<.01). We built the microsimulation model with the @RISK©
4.5.2 software (Palisade Corporation, Newfield, NY, www.palisade.com).
9
Microsimulation of organ dysfunction
Results
Study population
The characteristics of the development and validation cohorts are provided in Table 1.
The development and validation cohorts had similar distributions regarding age, sex, and
location prior to the ICU admission. There were more emergent surgical and fewer acute
coronary patients in the development cohort. The development cohort had less severe renal
dysfunction over the time course of the ICU stay. Otherwise, there was no difference in the
cumulative global SOFA between the cohorts. ICU mortality was similar in the development
and validation cohorts. Mean ICU length of stay was not significantly different between the
cohorts. Of the entire cohort, 544 (37.5%), 257 (17.7%), 128(8.8%) and 58 (4.0%) patients stayed
at least 7, 14, 21 or more than 30 days in the ICU, respectively.
The proportion of missing values was 11.0% overall, was higher for the hepatic system
(40.3%), and lower for the neurologic (10.9%), cardiovascular (8.5%), pulmonary (3.1%), renal
(1.7%) and coagulation systems (1.6%). There were twice as many missing values in the second
half of the ICU stay compared to the first half (p<0.001) and an equal proportion of missing
values in the development and validation sets (p=0.84).
Model performance
We provide the coefficients of the multinomial equations derived to predict ICU
outcomes (death, discharged alive from the ICU, remained in ICU until next day) and SOFA
scores as supplementary data (Tables E1-E22).
Organ dysfunction
The model performed well in predicting the cumulative SOFA in the development and
validation cohorts for both single organ and global scores (Table 2). In the validation cohort,
10
Microsimulation of organ dysfunction
actual (5.5 [95% CI: 5.2-5.9]) and predicted mean daily SOFA scores (5.1 [4.8-5.5]) were close
(p=0.32). Actual and predicted global SOFA score per patient were 48.2 (41.6-54.8) and 45.2
(39.8-50.6) (P=0.47). The ability of the model to describe the time course of organ dysfunction is
displayed for the validation cohort in Figures 2. The model predicted sequential organ
dysfunction better in some organ systems than others, most noticeably underestimating renal
and hepatic dysfunction late in the ICU course.. Although the model appeared to predict the
general trends correctly, it did not reflect acute changes in mean SOFA scores, as can be seen for
the hematologic system scores (Figure 2). Age and type of admission and ICU day were
predictive of outcome. Sex was only predictive of the evolution of the pulmonary score. Age
was predictive of the evolution of all scores except the hepatic and hematologic scores. Type of
admission was predictive of all scores except for the hematologic system. Finally, ICU day was
a significant predictor of the evolution for scores for all systems except the pulmonary system.
ICU length of stay
In the development cohort, the model predicted that 18 patients, or 1.8% (1.1%-2.6%),
would still be in the ICU at the end of the 30-day study period and thus underestimated the
observed proportion of 3.9% by 21 patients. Consequently, the model predicted 223 (2.9%)
fewer ICU-days than observed (7,576 days) in this cohort of 989 patients. However, the
predicted and observed mean ICU lengths of stay were not significantly different (7.5 vs. 7.7
days, p=0.14). In the validation cohort, the model predicted that 3.1% of the validation cohort
(1.5%-4.7%) would still be in the ICU at the end of the 30-day period, while the observed
proportion of patients was 4.0% (Figure 3). The model underpredicted the observed number of
3,766 ICU-days by 204 days, or 5.5%, in this cohort of 460 patients. The predicted and observed
mean ICU lengths of stay were similar (7.7 vs. 8.2 days, p=0.30). Using different imputation
methods for missing data resulted in predicted ICU lengths of stays ranging from 7.4 to 7.8
11
Microsimulation of organ dysfunction
days.
ICU mortality
The microsimulation predicted overall ICU mortality well in both the development and
validation cohorts (Table 2). The predicted and observed mortalities for the development and
validation cohorts were 21.0% (17.9%-24.9%) and 21.0%, and 20.1% (16.2%-24.0%) and 20.0%,
respectively. The Observed and predicted survival curve for the validation cohort are very
close (Figure 3). Using different imputation methods for missing data resulted in predicted
mortality ranging from 19.8% to 21.7% in the validation cohort.
Organ-organ interaction
Organ-organ interaction is presented in Table 3. Not surprisingly, a given organ SOFA
score was most predictive of the SOFA score of that organ on the next day. However, the
model revealed many organ-organ interactions. SOFA score of several organ systems were
predictive of those of other organ system (e.g., cardiovascular, hematologic, and neurologic
dysfunctions were all significant predictors of the pulmonary SOFA score on the next day,
while hepatic and renal dysfunction were not [Table 3, first row]). As suggested by the
magnitude of the regression coefficients, there is strong two-way cardio-pulmonary interaction.
Cardiovascular dysfunction is associated with dysfunction in all other organ systems,
confirming its central role. Interestingly, the hematologic dysfunction appears to be associated
with subsequent hepatic dysfunction more than the reverse.
12
Microsimulation of organ dysfunction
Discussion
We constructed a dynamic microsimulation model to predict the temporal patterns of
multiple outcomes, such as ICU mortality, organ dysfunction, and ICU length of stay in
critically ill cohorts of patients based on demographic and clinical characteristics on the day of
ICU admission. The model was developed from previously assembled data in a heterogeneous
ICU population from 37 ICUs and validated in patients from 12 other ICUs, where treatment,
processes of care, and ICU discharge policies might have varied widely compared to the
development cohort. The model predicted ICU mortality, average daily and cumulative
amount of organ failure, and ICU length of stay well.
Many authors have used organ failure scores as mortality prediction tools [10, 12, 20,
21]. We have not considered this application because our purpose was not to predict a
particular outcome for an individual patient. Instead, we developed a tool using a single
modeling platform to predict the longitudinal time course of multiple outcomes such as death,
ICU discharge, and amount of organ failure in cohorts of ICU patients with known
characteristics on admission. Rangel-Frausto, et al. recently developed a Markov model of the
natural history and time course of sepsis in the ICU population. These, authors presented an
elegant Markov model of the progression of sepsis, but did not report on organ dysfunction,
nor did they allow transitional probabilities to vary in time during the ICU stay [22]. Because
we wished to use organ dysfunction as the main predictor of outcome, it would have been
difficult to use a standard Markov paradigm[23], given the very large number of states needed
(one for each organ dysfunction combination, ICU discharge, and death), the lack of sufficient
data to generate time-dependent transition probabilities requires to populate such a model.
Although previous reports described the prognostic importance of changes in the levels
of organ dysfunction, the current study is the first to describe direct estimates of organ system
13
Microsimulation of organ dysfunction
dysfunction and interaction in a large cohort of ICU patients, and to predict its course.
Microsimulation is particularly well suited to model systems that cannot be modeled with
standard analytic techniques [16]. Because of the time-dependence of the SOFA scores, the
problem of predicting trajectories dynamical aspects of the problem remains the topic on
ongoing statistical research and is proving to be very difficult. Microsimulation has become the
method of choice to simulate the dynamics of complex systems with a large number of
configurations, the transitions between which vary in time. Applications range from molecular
processes to population dynamics.
The model provided several new insights into organ-organ interaction. The level of
dysfunction in any organ system was the most important predictor of the level of dysfunction
in the same organ on the following day. However, the logistic equations identified several
significant interactions, where the level of dysfunction of a specific organ system on the
following day was also determined by the dysfunction of other organ systems. The model
confirmed the central role of the cardiovascular system in organ-organ interaction and the
particular strength of the cardio-pulmonary interaction. The current dataset is not sufficiently
detailed to allow a thorough exploration of such interactions, but our analysis suggests that
mechanisms can be hypothesized from analysis of observation data otherwise not collected for
that purpose.
There are a number of potential limitations of this study. The size of the development
set did not allow inclusion of potentially important predictors, such as underlying disease or
diagnosis on admission. The SOFA score may not reflect organ dysfunction in a timely and
accurate fashion. We constructed the model on a heterogeneous ICU population, which may
have compromised the predictive ability of the model. Constructing the model on a more
restricted case-mix (e.g., a cohort of patients with sepsis) would be expected to improve
14
Microsimulation of organ dysfunction
predictability in this population. Generalizability to other ICU populations could be limited.
As ICU discharge criteria may vary significantly from institution to institution, we attempted to
address this limitation by clustering the development and validation sets at the ICU level. To
maximize the amount of data available for modeling, we imputed 11% of the organ failure
scores using a scheme that seemed to recapitulate existing physiology and clinical decision
making. Stochastic methods of imputation such as multiple imputation [19], although clearly
more elegant and appropriate in situations where missing data is random, may not be
appropriate in situations where data are missing because they were presumed known by
treating physicians. There is no standard way to assess the fit and validate microsimulation
models when actual data is not available. Fortunately, we could compare to empiric
observation. We therefore used a pragmatic approach to describe the longitudinal time course
of a patient cohort where no standard statistical techniques are available to measure the
closeness of a projected and observed trajectory in time (e.g., describing a mean organ failure
scores over the ICU stay). We may have underestimated the uncertainty associated with the
microsimulation. We presented uncertainty around estimates originating from the stochasticity
of the microsimulation, but did not consider additional uncertainty associated with the
imperfect knowledge of model parameters themselves (such as regression parameters). By
retaining only highly significant parameters in the regression equations we decreased this
uncertainty, but did not eliminate it [16]. This level of uncertainty relates to the
epidemiological concept of precision of prediction. In addition, there exists the possibility of
the presence of a systematic bias embodied in the structure of the model, the prediction
equations, or less likely, because the development population was fundamentally different
from the validation population in a way that we could not evaluate.
15
Microsimulation of organ dysfunction
We conclude that dynamic microsimulation can forecast the temporal pattern of
multiple outcomes such as mortality, ICU discharge, burden of organ failure, and resource use
in heterogeneous cohorts of ICU patients from the profile of organ dysfunction observed on
admission. We suggest that such techniques may prove practical as prediction tools that
evaluate ICU performance on additional dimensions besides the risk of death. Furthermore,
such techniques could also assist in staffing decisions, resource allocation, and the economic
evaluation of ICU specific interventions presumed to impact on organ failure.
16
Microsimulation of organ dysfunction
References
1. Baue AE (1997) Multiple organ failure, multiple organ dysfunction syndrome, and systemic
inflammatory response syndrome. Why no magic bullets? Arch Surg 132: 703-707
2. Sauaia A, Moore FA, Moore EE, Norris JM, Lezotte DC, Hamman RF (1998) Multiple organ failure
can be predicted as early as 12 hours after injury. J Trauma 45: 291-301
3. Baue AE (1975) Multiple, progressive, or sequential systems failure. A syndrome of the 1970s . Arch
Surg 110: 779-781
4. Baue AE (1992) The horror autotoxicus and multiple-organ failure. Arch Surg 127: 1451-1462
5. (1992) American College of Chest Physicians/Society of Critical Care Medicine Consensus
Conference: Definitions for sepsis and organ failure and guidelines for the use of innovative
therapies in sepsis. Crit Care Med 20: 864-874
6. Sznajder M, Aegerter P, Launois R, Merliere Y, Guidet B, CubRea (2001) A cost-effectiveness
analysis of stays in intensive care units. Intensive Care Med 27: 146-153
7. Knaus WA, Draper EA, Wagner DP, Zimmerman JE (1985) Prognosis in acute organ-system
failure. Ann Surg 202: 685-693
8. Vincent JL, Moreno R, Takala J, Willatts S, de Mendonca A, Bruining H , Reinhart CK, Suter PM,
Thijs LG (1996) The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ
dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European
Society of Intensive Care Medicine. Intensive Care Med 22: 707-710
9. Le Gall JR, Klar J, Lemeshow S, Saulnier F, Alberti C, Artigas A, Teres D (1996) The Logistic Organ
Dysfunction system. A new way to assess organ dysfunction in the intensive care unit. JAMA 276:
17
Microsimulation of organ dysfunction
802-810
10. Hebert PC, Drummond AJ, Singer J, Bernard GR, Russell JA (1993) A simple multiple system
organ failure scoring system predicts mortality of patients who have sepsis syndrome . Chest 104:
230-235
11. Wheeler A, Carmichael L, Christman B (1995) Renal function abnormalities in sepsis. Am J Respir
Crit Care Med 151: A317.
12. Ferreira FL, Bota DP, Bross A, Melot C, Vincent JL (2001) Serial evaluation of the SOFA score to
predict outcome in critically ill patients. JAMA 286: 1754-1758
13. Cox DR, Oakes D (1984) Analysis of Survival Data. Chapman & Hall, London
14. Gray RJ (1994) Spine-based tests in survival analysis. Biometrics 50: 640-652
15. Vincent JL, de Mendonca A, Cantraine F, Moreno R, Takala J, Suter PM, Sprung CL, Colardyn F,
Blecher S (1998) Use of the SOFA score to assess the incidence of organ dysfunction/failure in
intensive care units: Results of a multicenter, prospective study. Crit Care Med 26: 1793-1800
16. Cronin KA, Legler JM, Etzioni RD (1998) Assessing uncertainty in microsimulation modelling with
application to cancer screening interventions. Stat Med 17: 2509-2523
17. Moreno R, Vincent JL, Matos R, Mendonca A, Cantraine F, Thijs L, Takala J, Sprung C, Antonelli
M, Bruining H, Willatts S (1999) The use of maximum SOFA score to quantify organ
dysfunction/failure in intensive care. Results of a prospective, multicentre study. Working Group
on Sepsis related Problems of the ESICM. Intensive Care Med 25: 686-696
18. Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ,
18
Microsimulation of organ dysfunction
Lotring T, Damiano AM, Harrell FEJ (1991) The APACHE III prognostic system. Risk prediction of
hospital mortality for critically ill hospitalized adults. Chest 100: 1619-1636
19. Rubin DB (1987) Multiple Imputation for Nonresponse in Surveys. John Wiley & Sons, New York,
NY
20. Barie PS, Hydo LJ, Fischer E (1994) A prospective comparison of two multiple organ
dysfunction/failure scoring systems for prediction of mortality in critical surgical illness. J Trauma
37: 660-666
21. Zimmerman JE, Knaus WA, Wagner DP, Sun X , Hakim RB, Nystrom PO (1996) A comparison of
risks and outcomes for patients with organ system failure: 1982-1990. Crit Care Med 24: 1633-1641
22. Rangel-Frausto MS, Pittet D, Hwang T, Woolson RF, Wenzel RP (1998) The dynamics of disease
progression in sepsis: Markov modeling describing the natural history and the likely impact of
effective antisepsis agents. Clin Infect Dis 27: 185-190
23. Sonnenberg FA, Beck JR (1993) Markov models in medical decision making: A practical guide.
Med Decis Making 13: 322-338
19
Microsimulation of organ dysfunction
Figure legends
Figure 1. Conceptual model of a patient’s progression through the time-course of an ICU
stay. The microsimulation starts on the day of ICU admission. Each day, a patient may either
be discharged alive from the ICU, remain in the ICU, or die in the ICU. The probabilities of
these events are derived from the development cohort and based on the SOFA scores on that
day. The SOFA scores of patients remaining in the ICU are recalculated for the next day, based
on the SOFA scores of the actual day and demographic characteristics.
Figure 2. Predicted and observed sequential organ dysfunction in the validation cohort.
The predictions of mean daily SOFA scores (solid line) are based on 500 simulations. Observed
mean daily scores are presented as 95% confidence intervals (dashed lines) around the value of
the mean. The model described well the observed SOFA scores of the validation cohort.
However, there was an underestimation of the hepatic and renal scores for patients with longer
ICU stays.
Figure 3. Predicted and observed outcomes in the validation cohort. The predictions are
based on 500 simulations. The model predicted mortality well in the validation cohort, as both
predicted mortality and ICU discharge curves (solid lines) and observed curves (dashed lines)
closely paralleled each other.
20
Microsimulation of organ dysfunction
Table 1. Characteristics of the development and validation cohorts.
Development cohort
(N = 989)
Validation cohort
(N = 460)
37
12
Age (yrs ± SD)
53.8 ± 20.2
54.6 ± 19.4
0.51
Males [N (%)]
621 (63.6)
288 (63.0)
0.52
Intensive care units
Admission source [N (%)]
P-value
0.38
Emergency room
346 (35.9)
174 (38.2)
0.31
Hospital ward
256 (26.6)
114 (25.1)
0.55
Operating room
255 (26.5)
121 (26.6)
0.96
Other hospital
107 (11.1)
46 (10.1)
0.57
Admission type [N (%)]
<0.001
Elective surgery
180 (18.3)
80 (17.4)
0.71
Emergent surgery
187 (19.0)
66 (14.4)
0.03
Trauma
127 (12.9)
54 (11.8)
0.55
Medical
460 (46.7)
213 (46.4)
0.93
32 (3.2)
46 (10.0)
0.01
14.0 ± 18.4 (7)
13.7 ± 18.7, 7
0.75
Cardiovascular
8.3 ± 14.5 (2)
7.8 ± 14.4, 2
0.59
Neurologic
9.0 ± 17.6 (1)
9.1 ± 20.2, 0
0.93
Renal
5.8 ± 14.7 (1)
8.2 ± 16.6, 2.5
Hematologic
4.6 ± 8.6 (1)
4.8 ± 9.1, 1
0.68
Hepatic
4.3 ± 10.6 (0)
4.6 ± 12.4, 0
0.65
Global†
46.0 ± 63.5 (21)
48.2 ± 71.0, 23
0.87
Acute coronary
Cumulative SOFA score*
[mean ± SD (median)]
Respiratory
ICU mortality (%)
ICU length of stay [days ± SD
(median)]
21.0
7.7 ± 7.5 ( 5)
21
21.1
8.2 ± 7.8, 5
0.005
0.76
0.29
Microsimulation of organ dysfunction
Abbreviations: N=number of patients; SD=standard deviation; SOFA=sequential organ
function assessment; ICU=intensive care unit.
* Cumulative SOFA score is the sum of scores for an individual patient over the entire ICU stay.
† Global SOFA score is the sum of single organ scores for an individual patient on a given day.
22
Microsimulation of organ dysfunction
Table 2. Model performance in the development and validation cohorts.
Development cohort
Predicted
Actual
SR (95% CI)
13.7
14.0
0.98 (0.87-1.08)
Cardiovascular
8.1
8.3
Hematologic
4.9
Neurologic
Validation cohort
Predicted
Actual
SR (95% CI)
13.7
13.7
1.00 (0.86-1.03)
0.97 (0.82-1.12)
8.5
7.8
1.08 (0.93-1.24)
4.5
1.08 (0.92-1.24)
5.3
4.6
1.15 (0.97-1.33)
8.5
8.9
0.95 (0.78-1.12)
7.9
9.2
0.86 (0.69-1.03)
Hepatic
3.0
4.3
0.70‡ (0.56-0.85)
3.4
4.6
0.74‡ (0.59-0.89)
Renal
5.3
5.8
0.92 (0.76-1.07)
6.5
8.3
0.78‡ (0.65-0.90)
43.4
45.9
0.95 (0.84-1.05)
45.2
48.4
1.04 (0.91-1.19)
ICU length of stay
(days)
7.5
7.7
0.99 (0.91-1.08)
7.7
8.1
0.95 (0.86-1.03)
ICU mortality (%)
20.2
21.0
0.96 (0.78-1.14)
20.1
19.9
1.01 (0.81-1.21)
Cumulative SOFA
score* (mean)
Respiratory
Global†
Abbreviations: SR=standardized ratio; CI=confidence interval; SOFA=sequential organ
function assessment; ICU=intensive care unit.
* Cumulative SOFA score is the sum of scores for an individual patient over the entire ICU stay.
† Global SOFA score is the sum of single organ scores for an individual patient on a given day.
‡ Significant difference between predicted and observed values (p<0.05).
23
Microsimulation of organ dysfunction
Table 3. Organ-organ-interactions as evaluated by regression coefficients.*
Today’s score
Yesterday’s score
Pulmonary
Cardiovascular
Pulmonary
0.707
0.078
0.038
Cardiovascular
0.056
0.814
NS
Hematologic
Neurologic
Renal
Hepatic
Hematologic Neurologic
Renal
Hepatic
0.032
NS
NS
0.038
0.027
0.038
0.022
0.040
0.836
NS
0.019
0.026
0.017
0.027
NS
0.926
0.012
NS
NS
0.037
0.034
NS
0.845
0.038
0.033
0.044
0.098
0.012
0.034
0.772
* The linear regression coefficients were derived from the entire set of observations and convey
the strength of association between yesterday’s SOFA scores and today’s SOFA score across
organ systems. A positive coefficient reflects a positive correlation (worse [better] scores
yesterday correlate with worse [better] scores today), and a possible physiologic interaction.
NS=Not a significant predictor in the model (p<0.05) and therefore no associated coefficient in
the final multinomial equations.
24
Figure 1
SOFA dataset
1449 patients
11417 ICU-days
Development set
Validation set
37 ICUs
990 patients
12 ICUs
459 patients
Outcome sub-model
•Daily probability of each
of 3 outcomes
SOFA sub-models
•Daily probability of each
of 5 scores (0-4)
•6 sub-models, 1 for each organ
systems
Global model
Integrate all submodels in a
microsimulation that propels 500
cohorts of patients with known
day 1 scores through ICU
discharge
Apply global model
Generate desired
outcomes from day 1
SOFA scores
Compare to actual
outcomes
Figure 2
Mean SOFA score
Respiratory
Cardiovascular
Hematologic
2.5
2.5
2.5
2
2
2
1.5
1.5
1.5
1
1
1
0.5
0.5
0.5
0
0
0
0
5
10
15
20
25
30
0
5
Neurologic
10
15
20
25
30
0
Hepatic
2.5
2.5
2
2
2
1.5
1.5
1.5
1
1
1
0.5
0.5
0.5
0
0
0
5
10
15
20
25
30
0
5
10
15
10
15
20
25
30
20
25
30
Renal
2.5
0
5
20
ICU day
25
30
0
5
10
15
Figure 3
1
Deceased
0.8
Cumulative proportion
Still in ICU
0.6
Discharged from ICU
0.4
0.2
Predicted
Actual
0
3
6
9
12
15
ICU day
18
21
24
27
30