ONLINE APPENDIX
Adjudicated Cause of Cardiac and Non-cardiac Deaths
The adjudicated cause of cardiac and non-cardiac deaths in our cohort was as follows: (1)
Cardiac deaths were adjudicated as fatal myocardial infarction (n=0), sudden cardiac death
(n=3), worsening heart failure (n=6), and other cardiac death (n=3); (2) Noncardiac deaths
were adjudicated as vascular death (n=0), non-hemorrhagic stroke (n=0), other (n=16), and
unknown death (n=2).
Sensitivity Analysis for Incomplete Follow-Up
Methods
Complete follow-up, defined as >90 days or through the end of the study period was not
available for 71 (7.9%) patients. In order to evaluate the effects of incomplete follow-up,
sensitivity analyses utilizing several approaches were conducted. First, right-point
imputation was conducted wherein all patients with incomplete follow-up were imputed to
have no events through the end of the study period. Although, the overall event rate among
patients with complete follow-up was low (6.7%), this assumption was unlikely to be true. A
second, more robust approach utilizing multiple imputation was also conducted. The
primary outcome and time to this event were multiply-imputed with 25 imputations
(relative efficiency >99.7%). Because event times were approximately exponentially
distributed, they were log transformed prior to imputation. Cox proportional hazard models
were fit to the imputed or multiply-imputed data. Parameter estimates for
multiply-imputed data sets were then combined (1).
Results
Right-point imputation, which assumes that none of the patients with incomplete data
experienced events during the follow-up period generated estimates for hazard ratios
comparable to analysis of non-imputed data (Supplemental Table 1). Although calcium score
(CAC) contained incremental prognostic information beyond clinical risk (Model 2), once
coronary flow reserve (CFR) was incorporated (Model 4), CAC was no longer a significant
predictor of the primary outcome (Model 4).
1
Multiple imputation resulted in imputation of between 1 and 9 MACE events, implying
event rates of 1.4-12.7% among patients with incomplete follow-up, compared to 6.7%
among those with complete follow-up. This approach also resulted in risk estimates that
were essentially identical to those based on non-imputed data (Supplemental Table 1).
References
1. Rubin DB. Multiple Imputation for Nonresponse in Surveys. Hoboken, NJ: John Wiley &
Sons, Inc. 1987.
2
Online Table 1. Multivariable Survival Analysis (CAC zero vs ≥1)
Model
Model 1: Clinical risk score*
Model 2: Clinical risk score
Model 3: Clinical risk score + CFR
Model 5: Clinical risk score + CAC + CFR
+ CAC
Analysis
Fit statistic
p-value
Fit statistic
p-value
Fit statistic
p-value
Fit statistic
p-value
Global χ2
19.0
ref
19.9
0.35
44.5
<0.0001
44.8
<0.0001 (vs. model 1)
<0.0001 (vs. model 2)
0.54 (vs. model 3)
AIC
588.6
ref
589.6
N/S
565.1
<0.0001
566.7
<0.0001 (vs. model 1)
<0.0001 (vs. model 2)
N/S (vs. model 3)
c-index
0.632
ref
[0.550-0.714]
0.638
0.57
[0.554-0.722]
0.719
0.004
[0.647-0.791]
0.719
0.003 (vs. model 1)
[0.648-0.791]
0.007 (vs. model 2)
0.95 (vs. model 3)
Covariate
Duke clinical risk
score
Hazard ratio
1.06
[1.03-1.10]
Hazard ratio
<0.0001
1.06
Hazard ratio
0.003
[1.03-1.09]
1.06
Hazard ratio
0.0002
[1.03-1.10]
1.06
0.001
[1.02-1.09]
(per 10%)
CAC (≥1 vs. 0)
1.35
0.35
1.22
[0.72-2.51]
CFR (per 10%
increase)
0.54
[0.65-2.31]
0.82
[0.76-0.89]
<0.0001
0.82
<0.0001
[0.76-0.89]
A log transformation of the Duke clinical risk score and CFR was used for the analyses to adjust for the rightward skew of the data and to reduce
heteroscedasticity. CAC was dichotomized as present or absent. Global χ2 indicated likelihood ratio chi-squared statistic for the entire model. AIC refers to
Akaike Information Criterion. P-values for model fit statistics (i.e. global χ2, AIC and c-index) are for comparisons with model 1, unless specified. C-indices are
calculated for 3-year event date. CAC = coronary artery calcium, CFR = coronary flow reserve. *The components to calculate Duke clinical risk score consist of
age, sex, typical angina, atypical angina, history of myocardial infarction, ECG Q-wave, ECG ST-T wave changes, smoking, dyslipidemia, diabetes, age×sex,
history of myocardial infarction×ECG Q-wave, age×sex, age×dyslipidemia, and sex×smoking (interaction).11
4
Online Table 2. Multivariable Survival Analysis (CAC <400 vs. ≥400)
Model
Model 1: Clinical risk score*
Model 2: Clinical risk score
Model 3: Clinical risk score + CFR
Model 5: Clinical risk score + CAC + CFR
+ CAC
Analysis
Fit statistic
p-value
Fit statistic
p-value
Fit statistic
p-value
Fit statistic
p-value
Global χ2
19.0
ref
22.7
0.05
44.5
<0.0001
46.5
<0.0001 (vs. model 1)
<0.0001 (vs. model 2)
0.14 (vs. model 3)
AIC
588.6
ref
586.8
0.18
565.1
<0.0001
565.0
<0.0001 (vs. model 1)
<0.0001 (vs. model 2)
0.75 (vs. model 3)
c-index
0.632
ref
[0.550-0.714]
0.623
0.46
[0.542-0.705]
0.719
0.004
[0.647-0.791]
0.712
0.006 (vs. model 1)
[0.641-0.783]
0.003 (vs. model 2)
0.22 (vs. model 3)
Covariate
Duke clinical risk
score
Hazard ratio
1.06
[1.03-1.10]
Hazard ratio
<0.0001
1.06
Hazard ratio
0.0005
[1.02-1.09]
1.06
Hazard ratio
0.0002
[1.03-1.10]
1.06
0.001
[1.02-1.09]
(per 10%)
CAC (≥400 vs <400)
1.76
0.05
1.53
[1.01-3.09]
CFR (per 10%
increase)
0.14
[0.87-2.70]
0.82
[0.76-0.89]
<0.0001
0.82
<0.0001
[0.76-0.89]
5
A log transformation of the Duke clinical risk score and CFR was used for the analyses to adjust for the rightward skew of the data and to reduce
heteroscedasticity. CAC was dichotomized as <400 or ≥400. Global χ2 indicated likelihood ratio chi-squared statistic for the entire model. AIC refers to Akaike
Information Criterion. P-values for model fit statistics (i.e. global χ2, AIC and c-index) are for comparisons with model 1, unless specified. C-indices are
calculated for 3-year event date. CAC = coronary artery calcium, CFR = coronary flow reserve. *The components to calculate Duke clinical risk score consist of
age, sex, typical angina, atypical angina, history of myocardial infarction, ECG Q-wave, ECG ST-T wave changes, smoking, dyslipidemia, diabetes, age×sex,
history of myocardial infarction×ECG Q-wave, age×sex, age×dyslipidemia, and sex×smoking (interaction).11
6
Online Table 3. Sensitivity Analysis
No Imputation
Hazard Ratio
P-Value
Right-Point Imputation
Hazard Ratio
P-Value
Multiple Imputation
Hazard Ratio
P-Value
Model 1
Clinical Risk Score (per 10% increase) 1.06 [1.03-1.10] <0.0001 1.06 [1.03-1.09]
0.0001
1.06 [1.03-1.09] <0.0001
Clinical Risk Score (per 10% increase) 1.05 [1.02-1.08]
0.003
1.05 [1.01-1.08]
0.006
1.05 [1.02-1.08]
0.004
CAC (per 2-fold increase)
0.02
1.09 [1.01-1.17]
0.02
1.08 [1.01-1.16]
0.02
0.0002
1.05 [1.02-1.09]
0.0006
1.06 [1.03-1.09]
0.0003
Model 2
1.09 [1.01-1.17]
Model 3
Clinical Risk Score (per 10% increase) 1.06 [1.03-1.09]
CFR (per 10% increase)
0.82 [0.76-0.89] <0.0001 0.83 [0.77-0.89] <0.0001 0.82 [0.76-0.89] <0.0001
Model 4
Clinical Risk Score (per 10% increase) 1.05 [1.02-1.09]
0.004
1.04 [1.01-1.08]
0.01
1.05 [1.01-1.08]
0.006
CAC (per 2-fold increase)
1.06 [0.98-1.13]
0.14
1.06 [0.99-1.14]
0.11
1.06 [0.98-1.13]
0.15
CFR (per 10% increase)
0.83 [0.76-0.89] <0.0001 0.83 [0.77-0.90] <0.0001 0.83 [0.76-0.90] <0.0001
CAC = coronary artery calcium, CFR= coronary flow reserve
Online Table 4. Risk Reclassification
Pre-test Model
(=Clinical Risk Score)
Post-CAC Model
Post-CFR Model
Post-CAC&CFR Model
Annual Risk
Annual Risk
Annual Risk
<2%
2-6%
>6%
<2%
2-6%
>6%
<2%
2-6%
>6%
0 (0)
0 (0)
0 (0)
0 (0)
0 (0)
0 (0)
0 (0)
0 (0)
0 (0)
2-6%
2.4 (3)
53.1 (76)
14.7 (21)
0 (0)
40.6 (61)
26.0 (39)
2.5 (4)
32.1 (50)
29.4 (46)
>6%
0 (0)
9.5 (14)
59.9 (86)
2.8 (4)
15.6 (22)
54.0 (75)
4.0 (6)
5.1 (7)
62.0 (87)
Events Group
Annual Risk <2%
Event NRI
0.025
0.043
0.131
No Events Group
Annual Risk <2%
80.0 (92)
7.0 (8)
0 (0)
69.0 (79)
18.0 (21)
0 (0)
67.0 (77)
20.0 (23)
0 (0)
2-6%
43.6 (11)
301.9 (77)
46.3 (12)
143.0 (36)
186.4 (47)
66.0 (17)
147.5 (37)
180.9 (45)
69.6 (17)
>6%
0 (0)
96.5 (34)
186.1 (66)
14.2 (5)
107.4 (38)
158.0 (57)
24.0 (9)
109.9 (39)
147.0 (52)
Non-Event NRI
0.116
0.237
0.255
Reclassification table for censored data using method of Steyerberg and Pencina20 from 3-year event data. NRI = net reclassification index.
Numbers in parenthesis reflect percentage of pre-test subpopulation.
8
Online Table 5. Net reclassification improvement and integrated discrimination index
Baseline Model
Addition
Net reclassification
Clinical risk score
Clinical risk score
Clinical risk score
Clinical risk score
Clinical risk score
+CAC
+CFR
+CAC
+CFR
+CAC
+CFR
+CFR
+CAC
0.324 [-0.045-0.734]
0.461 [0.098-0.848]
0.587 [0.225-0.948]
0.467 [0.111-0.863]
0.386 [0.025-0.755]
0.141 [-0.003-0.286]
0.280 [0.048-0.502]
0.386 [0.213-0.585]
0.270 [0.077-0.457]
0.136 [-0.012-0.332]
0.013 [0.009-0.019]
0.101 [0.073-0.145]
0.099 [0.070-0.143]
0.086 [0.059-0.125]
-0.003 [-0.010-0.004]
0.296 [0.209-0.384]
2.306 [1.760-2.927]
2.245 [1.702-2.862]
1.505 [1.137-1.932]
-0.018 [-0.063-0.025]
improvement
(Continuous)
Net reclassification
improvement (2 & 6%)
Integrated discrimination
index
Relative integrated
discrimination index
Log transformation of the Duke clinical risk score, CFR and CAC + 1 were used for the analyses to adjust for the rightward skew of the data and
to reduce heteroscedasticity. Estimates are presented with 95% confidence intervals. The net reclassification improvement across clinically
meaningful risk categories of <2%, 2-6% and ≥6% per year was used for analysis. CAC = coronary artery calcium, CFR= coronary flow reserve
9