Online data supplement
Exhaled nitric oxide as a marker of asthma control in
smoking patients
____________________________________________
Alain Michils1, MD; Renaud Louis2, MD; Rudi Peché3, MD; Sandra
Baldassarre1, MD; Alain Van Muylem1, PhD
1
Chest Department - CUB Erasme, Brussels, Belgium
2
Chest Department - CHU Sart-Tilmant, Liège, Belgium
3
Chest Department- CHU André Vésale, Montigny- le-Tilleul, Belgium
METHODS
Statistical methods
1. Intrinsic characteristics of an index
1.1. Sensitivity and specificity
If N events are rated as positive or negative (e.g. positive = well-controlled asthma; negative
= not-well controlled asthma) by a reference method (Gold Standard: GS), the ability of an
index to discriminate between these complementary states, for a given cut-off value, may be
assessed by using a contingency table. The index is considered as positive or negative
depending on its position relative to the cut-off value. The contingency table appears like
GS positive
n+
GS negative
n-
Index positive
TP
FP
Index negative
FN
TN
where n+ an n- are the amounts of positive and negative events (N = n+ + n-) according to the
Gold standard, respectively. TP and TN are the amounts of true positive and true negative
events, respectively; i.e. the amounts of events correctly rated by the index. FP and FN are the
amounts of false positive and false negative events, respectively, i.e. the amounts of events
incorrectly rated by the index.
From the contingency table, sensitivity (Se) and specificity (Sp) may be derived by
Se = TP/n+
(1)
Sp = TN/n-
(2)
Se and Sp are intrinsic characteristics of the index (for this given cut-off value). Se and Sp are
independent from the prevalence, the latter being defined as the positive events rate in a
considered context (the "prior" probability of a positive event). From the contingency table,
the prevalence is equal n+/N.
1.2. ROC curve
A contingency table may be constructed, and sensitivity and specificity computed, for a range
of cut-off values. The Receiving Operator Characteristics (ROC) curve is the true positive rate
(sensitivity) as a function of the false positive rate (1-specificity) for the considered range of
cut-off values. This curve allows visually assessing the relative "quality" of several indexes.
It also allows "choosing" an optimal cut-off value for a given index according to investigator's
criteria.
The area under the ROC curve (AUC) may be computed and its difference relative to 0.5
statistically assessed. If AUC not different from 0.5 (which is the area under the first
bisector), it means that the considered index is as useful as tossing a coin.
1.3. The Youden's index
The Youden's index (J), is the difference between the true positive rate and the false positive
rate. Maximizing this index allows to find, from the ROC curve, an optimal cut-off point
independently from the prevalence. According to its definition and as illustrated on Fig.1, J is
the vertical distance between the ROC curve and the first bisector (or chance line). If F(x) is
the function describing the ROC curve, with x = 1-specificity, we may write
J(x) = F(x)-x
(3)
When J is maximal, J'(x) = 0, where J' is the derivative of J.
From Eq. 1:
J'(x)= F'(x) -1,
(4)
where F' is the derivative of F.
Hence, when J is maximal, F'(x) = 1, meaning that the tangent to the ROC curve is parallel to
the first bisector (slope = 1). It implies that, around this point, a gain (or a loss) in specificity
results in a loss (or a gain) of the same amplitude in sensitivity.
Figure 1
100
Cc
urv
e
Youden index
80
60
true positive
rate
st
bi
s
se
ct
or
40
20
false positive
rate
fir
Sensitivity (%)
(true positive rate)
RO
0
0
20
40
60
80
100
100-specificity (%)
(false positive rate)
2. Predictive values
Positive (PPV) and negative (NPV) predictive values are the probability to rate correctly an
event using the index; i.e. the probability that the event is actually positive (negative) if the
index is positive (negative). They may be derived from the contingency table by
PPV = TP/(TP+FP)
(5)
NPV = TN/(TN+FN)
(6)
Contrary to Se and Sp, PPV and NPV are dependent on the context through the prevalence. It
implies that PPV and NPV, derived from the contingency table, may be used only if n+/N is
close to the actual prevalence of the considered event in the considered "real-life" context.
The accuracy (Acc) defined as (TP+TN)/N is also context dependent.
It is to be noted that PPV and NPV may be computed from Se and Sp (intrinsic
characteristics) and any given prevalence P (linked to a specific context) by the Baye's
formula:
PPV 
P  Se
P  Se  (1  P)  (1  Sp)
(7)
NPV 
1  P   Sp
P  Se  1  P   1  Sp 
(8).
RESULTS
Operating characteristics of FeNO at various cut-off values
Table 1: Cross-sectional assessment of asthma control
Non-smokers (N=411, n+=197)
p<0.001
Cut-off
(ppb)
TP
30
40
50*
60
70
112
134
142
171
179
TN
150
131
120
86
58
FP
64
83
94
128
156
FN
85
63
55
26
18
Se (%)
57
68
72
87
91
Sp (%)
70
61
56
40
27
PPV (%)
64
62
61
57
53
NPV (%)
64
68
68
77
76
Acc (%)
64
64
64
63
58
Smokers (N=59, n+=15)
p=0.39
Cut-off
(ppb)
TP
15
20
25*
35
45
7
10
10
11
12
TN
30
26
21
14
11
FP
14
18
23
30
33
FN
8
5
5
4
3
Se (%)
48
64
66
76
80
Sp (%)
68
59
48
32
24
PPV (%)
33
36
30
27
27
NPV (%)
79
84
81
78
79
Acc (%)
63
61
53
42
39
*: cut-off value corresponding to maximal Youden's index. N and n+ are the total amount of cases
and the amount of positive cases, respectively. A positive case is a (partially or totally) controlled
asthma. TP, TN, FP, FN are the amount of true positive, true negative, false positive, and false
negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity, specificity, positive
predictive value, negative predictive value, and accuracy, respectively; p is the statistical significance
of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO<cut-off associated with
a (partially or totally) controlled asthma.
Table 2: Assessment of a change from an uncontrolled (ACQ score  1.5) to a controlled
(ACQ < 1.5) asthma
Non-smokers (N=283, n+=133)
p<0.001
Cut-off
(%change)
TP
-20
-25
-30*
-35
-40
101
96
90
88
82
TN
93
98
107
113
120
FP
57
52
43
37
30
FN
32
37
43
45
51
Se (%)
76
72
68
66
62
Sp (%)
62
65
71
75
80
PPV (%)
64
65
68
70
73
NPV (%)
74
73
72
72
70
Acc (%)
69
69
70
71
71
Smokers (N=52, n+=17)
p=0.016
Cut-off
(%change)
TP
15
20
25*
35
45
12
12
12
11
9
TN
22
23
23
23
24
FP
13
12
12
12
11
FN
5
5
5
6
8
Se (%)
71
71
71
65
53
Sp (%)
64
65
66
66
69
PPV (%)
48
50
50
48
45
NPV (%)
81
82
82
79
75
Acc (%)
65
67
67
65
63
*: cut-off value corresponding to maximal Youden's index. N and n+ are the total amount of cases
and the amount of positive cases, respectively. A positive case is an uncontrolled asthma becoming
controlled. TP, TN, FP, FN are the amount of true positive, true negative, false positive, and false
negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity, specificity, positive
predictive value, negative predictive value, and accuracy, respectively; p is the statistical significance
of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO change <cut-off (e.g.
-40% vs -30%) associated with a positive case.
Table 3: Assessment of a change from a controlled (ACQ score < 1.5) to an uncontrolled
(ACQ 1.5) asthma
Non-smokers (N=360, n+=65)
p=0.001
Cut-off
(%change)
TP
40
45
50*
55
60
29
29
27
27
26
TN
212
215
221
224
227
FP
83
80
74
71
68
FN
36
36
38
38
39
Se (%)
44
44
42
41
40
Sp (%)
72
73
75
76
77
PPV (%)
26
27
26
28
28
NPV (%)
85
86
86
85
85
Acc (%)
67
68
69
70
70
Smokers (N=40, n+=10)
p=0.017
Cut-off
(%change)
TP
40
45
50*
55
60
7
7
7
6
6
TN
24
25
26
26
26
FP
6
5
4
4
4
FN
3
3
3
4
4
Se (%)
70
70
68
62
58
Sp (%)
80
84
87
87
88
PPV (%)
54
58
63
60
60
NPV (%)
89
89
89
87
87
Acc (%)
78
80
83
80
80
*: cut-off value corresponding to maximal Youden's index. N and n+ are the total amount of cases
and the amount of positive cases, respectively. A positive case is a controlled asthma becoming
uncontrolled. TP, TN, FP, FN are the amount of true positive, true negative, false positive, and false
negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity, specificity, positive
predictive value, negative predictive value, and accuracy, respectively; p is the statistical significance
of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO change >cut-off (e.g.
60% vs 50%) associated with a positive case.
Table 4: Improvement (ACQ < -0.5) assessment of asthma control
Non-smokers
ICS dose 500$
(N=306, n+=116
p<0.001
ICS dose 500$
(N=337, n+=97)
-10 -15 -20* -25 -30
-10 -15 -20* -25 -30
Total
(N=643, n+=257)
p<0.001
p<0.001
Cut-off -10 -15 -20* -25 -30
(%change)
146 177 164 159 149
TP
274 255 274 282 297
TN
85 82 86 73 68
100 96 78 82 79
127 135 127 143 150
108 122 145 133 141
FP
112 131 112 104 89
63 55 63 47 40
88 74 51 63 55
FN
Se (%)
111 80 93 98 108
31 34 30 43 48
41 45 63 59 62
57 69 64 62 58
73 71 74 63 59
71 68 55 58 56
Sp (%)
71 66 71 73 77
67 71 67 75 79
55 62 74 68 72
PPV (%) 57 57 61 62 63
57 60 58 61 63
53 56 60 57 59
NPV (%) 71 76 74 74 73
80 80 80 77 76
72 73 70 69 69
65 67 68 69 69
69 71 70 71 71
62 65 66 64 65
Acc (%)
Smokers
Total
(N=92, n+=40)
ICS dose 500$
(N=35, n+=14)
ICS dose >500$
(N=57, n+=26)
p<0.001
p<0.001
p=0.070
-10 -15 -20* -25 -30
-10 -15 -20* -25 -30
12 12
8
14 13 13 12 11
16 16 18 18 18
21 22 22 22 23
Cut-off -10 -15 -20* -25 -30
(%change)
25 24 23 20 20
TP
37 37 38 40 42
TN
9
8
FP
15 15 14 12 10
5
5
3
3
3
10
FN
Se (%)
15 16 17 20 20
2
2
5
6
6
12 13 13 14 15
62 60 57 51 49
88 86 62 54 54
54 50 50 46 42
Sp (%)
72 72 74 76 80
74 74 84 84 84
68 71 71 71 74
PPV (%) 63 62 62 63 67
71 71 75 73 73
58 59 59 57 58
NPV (%) 71 70 70 67 68
89 89 78 75 75
64 63 63 61 61
67 66 66 65 67
80 80 77 74 74
61 61 61 60 60
Acc (%)
9
9
9
8
*: cut-off value corresponding to maximal Youden's index. $ : ICS dose in g equ BDP.day-1 . N and
n+ are the total amount of cases and the amount of positive cases, respectively. A positive case is a
decrease amplitude of ACQ score > 0.5. TP, TN, FP, FN are the amount of true positive, true negative,
false positive, and false negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity,
specificity, positive predictive value, negative predictive value, and accuracy, respectively; p is the
statistical significance of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO
change <cut-off (e.g. -30% vs -20%) associated with a positive case.
Table 5: Worsening (ACQ > 0.5) assessment of asthma control
Non-smokers
Total
(N=643, n+=161)
ICS dose 500$
(N=306, n+=64)
ICS dose 500$
(N=337, n+=97)
p<0.001
p<0.001
p<0.001
Cut-off
*
(%change) 20 25 30 35 40
90 85 82 76 72
TP
342 357 366 376 381
TN
20 25 30* 35 40
43 43 43 39 38
20 25 30* 35 40
46 43 41 38 36
169 179 184 189 191
182 182 187 190 194
FP
140 125 116 106 101
73 63 58 53 51
58 58 53 50 46
FN
Se (%)
71 76 79 85 89
21 21 21 25 26
51 54 56 59 61
56 53 51 47 45
67 67 67 61 59
47 44 42 39 37
Sp (%)
71 74 76 78 79
70 74 76 78 79
76 76 78 79 81
PPV (%) 37 40 37 42 42
37 41 43 42 43
44 43 44 43 44
NPV (%) 84 82 84 82 81
89 90 90 88 88
78 77 77 76 76
67 69 70 70 70
69 73 74 75 75
68 67 68 68 68
Acc (%)
Smokers
Total
(N=92, n+=26)
ICS dose 500$
(N=35, n+=11)
ICS dose >500$
(N=57, n+=15)
p<0.001
p<0.001
p=0.037
20 25 30* 35 40
8 8 8 8 8
20 25 30* 35 40
10 10 10 10 9
20 21 22 22 23
28 30 30 30 30
20 16 15 15 14
4
3
2
2
1
14 12 12 12 12
9
3
3
3
3
3
5
Cut-off
*
(%change) 20 25 30 35 40
17 17 17 17 16
TP
46 50 51 51 52
TN
FP
9
9
9
10
5
5
5
6
FN
Se (%)
67 67 67 66 63
70 70 70 70 70
64 64 64 64 57
Sp (%)
70 76 77 78 79
82 86 91 93 96
66 71 71 71 71
PPV (%) 46 52 52 53 53
67 73 78 80 89
42 43 43 43 43
NPV (%) 84 85 86 85 84
87 88 87 88 88
85 85 85 85 83
68 73 74 74 74
80 83 86 86 89
67 70 70 70 68
Acc (%)
*: cut-off value corresponding to maximal Youden's index. $ : ICS dose in g equ BDP.day-1 . N and
n+ are the total amount of cases and the amount of positive cases, respectively. A positive case is a
increase amplitude of ACQ score > 0.5. TP, TN, FP, FN are the amount of true positive, true negative,
false positive, and false negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity,
specificity, positive predictive value, negative predictive value, and accuracy, respectively; p is the
statistical significance of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO
change >cut-off (e.g. 40% vs 30%) associated with a positive case.