Marie Kassapian1,2, Toufik Zahaf3, Fabian Tibaldi3
1
2
University of Hasselt
Frontier Science Foundation Hellas
3
GlaxoSmithKline (GSK) Vaccines
Tel Aviv, 22.04.2013
The disease
Herpes Zoster
 After a varicella (chicken-pox) incident, the virus
may be expressed again after several years.
 Basically in ages above 60 years old.
 Can turn out very severe in terms of pain.
Comparison of Statistical Tests in Presence of Many Zeros Data
2
Zoster Brief Pain Inventory (ZBPI) Questionnaire:
A
set of questions to determine the level of pain
interfering with functional status & quality of life
 Scale from 0 to 10
 Filled in every day during follow-up period (182
days)
 Score=0
Non-incident case
&
Score>0 Incident case
 Final score: Sum of worst daily scores (182-1820)
Comparison of Statistical Tests in Presence of Many Zeros Data
3
 The
resulted data after the end of the follow-up
period contain many zeros.
These zeros belong to the scores of those
individuals that did not experience zoster.
Need for methods capable of handling such
datasets.
 Important
to account both for the reduction in the
total number of cases as well as for the reduction
in the severity of pain.
Comparison of Statistical Tests in Presence of Many Zeros Data
4

Burden-of-Illness (BOI) Measure - Chang et al. (1994)
Test accounting for:
 Disease incidence
 Disease severity
Assign a score to each patient and create the Burden-ofIllness score by adding them.
Comparison of Statistical Tests in Presence of Many Zeros Data
5
Statistic:
where :
nj represents the total number of pts. in each
group.
mi represents the number of infected pts. in each
group.
Wji represents the BOI score of the ith patient in
the jth group.
For the groups:
0:placebo group
&
Comparison of Statistical Tests in Presence of Many Zeros Data
1:vaccine group
6




Choplump test - Follmann et al. (2009)
Sort the scores in each group.
Toss out the same number of zeros in both groups.
1 group with no zeros + 1 group with few zeros.
Statistic:
 n=number of pts randomized in each group
 m=max(m0,m1)
 S2m=pooled variance based on the m largest W’s in each group
Calculation of the p-value can be: Exact or Approximate
Comparison of Statistical Tests in Presence of Many Zeros Data
7
 Comparison
between the test suggested by
Chang et al. (1994) and the one suggested by
Follmann et al. (2009).
Comparison of Statistical Tests in Presence of Many Zeros Data
8
Comparison of Statistical Tests in Presence of Many Zeros Data
9
 No real data
 Simulated dataset based on assumptions for the
sample size, the incidence rate and the risk
reduction.
 Number of cases:
Placebo: Incidence rate * N0* years of follow-up
Vaccine: Incidence rate * N1 * Risk * years of
follow-up
Comparison of Statistical Tests in Presence of Many Zeros Data
10
N
Mean
Std. Dev. Median
Min.
Max.
All cases (W* ≥ 0)
Placebo (Z=0)
8,000
28.69
195.92
0
0
1431
Vaccine (Z=1)
8,000
4.01
50.58
0
0
690
Zoster cases only (W* > 0)
Placebo (Z=0)
168
1366.20
21.60
1366
1320
1431
Vaccine (Z=1)
50
641.54
21.02
641
597
690
*W: the Burden-of Illness score of a patient
Comparison of Statistical Tests in Presence of Many Zeros Data
11

Normality tests to observe the distribution of the
patients’ BOI scores.
 All cases:
Z=0
Z=1
p-value<0.01 (both groups)
Comparison of Statistical Tests in Presence of Many Zeros Data
12
 Zoster
cases only:
Z=0
p-value=0.128 (placebo)
Z=1
p-value=0.15 (vaccine)
Comparison of Statistical Tests in Presence of Many Zeros Data
13
Area Under the Curve for the two groups based on
the mean daily severity (BOI) scores.
Comparison of Statistical Tests in Presence of Many Zeros Data
14
Implementation of Chang et al. method:
Test
Statistic
p-value
Incidence Rate
63.87
<0.001
Severity score per case
209.49
<0.001
Burden-of-illness score
11.22
<0.0001
Findings:
 P-value from Chang et al. method much more
significant than those yielded for the separate
tests.
 Both methods (Choplump &
Comparison of Statistical Tests in Presence of Many Zeros Data
Chang) reject H0.
15
1st case: Exact p-value
Patient ID
1
W=score
0
Z=group
0
2
3
4
5
1326 1369 1387 1374
0
0
0
0
6
7
8
9
10
0
0
0
0
650
1
1
1
1
1
H0: No difference in B.O.I. scores between placebo
and vaccine group
p-value=0.047
Comparison of Statistical Tests in Presence of Many Zeros Data
16
Conclusion:
The treated groups differ in 2 ways:
 Difference in the number of incidents per group
 Difference in the mean severity scores per
group
Note:
• N=10 patients and M=5 incident cases: 252
permutations
• N=20 patients and M=10 incident cases: 182,756
permutations
Comparison of Statistical Tests in Presence of Many Zeros Data
17
2nd case: Approximate p-value
Simulated dataset (RR=70% , Incidence rate=0.7%) :
N=16,000 pts.
N0=N1=8,000 pts.
M=218 cases
M0=168 cases
M1=50 cases
K=15,782 zeros K0=7,732 zeros K1=7,950 zeros
H0: No difference in B.O.I. scores between placebo
and vaccine group
p-value=2.72*10-31
Comparison of Statistical Tests in Presence of Many Zeros Data
18
Conclusion:
Again, the groups differ in 2 ways:
 Difference in the number of incidents per group
 Difference in the mean severity scores per group
Comparison of Statistical Tests in Presence of Many Zeros Data
19
Chang method cannot compute very small p-values.
 Comparison between the tests not straightforward.
 Implementation of power analysis in order to find
the most powerful test.
Building of different scenarios based on:
 Sample size (1,000 , 2,000 , 5,000 , 10,000 , 20,000)
 Risk reduction (30% , 50% , 70%)
 Severity reduction (Yes , No)
 Simulation of 1,000 datasets for each scenario.
Comparison of Statistical Tests in Presence of Many Zeros Data
20
Hypothesis
Risk
Reduction
0%
Sample size
H0
HA(1)
Yes
30%
HA(2)
HA(3)
N
No
Yes
50%
HA(4)
HA(5)
No
Yes
70%
HA(6)
Severity
Reduction
No
No
Ranges for severity scores:
RR=0%
RR=30%
RR=50%
RR=70%
Placebo
1-10
4-10
4-10
4-10
Vaccine
1-10
3-9
2-8
1-7
Comparison of Statistical Tests in Presence of Many Zeros Data
21
Boxplots of scores under the different hypotheses (N=10,000)
Comparison of Statistical Tests in Presence of Many Zeros Data
22
Comments based on the summary statistics of the
resulted p-values:
 The alternative hypotheses that also account for
severity reduction, apart from risk reduction, present
incredibly small distances between the minimum and
the maximum values.
 More obvious in the case of the Choplump test.
 As N increases, the mean p-values decrease much
faster especially for the Choplump test.
Comparison of Statistical Tests in Presence of Many Zeros Data
23
Estimated type I error probabilities for each test:
N
1,000
2,000
5,000
10,000
20,000
Chang
0.01
0.011
0.013
0.02
0.026
Choplump
0.02
0.027
0.025
0.025
0.032
Estimated power:
Chang
30%
N
Choplump
50%
70%
30%
50%
70%
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
1,000
0.001
0.001
0.003
0.001
0.21
0.17
0.09
0.003
0.24
0.18
0.35
0.44
3,000
0.005
0.002
0.25
0.16
0.39
0.31
0.21
0.035
0.36
0.24
0.74
0.61
5,000
0.43
0.01
0.58
0.13
0.68
0.56
0.51
0.12
0.65
0.39
0.81
0.77
10,000
0.77
0.66
0.86
0.71
0.91
0.80
0.78
0.54
0.88
0.57
0.93
0.85
20,000
0.93
0.89
0.95
0.91
0.99
0.94
0.95
0.92
0.97
0.94
0.98
0.97
 Both
tests represent adequate approaches to the
issue of handling a lot of zeros.
 The Choplump test is dominant over its
competitor only in cases when the efficacy of the
vaccine is reflected by both risk and severity
reduction.
Comparison of Statistical Tests in Presence of Many Zeros Data
25
Thank you
Comparison of Statistical Tests in Presence of Many Zeros Data
26