Safety, Can You Paradigm?
A Statistical Lament
Janet Turk Wittes
Statistics Collaborative
Harms identified late
•fenflurmine-phentermine (Fen Phen)
•Rofecoxib
(Vioxx)
•Troglitazone
(Rezulin)
•HRT
(Premarin and PremPro)
•Celecoxib
(Celebrex)
•Telithromycin
(Ketek)
•Rosiglitazone
(Avandia)
•Antidepressants, anti-epileptics….
2
Could we have identified these harms earlier?
•Troglitazone (Rezulin)
-removed from market in 2000
 Lots of liver abnormalities
 Severe toxicities noted in 1997
 Other equally effective drugs didn’t have same problems
3
Could we have identified these harms earlier?
•Troglitazone (Rezulin)
• Rofecoxib (Vioxx)
-removed from market in 2000
-removed from market in 2004
 Every study showed excess heart attack
 Attributed to benefit of naproxen
4
Could we have identified these harms earlier?
•Troglitazone (Rezulin) – removed from market in 2000
•Rofecoxib (Vioxx) -removed from market in 2004
•HRT
(Premarin/PremPro)-major label change 2006
 Heart attacks in Puerto Rican girls on oral contraception -1960’s
 Men on estrogens had higher event rates – 1970’s
5
Could we have identified these harms earlier?
•Troglitazone (Rezulin) – removed from market in 2000
•Rofecoxib (Vioxx) -removed from market in 2004
•HRT
(Premarin and PremPro)-label change 2006
•Celecoxib
(Celebrex) – paper published 2005
•Telithromycin (Ketek) – major label change 2007
6
December 2004
7
“CELEBRATE :: CELEBREX”
How we statisticians help to save drugs
•We find safety boring
8
For efficacy we think hard about…
 Outcomes
 Population to study
 Protocol
 Analysis of primary outcome
 Control of Type I error rate
 Other outcomes
 Missing data
 Sensitivity analyses
9
How we statisticians save drugs
•Because we find safety boring….
 We don’t look at preclinical and early Phase data
 We don’t ask about
•Chemistry
•Biology
•What PK/PD studies show
•Safety part of analysis plan is an afterthought
10
How the statistical a-police protect drugs
•We test hypotheses
•Put events in correct body system
•Give precise definitions
•No data dredging
•Too many type 1 errors if we dredge
11
And we divide and…
•conquer
•obfuscate
12
e.g. Neuropathy
Event
T
C
Neuropathic pain
1
0
Neuropathy
1
0
Neuropathy NOS
5
2
Neuropathy peripheral
2
0
…
2
1
…
…
13
e.g. Neuropathy
Event
T
C
Parathesia
3
2
Parathesia NOS
4
0
Parathesia other
0
1
Peripheral motor neuropathy
6
0
Peripheral sensory neuropathy
3
2
…
…
14
True(ish) data from a coxib
Cardiac disorders
Respiratory
Vascular disorders
C
42
33
7
T
46
29
9
15
True(ish) data from a coxib
Cardiac disorders
•Angina
•Angina aggravated
•Angina unstable
•…
•Cardiac arrest
•Cardiac failure congest
•Coronary artery disease
•…
•Myocardial infarction
C
42
2
0
0
T
46
2
2
3
0
2
4
1
0
7
5
10
16
True(ish) data from a coxib
Respiratory
33
29
•Dyspnea
1
3
Vascular disorders
7
9
•Cerebral infarction
•Pulmonary embolism
•TIA
0
0
2
1
2
0
17
If you combined…
No. of people with at least one serious
thromboembolic event or evidence of
heart failure
Placebo
Coxib
16
27
18
Other ways to save drugs
Modified Daley’s Rule:
Censor early and often
19
e.g., Rofecoxib- short follow-up
20
Through 36 months
21
With denominators
(Bresalier et al. NEJM 2005 352:1092) (And see Adam Boyd’s poster!)
22
Known or suspected adverse events
•Monitor them
•Look at events, their (near) synonyms, labs
 Are they real?
 Are they too frequent?
23
Hierarchical multiplicity
•Think of biology
•Order hierarchy by decreasing
 Biological plausibility
 Objectivity
•Look for monotone decreasing hazard ratio
24
Which dose of celecoxib do you want?
25
APC Study (Placebo vs high dose)
Outcome
n
-----------------------------------------------CV death
6
+MI
19
+Stroke
26
+CHF
29
+Angina
34
+CV procedure 46
----------------------------------------------Other CV
62
26
Adenoma Prevention with Celecoxib (APC) Study
HR
CV death
5.1
+MI
3.8
+Stroke
3.4
+CHF
3.2
+Angina
2.1
+CV procedure 1.7
------------------------------------------------Other CV
1.1
27
APC Study
CV death
5.1
( 0.6, 43.2)
+MI
3.8
( 1.3, 11.4)
+Stroke
3.4
( 1.4, 8.3)
+CHF
3.2
( 1.4, 7.4)
+Angina
2.1
( 1.0, 4.3)
+CV procedure 1.7
( 1.0, 3.1)
-----------------------------------------Other CV
1.1
( 0.7, 1.8)
28
APC Study
CV death
5.1
( 0.6, 43.2)
0.14
+MI
3.8
( 1.3, 11.4)
0.015
+Stroke
3.4
( 1.4, 8.3)
0.007
+CHF
3.2
( 1.4, 7.4)
0.006
+Angina
2.1
( 1.0, 4.3)
0.05
+CV procedure 1.7
( 1.0, 3.1)
0.05
------------------------------------------------Other CV
1.1
( 0.7, 1.8)
0.7
Solomon (2006). Circulation 114:1028
29
Unknown harms: usual approach
•Respond by
 Agonizing
 Checking informed consent document
 Asking for more frequent looks
 Asking for more thorough analyses
•Worry about falsely discovered harm
30
Sentinel events
•Identify
•Follow in the next patients
•Invent formal statistical methods
31
Single sentinel event
•Childhood vaccine
•30 day follow-up for serious adverse events
•1 death occurred
•DSMB: did the vaccine cause the death?
32
Women’s Health Initiative
•Early in the trial, DSMB noted:
 Increase in stroke
 Increase in pulmonary embolism
 Increase in myocardial infarction
•Possible sentinel events
 Myocardial infarction
 The big meanies: stroke, PE, MI
33
Proposal
1.
2.
Identify sentinel event (or cluster or rate)
Monitor for subsequent occurrence(s)

Have reasonable power

Be statistically unbiased (exclude sentinel)

Type 1 error rate may be large (~0.2)
Lachenbruch, Wittes: 2007
34
Safety report sample –abnormal lab values
•Time
A
B
Total
•Point
[N= 150]
[N= 148]
[N= 298]
•_______________________________________________________________
SCREENING 0
0
0
•RANDOM
0
0
0
•WEEK 2
0
0
0
•WEEK 3
0
0
0
•WEEK 4
0
0
0
•WEEK 5
0
0
0
•WEEK 6
0
0
0
•WEEK 7
0
0
0
•WEEK 8
0
0
0
•___________________________________________________________________
35
But wait! You also get:
• Time
A
B
Total
• Point
[N= 150] [N= 148] [N= 298]
•_______________________________________________________________
• SCREENING 0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• RANDOM
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 2
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 3
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 4
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 5
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 6
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 7
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 8
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
•_______________________________________________________________
36
And 150 pages of where’s Waldo
• Time
A
B
Total
• Point
[N= 150]
[N= 148]
[N= 298]
•_______________________________________________________________
• SCREENING 0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• RANDOM
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 2
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 3
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 4
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 5
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 6
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 7
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• WEEK 8
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
• EARLY TERM
0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
UNSCHEDULED 0 (0.00 %) 0 (0.00 %) 0 (0.00 %)
•_______________________________________________________________
37
And if this isn’t enough…
•Change from baseline where missing is counted
as zero (change in HR=64????)
•Values out of temporal order
•Lots and lots of decimal places
•P-values to 3 and 4 significant digits
•Etc., etc. etc.
38
We need to change our habits
•Current statistical approach
 One variable at a time
 Template applied to all studies
 No wonder the docs don’t ask us to work with them!
•Simple change in attitude
 Safety parameters aren’t separable
 Focus first from biological insights and previous hints
 Then scan the other variables
 Then refocus
39
Conclusions
•Worry about multiplicity, but not too much
 Listen to Joe Heyse’s talk this afternoon
•Beware the censor-happy protocol and analysis
•Don’t be too much the statistician
•But don’t forget randomness
40