Bayesian Hierarchical Models for Detecting Safety Signals in

advertisement
Bayesian Hierarchical Models
for Detecting Safety Signals
in Clinical Trials
H. Amy Xia and Haijun Ma
Amgen, Inc.
MBSW 2009, Muncie, IN
March 20, 2009
Disclaimer: The views expressed in this presentation
represent personal views and do not necessarily represent
the views or practices of Amgen.
Outline
• Introduction
• A motivating example
• Bayesian Hierarchical Models
– Meta analysis of Adverse Events data from multiple
studies incorporating MedDRA structure
– Incorporate patient level data
– Effective graphics
• Closing Remarks
Three-Tier System for Analyzing
Adverse Events in Clinical Trials
• Tier 1: Pre-specified Detailed Analysis and
Hypothesis Testing
– Tier 1 AEs are events for which a hypothesis has
been defined
• Tier 2: Signal Detection among Common Events
– Tier 2 AEs are those that are not pre-specified and
“common”
• Tier 3: Descriptive Analysis of Infrequent AEs
– Tier 3 AEs are those that are not pre-specified and
infrequent
Gould 2002 & Mehrotra 2004
SPERT White Paper 2008
Multiplicity Issue in Detecting
Signals Is Challenging
• Detection of safety signals from routinely
collected, not pre-specified AE data in clinical
trials is a critical task in drug development
• Multiplicity issue in such a setting is a
challenging statistical problem
– Without multiplicity considerations, there is a potential
for an excess of false positive signals
– Traditional ways of adjusting for multiplicity such as
Bonferroni may lead to an excessive rate of false
negatives
– The challenge is to develop a procedure for flagging
safety signals which provides a proper balance
between ‘no adjustment’ versus ‘too much
adjustment’
Considerations Regarding Whether
Flagging an Event
•
•
•
•
Actual significance levels
Total number of types of AEs
Rates for those AEs not considered for flagging
Biologic relationships among various AEs
1st two are standard considerations in the
frequentist approach. The 2nd two are not,
but relevant in the Bayesian approach
-- Berry and Berry, 2004
Bayesian Work in Signal Detection
• Spontaneous adverse drug reaction
reports
– Gamma Poisson Shrinker (GPS) on FDA
AERS database (DuMouchel,1999)
– Bayesian Confidence Propagation Neural
Network (BCPNN) on WHO database (Bate,
et al. 1998)
• Clinical trial safety (AE) data
– Bayesian hierarchical mixture modeling (Berry
and Berry, 2004)
Meta Analysis
• Glass (1976)
Meta-analysis refers to a statistical analysis that
combines the results of some collection of
related studies to arrive a single conclusion to
the question at hand
• Meta-analysis based on
– aggregate patient data (APD meta-analysis)
– Individual patient data (IPD) meta-analysis
• Bayesian modeling is a natural choice to
incorporate the complex hierarchical structure of
the data
George Chi, H.M. James Hung,
Robert O’Neill (FDA CDER)
“Safety assessment is one area where
frequentist strategies have been less
applicable. Perhaps Bayesian approaches
in this area have more promise.”
(Pharmaceutical Report, 2002)
An Example
• Data from four double-blind placebo-controlled studies
on drug X. Study populations are similar.
• Sample sizes:
Study
Drug X
N
Drug X
Subj-yr
Placebo
N
Placebo
Subj-yr
Study A
57
28.25
55
19.02
Study B
486
104.75
166
34.93
Study C
390
85.44
193
40.97
Study D
312
68.78
306
65.91
• After converting all AEs into same MedDRA version,
reported AEs are coded to 464 PTs under 23 SOCs
and 233 HLTs
SYSTEM ORGAN CLASS
INFECTIONS AND INFESTATIONS
HIGH LEVEL TERM
HERPES VIRAL
INFECTIONS
INFECTIONS AND INFESTATIONS
PREFERRED TERM STUDY n_0 n_1 N_0
N_1
rt_0
rt_1
HERPES SIMPLEX
1
2
3
4
0
0
1
2
3
5
8
2
55
166
193
306
57
486
390
312
0.00
0.00
0.52
0.65
5.26
1.03
2.05
0.64
UPPER RESPIRATORY
TRACT INFECTIONS
SINUSITIS
1
2
3
4
2
6
0
4
8
19
8
11
55
166
193
306
57
486
390
312
3.64
3.61
0.00
1.31
14.04
3.91
2.05
3.53
INFECTIONS AND INFESTATIONS
UPPER RESPIRATORY
TRACT INFECTIONS
RHINITIS
1
2
3
4
0
0
1
0
0
0
4
0
55
166
193
306
57
486
390
312
0.00
0.00
0.52
0.00
0.00
0.00
1.03
0.00
SKIN AND SUBCUTANEOUS TISSUE
DISORDERS
URTICARIAS
URTICARIA
1
2
3
4
0
1
0
1
1
1
4
4
55
166
193
306
57
486
390
312
0.00
0.60
0.00
0.33
1.75
0.21
1.03
1.28
SKIN AND SUBCUTANEOUS TISSUE
DISORDERS
PURPURA AND RELATED
CONDITIONS
ECCHYMOSIS
1
2
3
4
0
0
0
0
0
4
5
3
55
166
193
306
57
486
390
312
0.00
0.00
0.00
0.00
0.00
0.82
1.28
0.96
INJURY, POISONING AND
PROCEDURAL COMPLICATIONS
NON-SITE SPECIFIC
INJURIES NEC
EXCORIATION
1
2
3
4
0
0
0
0
0
7
0
1
55
166
193
306
57
486
390
312
0.00
0.00
0.00
0.00
0.00
1.44
0.00
0.32
INJURY, POISONING AND
PROCEDURAL COMPLICATIONS
NON-SITE SPECIFIC
PROCEDURAL
COMPLICATIONS
PROCEDURAL PAIN
1
2
3
4
0
0
0
2
0
0
1
1
55
166
193
306
57
486
390
312
0.00
0.00
0.00
0.65
0.00
0.00
0.26
0.32
N_0: sample size in placebo arm; N_1: sample size in treatment arm
n_0: # subject with AE in placebo arm; n_1: # subject with AE in treatment arm
rt_0: subject incidence in placebo arm; rt_1: subject incidence in treatment arm
Proposed Bayesian Approach
• Hierarchical mixture models for aggregated
binary responses was constructed based on the
work by Berry & Berry(2004)
– Explore impact of using different MedDRA hierarchy
– Inclusion of study effects
– Further extended to a hierarchical Poisson mixture
model, to account for different exposure/follow-up
times between patients
• Individual patient level models are discussed
• Implemented the above models with available
software
– WinBUGS for model implementation
– S-Plus graphics for inference
MedDRA
• MedDRA (the Medical Dictionary for Regulatory
Activities Terminology) is a controlled vocabulary
widely used as a medical coding scheme.
• MedDRA Definition (MSSO):
– MedDRA is a clinically-validated international medical
terminology used by regulatory authorities and the
regulated biopharmaceutical industry. The
terminology is used through the entire regulatory
process, from pre-marketing to post-marketing, and
for data entry, retrieval, evaluation, and presentation.
https://eudract.emea.europa.eu/docs/medDRA/Introduction_To_MedDRA.ppt
MSSO: Introduction to MedDRA
http://www.fda.gov/cder/present/dia-699/dia628/index.htm
MedDRA and Pharmacovigilance - The Way Forward, 7/8/99
http://www.fda.gov/cder/present/dia-699/dia628/index.htm
MedDRA and Pharmacovigilance - The Way Forward, 7/8/99
Example of MedDRA Hierarchy
SOC = Respiratory, thoracic and
mediastinal disorders
HLGT = Respiratory tract
infections
SOC = Infections and
infestations
HLGT = Viral infectious
disorders
HLT =Viral upper respiratory
tract infections
HLT = Influenza viral
infections
PT = Influenza
https://eudract.emea.europa.eu/docs/medDRA/Introduction_To_MedDRA.ppt
MSSO: Introduction to MedDRA
Hierarchical Structure of MedDRA
• Bayesian hierarchical models allow for explicitly
modeling AEs with the existing coding structure
– AEs in the same SOC more likely to be similar within than across
SOCs.
– Allow for this possibility, but does not impose it, depending on
the actual data
– SOC tends to be too broad. HLT is more closely related to
medical concepts.
• In fact, clinical and safety people would (informally)
consider the similarity of the AEs, say, within SOCs when
they review AE tables
– For example, if differences in several CV events were observed,
then each would be more likely to be causal than if differences
came from medically unrelated areas (eg, skin, neurological,
thrombosis, cancer)
• Bayesian hierarchical modeling allows a scientific,
explicit, and more formal way to take it into consideration
Notations
• Study i=1,…I, SOC b=1,…B and PT
j=1,…kb
• Data: For AEibj
– Treatment group: Yibj incident events observed
in Nit patients with Tibj subjects’ exposure
– Control group: Xibj incident events observed in
Nic patients with Cibj subjects’ exposure
Bayesian Meta Analysis
Hierarchical Logistic Regression
Yibj ~ Binomial (N it , tibj ), X ibj ~ Binomial (N ic , cibj ),
where tibjand cibj are event rates for AE ibj in the treatment and
control groups, respective ly.
•Common treatment effect for same PT across
studies logit(c )   , logit(t )    
ibj
ibj
ibj
ibj
bj
ibj ~ N( bj ,  2,bj ) ,  bj ~ N( b ,  2b )
Mixture Prior for treatm ent effect log - OR :
Stage 1 prior : bj~  b (0)  (1   b ) N (  b ,  2b );
Stage 2 prior :  b ~ Beta ( p ,  p );
 p ~ exp( ) I ( p  1);
Stage 3 prior :
 p ~ exp(  ) I (  p  1)
Bayesian Meta Analysis
Hierarchical Logistic Regression (Cont.)
• Treatment effect with additive study effects:
logit(c
ibj
)  ibj , logit(t
ibj
)  ibj   bj   i Studyi
ibj ~ N( bj ,  2,bj ) ,  bj ~ N( b ,  2b )
 bj~  b (0)  (1   b ) N (  b ,  2b )
 i ~ N (0,0.01)
• A random treatment effect/multiplicative model:
logit(c
ibj
)  ibj , logit(t
ibj
)  ibj   ibj
ibj ~ N( bj ,  2,bj ) ,  bj ~ N( b ,  2b )
 ibj~ N(bj ,  2bj ) , bj~  b (0)  (1   b ) N (  b ,  2b )
Bayesian Meta Analysis
Hierarchical Logistic Regression (Cont.)
• Other priors
– Stage 1 λbj ~N(μλb , σ2λb );
– Stage 2 μλb~N(μλ0, σ2λ0 ); σ2λb ~IG(αλ, βλ);
μθb ~N(μθ0, σ2θ0 ); σ2θb ~IG(αθ, βθ)
- Stage 3 μλ0~N(μλ00 , σ2λ00 ); σ2λ0 ~IG(αλ00, βλ00)
μθ0~N(μθ00 , σ2θ00 ); σ2θ0 ~IG(αθ00, βθ00)
Hyperparameters μλ00, σ2λ00, αλ00, βλ00, μθ00,
σ2θ00, αθ00, βθ00, αλ, βλ , αθ, βθ are fixed constants
Inference
• AEbj is flagged if
– Pr( θbj > d*| Data) > p, where θbj is log-OR in
Binomial models and log-RR in Poisson
models.
– d* and p are all prespecified constants.
• Graphs are useful tools in deciphering
data and presenting results
Model Selection
• Deviance Information Criteria (DIC) was used to
compare models with same data
• Limited sensitivity analyses were done to check
the robustness of the models
• Different levels of MedDRA structures were used
– SOC/PT, HLT/PT and SOC/HLT/PT
• Treatment effect with additive study effects
model using SOC/PT structure was chosen
Bayesian Meta Analysis
Hierarchical Log-linear Regression
• Poisson models
– Adjust for different exposures in treatment and control
– Assume constant hazard over time
– Unless AEs are fairly common or follow up of studies are
quite unbalanced between treatment arms, usually are not
very different from Binomial models
Yibj~Pois(tibj Tibj ); Xbj~Pois(cibj Cibj)
where tibj and cibj are event rates, and Tibj and Cibj for AEibj are
total exposure times in the treatment and control groups,
respectively
log(cibj )=λibj; log(tibj )=λibj + θbj ,
Note that θbj =log(RRbj)
2.5
Fisher's Exact Test 2-sided P-values
2.0
SINUSITIS
1.5
FATIGUE
EXCORIATION
0.5
1.0
HERPES SIMPLEX
0.0
- log10 (Fisher exact p-value)
ECCHYMOSIS
GASTROINTESTINAL DISORDER...
INFECTIONS AND INFESTATIO...
SKIN AND SUBCUTANEOUS TIS...
NEOPLASMS BENIGN, MALIGNA...
GENERAL DISORDERS AND ADM...
PSYCHIATRIC DISORDERS
INJURY, POISONING AND PRO...
RESPIRATORY, THORACIC AND...
CARDIAC DISORDERS
MUSCULOSKELETAL AND CONNE...
REPRODUCTIVE SYSTEM AND B...
RENAL AND URINARY DISORDE...
EYE DISORDERS
INVESTIGATIONS
NERVOUS SYSTEM DISORDERS
METABOLISM AND NUTRITION ...
IMMUNE SYSTEM DISORDERS
EAR AND LABYRINTH DISORDE...
SURGICAL AND MEDICAL PROC...
VASCULAR DISORDERS
ENDOCRINE DISORDERS
HEPATOBILIARY DISORDERS
BLOOD AND LYMPHATIC SYSTE...
-0.01
0.0
0.01
Raw Risk Difference(%), Treatment - Placebo
0.02
Peto's Method
M-H TA CC Method
ECCHYMOSIS
FATIGUE
SINUSITIS
SINUSITIS
FATIGUE
HERPES SIMPLEX
HERPES SIMPLEX
ECCHYMOSIS
DYSPEPSIA
HEADACHE
INJECTION SITE BRUISING
INJECTION SITE BRUISING
HEADACHE
DYSPEPSIA
EXCORIATION
INFLUENZA
URTICARIA
NASOPHARYNGITIS
BASAL CELL CARCINOMA
DIZZINESS
TENSION HEADACHE
BRONCHITIS
DIZZINESS
URTICARIA
BRONCHITIS
PHARYNGITIS
INFLUENZA
UPPER RESPIRATORY TRACT INFECTION
GASTROENTERITIS VIRAL
TENSION HEADACHE
0
1
2
log Odds Ratio
3
0
1
2
log Odds Ratio
3
ECCHYMOSIS
0.4
0.6
BLOOD AND LYMPHATIC SYSTE...
CARDIAC DISORDERS
EAR AND LABYRINTH DISORDE...
ENDOCRINE DISORDERS
EYE DISORDERS
GASTROINTESTINAL DISORDER...
GENERAL DISORDERS AND ADM...
HEPATOBILIARY DISORDERS
IMMUNE SYSTEM DISORDERS
INFECTIONS AND INFESTATIO...
INJURY, POISONING AND PRO...
INVESTIGATIONS
METABOLISM AND NUTRITION ...
MUSCULOSKELETAL AND CONNE...
NEOPLASMS BENIGN, MALIGNA...
NERVOUS SYSTEM DISORDERS
PSYCHIATRIC DISORDERS
RENAL AND URINARY DISORDE...
REPRODUCTIVE SYSTEM AND B...
RESPIRATORY, THORACIC AND...
SKIN AND SUBCUTANEOUS TIS...
SURGICAL AND MEDICAL PROC...
VASCULAR DISORDERS
URTICARIA
0.2
SINUSITIS
HERPES SIMPLEX
BASAL CELL CARCINOMA
FATIGUEBLOOD PRESSURE INCREASED
TENSION HEADACHE
DIZZINESS
FOLLICULITIS
DYSPEPSIA
ANXIETY
PHARYNGITIS
MIGRAINE
HYPERSENSITIVITY
DERMAL
CYST
NERVE
COMPRESSION
CAROTID
ARTERY
STENOSIS
BLOOD
CHOLESTEROL
INCREASED
FLUSHING
LIVER
DISORDER
MEMORY
IMPAIRMENT
GASTROENTERITIS
VIRAL
HYPERTONIA
DIZZINESS
POSTURAL
INSUFFICIENCY
DISTURBANCE
IN SYNDROME
ATTENTION
HEADACHE
SENSORY
DISTURBANCE
NEURALGIA
SYNCOPE
CEREBROVASCULAR
FACIAL
FURUNCLE
PALSY
ACCIDENT
SOMNOLENCE
HYPERSOMNIA
CYSTITIS
VAGINAL
INFECTION
DYSGEUSIA
CARPAL
TUNNEL
BRONCHITIS
RESTLESS
LEGS
ENDODONTIC
PROCEDURE
ACNE
THROAT
IRRITATION
SKIN
INFECTION
PARAESTHESIA
VIRAL
UPPER
RESPIRATORY
TRACT INFECTION
LETHARGY
SCIATICA
SEBORRHOEIC
KERATOSIS
ECZEMA
BURNING
SENSATION
EAR
WEIGHT
INFECTION
DECREASED
INFECTED
INSECT
BITE
BRONCHITIS
VERTIGO
BLOOD
GLUCOSE
CHRONIC
INCREASED
INFLUENZA
NIGHT
SWEATS
SYNCOPE
LABYRINTHITIS
VASOVAGAL
ERECTILE
DYSFUNCTION
ONYCHOMYCOSIS
WEIGHT
INCREASED
BODY
TEMPERATURE
INCREASED
DRY
EYE
HERPES
RHINORRHOEA
VIRUS
INFECTION
MALAISE
CELLULITIS
CONJUNCTIVITIS
STAPHYLOCOCCAL
HYPOAESTHESIA
FUNGAL
EXCORIATION
INFECTION
VIRAL
NASAL
INFECTION
CONGESTION
LARYNGITIS
VERTIGO
POSITIONAL
ONYCHOMADESIS
TOOTH
SKIN
SKIN
LESION
PAPILLOMA
ABSCESS
ABDOMINAL
INFECTION
GINGIVAL
INFECTION
DIVERTICULITIS
TINEA
VERSICOLOUR
STRESS
SINUS
HAEMATURIA
HEADACHE
GUTTATE
SIALOADENITIS
SKIN
HAEMORRHAGE
PSORIASIS
BIPOLAR
TINNITUS
IINCONTINENCE
DISORDER
ALCOHOLISM
INFECTED
URINARY
PREMATURE
CRURIS
INCONTINENCE
SEBACEOUS
EJACULATION
CYST
RHINITIS
MASTITIS
PULPITIS
IRRITABILITY
DENTAL
CAT
ACUTE
POSTOPERATIVE
HYPERCHOLESTEROLAEMIA
STRESS
SCRATCH
SINUSITIS
DISEASE
WOUND
INFECTION
ARTERIOSCLEROSIS
EPISTAXIS
CORONARY
VAGINAL
APPENDICITIS
INFECTIOUS
DYSPHONIA
CANDIDIASIS
MONONUCLEOSIS
PUNCTURE
AGITATION
ATTENTION
SITE
DEFICIT/HYPERACTIVITY
INFECTION
DISORDER
GENITAL
EYELID
INFECTION
INFECTION
FEMALE
TONSILLITIS
CANDIDIASIS
ERYSIPELAS
PNEUMONIA
ARRHYTHMIA
NERVOUSNESS
TENDONITIS
TINEA
ACTINIC
INFECTION
KERATOSIS
RASH
PUSTULAR
INJECTION
SITE
BRUISING
INGROWING
NAIL
DYSPHORIA
BUNDLE
BRANCH
BLOCK
LEFT
SUBCUTANEOUS
ABSCESS
ORAL
TRIGONITIS
FALL
PUSTULE
CERVICITIS
CONDYLOMA
OTITIS
FIBROUS
EXTERNA
HISTIOCYTOMA
ACUMINATUM
URINARY
CERVICAL
RETENTION
DYSPLASIA
LIPOMA
CALCULUS
DERMATITIS
BLADDER
CONTACT
EAR
SQUAMOUS
SLEEP
DISCOMFORT
APNOEA
CELL
SYNDROME
CARCINOMA
OFARTERY
SKIN
LYMPHOEDEMA
SINUS
CONGESTION
PROSTATE
CANCER
LOWER
RESPIRATORY
TRACT
INFECTION
INJECTION
POLLAKIURIA
ALLERGIC
COUGH
SITE
HAEMATOMA
INTERTRIGO
LIBIDO
DECREASED
INTERVERTEBRAL
SNEEZING
DYSPNOEA
DISC
DEGENERATION
SNORING
NASOPHARYNGITIS
TOOTH
DYSPNOEA
CONJUNCTIVAL
INFECTION
EXERTIONAL
HAEMORRHAGE
ABDOMINAL
FOOT
FRACTURE
PAIN
UPPER
GASTROINTESTINAL
INFLAMMATION
SHOULDER
PAIN
NEPHROLITHIASIS
HYPERVENTILATION
VAGINAL
RHINITIS
ALLERGIC
LACERATION
VULVOVAGINAL
RHONCHI
DRYNESS
HAND
OTITIS
MEDIA
JOINT
DRY
EYELID
POST-TRAUMATIC
MOUTH
EFFUSION
OEDEMA
PAIN
PRURITUS
SEASONAL
FLATULENCE
ALLERGY
BENIGN
ABDOMINAL
PROSTATIC
DISTENSION
HYPERPLASIA
EYE
INFLAMMATION
OVARIAN
CATARACT
HOT
FOOD
FLUSH
POISONING
CYST
RUPTURED
DIPLOPIA
GASTROOESOPHAGEAL
REFLUX DISEASE
STOMACH
DISCOMFORT
HYPERTENSION
GASTROINTESTINAL
HYPERMOTILITY
PAIN
GINGIVAL
THERMAL
BLEEDING
BURN
DEPRESSION
INSOMNIA
DISORDER
ABDOMINAL
TENDERNESS
ROAD
SCAPULA
TRAFFIC
FRACTURE
ACCIDENT
TOOTH
FRACTURE
HEAT
STROKE
MYALGIA
TOOTHACHE
INJURY
ANIMAL
BITE
WOUND
BACK
PAIN
TENDON
INJURY
CONTUSION
DIARRHOEA
NAUSEA
0.0
P(OR > 1|Data)
0.8
1.0
Posterior Summary of Bayesian Hierarchical Meta-Analysis Model
-3
-2
-1
0
log-OR posterior means
1
2
Inferences of Binomial Hierarchical
Model with Mixture Prior
SOC
GENERAL
DISORDERS AND
ADMINISTRATION
SITE
INFECTIONS AND
INFESTATIONS
PT
FATIGUE
HERPES
SIMPLEX
INFECTIONS AND
INFESTATIONS
SINUSITIS
SKIN AND
SUBCUTANEOUS
TISSUE
DISORDERS
ECCHYMOSIS
Two
sided
Post Post Post
Post
Post
Fisher's Prob Prob Prob
Prob
Prob
Exact
OR=1 OR>1 OR>1.1 OR>1.2 OR>2
0.019
0.82
0.18
0.18
0.18
0.10
0.039
0.75
0.25
0.25
0.25
0.21
0.012
0.72
0.27
0.27
0.27
0.19
0.005
0.11
0.89
0.89
0.89
0.89
Bayesian Patient Level Models
• IPD models to include within patient correlation and
patient level factors while incorporating MedDRA coding
hierarchy
• Data from one study is used:618 subject, 207 unique
AEs, N = 127926
Ykbj ~ Bernoulli (tkbj ), X kbj ~ Bernoulli (ckbj ),


logit(c kbj )  bj   Zk   k , logit(t kbj )  bj   bj   Zk   k

where Zk is the set of covariates for subject k

 2
 ~ MVN(0,  b I),  k ~ Normal (0,  2 )
Simulation Study
• Simulation scheme:
– Randomly assign subjects to treatment or placebo to create a
“null” scenario
– Adverse events within subject remain unchanged to maintain the
SOC/PT hierarchy
– 1000 simulated datasets
• Family-wise error rates (also FDR in this case) of
Fisher’s exact text unadjusted for multiplicity and
Poisson regression with mixture prior are compared
• Percentage of simulated datasets yielding Y=0, 1, 2 or
≥3 incorrectly flagged adverse events out of 464 PTs are
also compared
Distribution of Y (%)
FWER/
FDR
1
2
>=3
Non-adjusted 2sided Fisher’s
exact test
2-sided test, p-value <=0.05
99.1
2.7
5.4
91
Non-adjusted 1sided Fisher’s
exact test
p-value<=0.05
90.1
22.7
26.9
40.5
Bayes
Hierarchical
Poisson Model *
c=1, p=0.7
10.2
7.5
2.0
0.7
c=1, p=0.8
6.1
4.6
1.3
0.2
c=1, p=0.9
2.8
2.7
0.1
0.0
c=1.2, p=0.7
10.2
7.6
1.9
0.7
c=1.2, p=0.8
6.0
4.6
1.2
0.2
c=1.2, p=0.9
2.7
2.6
0.1
0.0
c=2, p=0.7
7.0
5.3
1.5
0.2
c=2, p=0.8
3.7
3.3
0.4
0.0
c=2, p=0.9
1.3
1.2
0.1
0.0
RR≠1, p=0.7
13.8
10.3
2.5
1.0
RR≠1, p=0.8
7.9
6.3
1.4
0.2
RR≠1, p=0.9
3.6
3.5
0.1
0.0
*
Pr( RR bj  c | Data )  p
Simulation Study
• 464 independent tests with alpha=0.05 would
yield in average about 23 signals and have
FDR=1 if no multiplicity is adjusted for.
• Correlation of the AE data reduced the error rate
in our simulation study
• But the FDR is still as high as 99.1%. For 91%
cases there are at least 3 falsely identified
signals.
• The FWERs/FDRs for all Bayes model results
are much lower and acceptable.
Closing Remarks
• Current traditional approach of flagging
routinely collected AEs based on
unadjusted p-values or CIs can result in
excessive false positive signals
– As a result, it can cause undue concern for
approval/labeling/post marketing commitment
• Commonly used meta-analysis methods
for aggregated binary outcome (OR)
– Peto’s method is not recommended for
severely unbalanced studies or common
events unless treatment effects are small
– MH method: needs continuity correction
Closing Remarks (Cont.)
• Bayesian meta-analysis hierarchical mixture modeling
provides a useful tool to analyze data from multiple
studies and address multiplicity
– Allows for explicitly modeling AEs with the existing MedDRA coding
structure
– Use a mixture prior by allowing a point mass on equality of the
treatment and control rates
– Study differences can be accounted for
– No need to add continuity correction. Double zero studies are included.
– For less common AEs and studies without a great amount of follow-up
variation between treatment groups, inferences from Poisson regression
and logistic regression models are very similar
• Computation for signal detection using IPD is
challenging
• Graphics are effective in displaying flagged signals
Future Work
• More sensitivity analysis on the performance of the
models
• Further simulation study on type II error and operating
characteristics of Bayesian models
• Zero-inflated Poisson model might be a good approach
for relatively healthy population
• Incorporating severity information of AEs
• Multi-axial structure of MedDRA coding system
• The field of clinical trial signal detection is still in its
infancy
– More research and practice are needed
– Statisticians need to work with clinicians/safety scientists closely
to further advance this field
References
•
•
•
•
•
•
•
•
Bate A, Lindquist M, Edwards, IR, Olsson S, Orre R, Lansner A, and De
Freitas RM (1998). A Bayesian neural network method for adverse drug
reaction signal detection. Eur J Clin Pharmacol 54:315-321
Berry S and Berry D (2004) Accounting for multiplicities in assessing drug
safety: a three-level hierarchical mixture model. Biometrics, 60: 418-426
Chi G, Hung HMJ, and O’Neill R (2002). Some comments on “Adaptive
Trials and Bayesian Statistics in Drug Development” by Don Berry. In
Pharmaceutical Report, Vol 9, 1-11
Crowe B, Xia A, Watson D, Shi H, Lin S, Kuebler J, Berlin J, et al. (2008).
Recommendations for Safety Planning, Data Collection, Evaluation and
Reporting During Drug, Biologic and Vaccine Development: A Report of the
PhRMA Safety Planning, Evaluation and Reporting Team (SPERT).
Manuscript in preparation.
DuMouchel W (1999). Bayesian data mining in large frequency tables, with
an application to the FDA Spontaneous Reporting System (with discussion).
The American Statistician 53:177-202
Gould AL. Drug safety evaluation in and after clinical trials. Deming
Conference, Atlantic City, 3 December 2002
Mehrotra, DV and Heyse, JF (2004). Multiplicity considerations in clinical
safety analysis. Statistical Methods in Medical Research 13, 227-238
Spiegelhalter DJ, Best NG, Carlin BP and van der Linde A (2002) Bayesian
measures of model complexity and fit (with discussion). J. Roy. Statist. Soc.
B. 64, 583-640.
Thank You!
Back-up Slides
50
0
10
20
30
40
Min. 0
1st Qu. 0
Median 0
Mean 0.003
3rd Qu. 0.003
Max. 0.333
0.00
0.10
0.20
0.30
80
Incidence Rate of Treatment Arm
0
20
40
60
Min. 0
1st Qu. 0
Median 0
Mean 0.003
3rd Qu. 0
Max. 0.164
0.00
0.05
0.10
0.15
Incidence Rate of Placebo Arm
Bayesian Hierarchical Model for AE
(Berry & Berry 2004)
• PT level assumptions:
 ij ~ N ( i ,  2 )
 ij ~  i I [0]  (1   i ) N ( i ,  2 )
• SOC level
assumptions:
i
 i
i
~ N (   0 , T2 )
~ N (  0 , T2 )
~ Beta (  ,   )
• Global
assumptions:
 0
 0
~
N (  00 , T200 )
~
N (  00 , T200 )
T2
~
IG ( T ,  T )
T2
~
IG ( T ,  T )
 2
 2


~
IG (  ,   )
~
IG (  ,   )
~
Exp( ) I (   1)
~
Exp( ) I (   1)
SOC: Injury, Poisoning and
Procedural Complications
Preferred Term
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Alcohol Poisoning
Animal Bite
Arthropod Bite
Back Injury
Cartilage Injury
Chemical Eye Injury
Closed Head Injury
Concussion
Contrast Media Reaction
Contusion
Epicondylitis
Excoriation
Eye Injury
Fall
Foot Fracture
Foreign Body in Eye
Hand Fracture
Heat Exhaustion
Heat Stroke
Hip Fracture
Injury Corneal
Jaw Fracture
Joint Injury
Joint Sprain
Ligament Injury
Ctrl
r(%)
0.00
0.14
0.56
0.69
0.14
0.14
0.00
0.28
0.14
0.83
0.14
0.00
0.14
0.00
0.00
0.14
0.00
0.14
0.00
0.00
0.14
0.14
0.00
0.28
0.14
Trt
r(%)
0.08
0.24
0.24
0.24
0.00
0.00
0.08
0.00
0.00
0.80
0.08
0.64
0.08
0.32
0.32
0.00
0.24
0.00
0.08
0.08
0.00
0.08
0.08
0.16
0.08
diff
(Trt-Ctrl )
0.080
0.102
-0.315
-0.453
-0.139
-0.139
0.080
-0.278
-0.139
-0.030
-0.059
0.643
-0.059
0.321
0.321
-0.139
0.241
-0.139
0.080
0.080
-0.139
-0.059
0.080
-0.117
-0.059
Preferred Term
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
Limb Injury
Mouth Injury
Muscle Injury
Muscle Strain
Neck Injury
Pain Trauma Activated
Pneumothorax Traumatic
Post Procedural Complication
Post Procedural Pain
Procedural Pain
Rib Fracture
Road Traffic Accident
Scapula Fracture
Skeletal Injury
Skin Laceration
Splinter
Sunburn
Tendon Injury
Thermal Burn
Tooth Fracture
Tooth Injury
Traumatic Haematoma
Wound
Wrist Fracture
Ctrl
r(%)
0.14
0.00
0.00
0.69
0.14
0.14
0.00
0.14
0.00
0.28
0.14
0.00
0.00
0.42
1.11
0.14
0.28
0.00
0.00
0.00
0.00
0.28
0.14
0.14
Trt
r(%)
0.16
0.08
0.08
0.72
0.08
0.40
0.08
0.00
0.08
0.08
0.08
0.16
0.08
0.00
0.72
0.08
0.24
0.08
0.16
0.08
0.16
0.08
0.24
0.00
diff
(Trt-Ctrl )
0.022
0.080
0.080
0.028
-0.059
0.263
0.080
-0.139
0.080
-0.197
-0.059
0.161
0.080
-0.417
-0.388
-0.059
-0.037
0.080
0.161
0.080
0.161
-0.197
0.102
-0.139
Download