The Role of the Statistician in the
Evolving Preclinical Drug Safety
Environment
Steven Bailey
Pfizer, Drug Safety R&D Statistics
Pfizer Internal Use
Introduction
Preclinical Drug Safety testing
• integral part of drug development for decades
• Historically more emphasis on running studies
needed for regulatory approval
Recently scope of work in Drug Safety has been
evolving
• Goal of identifying issues earlier
Pfizer Internal Use
2
‘Conclusions’
Drug Safety getting involved in new areas
Interesting new areas of support
Openness to statistical support
To date, our response have been underwhelming
- resource limitations
Pfizer Internal Use
3
The past
Regulatory studies required for IND, NDA submissions
Toxicology
Reproductive Toxicology
Carcinogenicity
Genetic Toxicology
Safety Pharmacology
Juvenile Animal Toxicity
Generally standard study design
Standardized statistical methods
Challenges: generally basic from a statistical point of view
haven’t changed much over time
Pfizer Internal Use
4
In 2004 – ‘New Topics’
Session at the Midwest Biopharm Statistics Workshop
Title: Statistical Analysis of New Studies Required in
Preclinical Safety Testing
Talks:
• New methodologies for QT interval prolongation adjustments
in preclinical safety pharmacology studies
• Design and analysis of CNS/FOB studies in preclinical safety
pharmacology testing
• Design and analysis of juvenile animal toxicity studies in
support of pediatric drug products
• Statistical aspects of auditory startle experiments in
behavioral toxicity studies
Pfizer Internal Use
5
Evolving Development Landscape
Currently:
5/5000 compounds progress to clinical trials
1/5000 compounds become approved drugs
Average time for development to approval is 12 years
- MedicineNet.com
Avg. cost for R&D for each drug approved is $4 billion
- Forbes, 2012
Failure rate increasing, and development cost increasing over time
More conservative, safety conscious FDA
In preclinical Drug Safety, more types of testing required
Pfizer Internal Use
6
Evolving Development Landscape
New development paradigms
Fail early
- Identify problems early so compounds killed early
Personalized medicine
Biopharmaceuticals
Creates new types of development activities,
and new opportunities for statistical support
Pfizer Internal Use
7
New Areas of Support
Quantitative Pathology
- Imaging
- Stereology
- Comparison of Methods
Efficacy Model Development
Safety Biomarker Development and Qualification
miRNA biomarkers
Pfizer Internal Use
8
Quantitative Pathology - Imaging
(Automated) imaging data is increasingly used to
assess pathology
– Less subjective
– Once processes is set up (staining, equipment
learning), is less expensive.
– Process is time consuming, but equipment can be set
and left to operate automatically
Pfizer Internal Use
9
Example: Quantitative Immunohistochemistry
for Nephrin
Objective: Quantify nephrin expression by QIHC on
sections of kidney
• Dosed animals vs. controls
• IHC for nephrin on sections of kidney using Ventana
Discovery XT automated immuno-staining platform
• Nanozoomer slide scanner used to capture images
• PE Vectra image analysis software used to measure
area in the region of interest (glomerular tuft) that is
occupied by chromogen (IHC%)
Pfizer Internal Use
10
QIHC: Nephrin Expression
A
%
Glomerular Tuft Expressing Nephrin
B
1525F
400x
Nephrin+ area /
glom tuft area (%)
80
Region
of
interest
(ROI)
60
40
1525F
4525F
20
4027M
0
Vehicle
C
Vehicle
22.5 mpk/cycle 50.0 mpk/cycle
4525F
50.0 mpk/cycle
400x
D
4027M
400x
50.0 mpk/cycle
Ben Wei, Yutian Zhan and Shawn O’Neil
Quantitative Pathology - Imaging
Example Data
sample
Nephrin
IHC (µm²)
Area of All
Tufts (µm²)
Nephrin
IHC Area %
1525
142482
306656
46.46
4027
135534
493650
27.46
4525
117349
306323
38.31
Pfizer Internal Use
12
Image Analysis Quantification of CD25+/FoxP3+
Pseudocoloring via Nuance Multispectral Camera
CD25 / FoxP3
CD25 / FoxP3
CD25+/FoxP3+
CD25+
FoxP3+
CD25+/FoxP3+
Pfizer Confidential │ 13
Quantitative Pathology - Imaging
Imaging technologies can be used to measure:
– Area
– Cell counts
• Cell area
• Cell nuclei
– Cells exhibiting specific markers
Pfizer Internal Use
14
Quantitative Pathology - Stereology
Stereology – the study of 3 dimensional structures
using two-dimensional cross sections
- volume
- surface area
- length
- number
Techniques require a few 'representative' plane
sections, then statistically extrapolate to threedimensional object
Pfizer Internal Use
15
Quantitative Pathology - Stereology
Very old field – can be traced back to the Buffon’s
needle problem, posed by Geoges-Louis Leclerc,
Comte de Buffon in 1777
Suppose we have a floor made
of parallel strips of wood, each
the same width, and we drop a
needle onto the floor. What is
the probability that the needle
will lie across a line between
two strips?
Image Created By: Wolfram MathWorld
Pfizer Internal Use
16
Quantitative Pathology - Stereology
In the past – heavily reliant on assumptions
Model based
Assumptions regarding structure (shape and
homogeneity) of the 3-dimensional object
‘New’ stereology – “Design-Based Stereology”
- assumption and model-free
- unbiased
- also called ‘Assumption free stereology’
Pfizer Internal Use
17
Example – Count of neurons in ganglia
21
1
Sections 1-20
(360 µm)
41
81
61
101
18 µm thickness
Total # neurons counted: 178
Avg thickness = 10.8251 µm
6
5
6
7
8 µm Dissector height
(fixed)
(1 µm guard zones)
ASF = 0.025 (2.5%)
TSF = 8/10.8251 (0.739)
121
SSF = 24/480 (0.05)
Estimated Total Count
= # counted x 1/SSF
x 1/TSF x 1/ASF
= 178 x 1/0.05 x
1/0.739 x 1/0.025
= 178 x 1082
= 192,596
Stereology - Uses
Nerves
- Neurons within ganglia
count, volume
Kidney
- measurement of glomeruli in kidney
volume, number
- measurement of mesangial matrix within
glomeruli
volume, percent of tissue
Lungs
- Alveoli
volume, surface area
Pfizer Internal Use
19
Stereology - Issues
Sampling plan
- Define reference space
- Design a ‘systematic-random’ sampling plan
-
removes subjective choices and options
independent of properties of the tissue
efficient, unbiased
Multiple layers of sampling
- Variation in each layer
- Sampling plan dependent on precision needed
Model assumptions – introduce bias
- modeling of the geometry of the structures
- popular in materials science-simple geometry
Replication of results impossible due to sampling plans
Pfizer Internal Use
20
Imaging and Stereology – Statistical Issues
Appropriate definition of reference space
Appropriate sampling strategies
Quantification of variability
- method variability
- biological variability
Appropriate endpoint
- not always correct
- stereology example: percentage mesangial
matrix/glomeruli vs. total area mesangial matrix
Pfizer Internal Use
21
Imaging and Stereology – Statistical Issues
Sample size/power calculations
Reproducibility
- method (staining, sampling, observer effect, etc.)
Appropriate statistical methodology
nested models
repeated measures
parametric or nonparametric approaches
unequal variance
Pfizer Internal Use
22
Quantitative Pathology – Comparison of Methods
-
Factor of imaging, stereology, and histopathology is
cost
-
Example (imaging): Comparison of Cri Vectra vs.
Stereologer in measuring glomerular area
-
Example (imaging vs. histopathology):
histopathological scoring for severity of spinal cord
demyelination and quantitative image analysis for
myelin content
Pfizer Internal Use
23
Quantitative Pathology – Comparison of Methodsl
There was a statistically significant inverse correlation between
histopathological scoring for severity of spinal cord demyelination
and quantitative image analysis for myelin content based on Luxol
Fast Blue histochemical staining at all levels of spinal cord
evaluated (p<0.05).
p < 0 .0 5
H is t o p a t h o lo g y :
d e m y l e in a t io n s c o r e
5
L u m b a r S p in a l C o r d
T h o r a c ic S p in a l C o r d
C e r v ic a l S p in a l C o r d
p < 0 .0 0 0 1
5
4
4
3
3
p < 0 .0 0 0 1
6
5
4
3
2
2
2
1
1
0
0
0
20
40
60
80
L F B /P A S Q IH C - C e rv ic a l
100
1
0
0
20
60
80
L F B /P A S Q IH C - T h o ra c ic
*Statistical analysis: Linear regression .
Pfizer Confidential │ 24
40
100
0
20
40
60
80
L F B /P A S Q IH C - L u m b a r
100
Quantitative Pathology – Comparison of Methods
Now:
- Regression / p-value / R2
-
Correlation
Better:
- Bland Altman
-
Lin L, Hedayat A, Wu W, (2012), Statistical Tools for
Measuring Agreement, Springer
-
Hshieh and Ng, Assessing agreement: a graphical
approach, poster presentation at 2015 JSM
-
Simulations – do the two methods arrive at the same
conclusion?
Pfizer Internal Use
25
Efficacy Model Development
Example: Mouse Collagen Induced Arthritis Model
• Collagen Induced Arthritis (CIA) model used to test
efficacy of new therapeutics
– Arthritis induced using monoclonal antibodies
– Used to simulate rheumatoid arthritis
– Compound administered, joints examined
pathologically for inflammation
– Compound screening completed in a short time
• Through the use of historical data characterization and
simulations, were able to determine that only lost 2% in
power (98% power to 96%) despite reducing the
number of pathology slides read by 50%.
Pfizer Internal Use
26
Safety Biomarker Development
• Develop Safety Biomarkers so we can:
- Screen compounds faster at less cost
- Less compound needed
- Fewer animals
- Faster
- Possibly go into clinical trials by closely monitoring
compounds that have side effects
Pfizer Internal Use
27
Safety Biomarker Development
‘Standard’ Development Paradigm
– Study with 2 groups – control and treated
Treated @ dose expected to cause 100% response
– Treated group is dosed with an article that is known to
cause effect (eg, kidney damage)
– Perform simple statistics (eg, t-test) on every
parameter of potential interest
Pfizer Internal Use
28
Safety Biomarker Development
Issues
– Correlation does not imply causality
– Need biomarkers specific to injury under study, but
may find biomarkers indicative of non-specific injury
– No ability to develop biomarkers that depend on
combinations of parameters
– How to choose among set of candidates with significant
p-values
– How to handle transient effects
Pfizer Internal Use
29
Safety Biomarker Development
Statistical Issues
– Should be comparing affected vs. non-affected
animals, not dosage groups
– Design: prefer additional dosage groups where only a
percentage of animals respond
– Sample size / lack of reproducibility
Pfizer Internal Use
30
Safety Biomarker Development
Statistical tools
-
Predictive modeling
-
Classification and Regression Trees (CART)
-
Random Forest
-
Logistic Regression/Linear Discriminant Analysis
-
Allow models using multiple parameters
-
Allow ranking of models
Reference: Johnson, Kjell and Kuhn, Max, (2013), Applied
Predictive Modeling, Springer.
Pfizer Internal Use
31
microRNAs (miRNAs)
 MicroRNAs (miRNAs) are non-coding, single-stranded, short
(∼22-nucleotide) regulatory RNAs
 Estimated ∼ 1000 miRNA genes in human; ~1% of human
genome
 Wide range of expression: Few to 100,000 copies per cell
 miRNA are highly conserved and often tissue specific
Landgraf et al., Cell,
2007
miRNA research trends
The growing
awareness of the
importance of miRNAs
has generated intense
activity in the
biomedical research
community.
Vergoulis T et al. 20111
 miRNAs in human body fluids (blood, urine, saliva, feces) are
non-invasive markers for disease.
 miRNA expression patterns are tissue specific
 miRNA expression can easily be detected
 miRNAs are conserved across multiple species
 miRNAs may be used to localize the site of injury
 miRNAs may reliably track progression of injury/recovery
Issues with analysis of miRNAs
Normalization
Needed to account for sample preparation
and other technical artifacts
Missing Values
High percentage of missing values. Missing
values can be due to:
1.Technical issues - missing at random
2.Value greater than machine sensitivity
3.Value unreliable (greater than a threshold,
eg 32, that the scientist sets) – not missing
at random
Multiplicity
Depends on experiment. Cards we use have
384 wells. For screening, we generally
collect data on 384 or 768 miRNAs.
Statistical Techniques
Normalization
1.Quantile normalization
2.LOESS normalization
3.MiR-Adaptive normalization (Zhao and Zhu, Poster at
Joint Statistical Meetings, 2015)
1.Many miRNAs have no analysis due to prevalence of
missing values (eg, in a recent investigation, only 110 of
384 miRNAs had at least 3 observations per group.
2.Technical issues are missing at random; can be ignored
Missing Values
3.Values greater than machine sensitivity or with
unreliable values are not missing at random. Currently
are being ignored and only captured in biological
interpretation through scan of missing values pattern.
Multiplicity
What are appropriate adjustment procedures. Use of
False Discovery Rate adjustment prevalent.
To Get Involved - What’s Needed
Willingness to do ‘Business Development’
Interest in learning subject area(s)
- Can be extremely time-consuming
Ability to think ‘outside the box’
- Not just about statistics
STRONG consulting skills
- ability to work with scientists without
overwhelming them
Resources – i.e., time
Pfizer Internal Use
36
Acknowlegements
Investigative Pathology
Shawn O’Neil
Yutian Zhan
Pathology
Scott Schelling
Tim Labranche
Bruce Homer
Safety Biomarkers
Shashi Ramaiah
Rounak Nassirpour
Statistics
David Potter
Pfizer Internal Use
37