REGRESSION TO THE MEAN AS A
TOOL TO UNDERSTAND OUT-OFSPECIFICATION (OOS) RESULTS
Jyh-Ming Shoung and Stan Altan
jshoung@its.jnj.com
saltan@its.jnj.com
Midwest Biopharmaceutical Statistics Workshop
Session on Statistical Mitigation of OOS
and OOT in a QbD World
1
May 24, 2011
OUTLINE
o
Introduction

o
o
o
OOS Guidance for Industry (October 2006) –
the current regulatory view
Out-of-Specification (OOS) results viewed
through the prism of Regression to the
Mean(RtM)
OOS Case Study – applying RtM
principles
Recommendations
2
INTRODUCTION
THE CURRENT REGULATORY VIEW
OOS Guidance for Industry (October 2006)
 Prescriptive approach to investigations addressing
the question of root cause
 Is there a flaw in the product or process?

Statistical references ;



Outlier testing - write it into the SOP
 It should never be assumed that the reason for an outlier is
error in the testing procedure, rather than inherent
variability in the sample being tested.
Retesting (of the original sample)
 The number of retests to be performed on a sample should
be specified in advance by the firm in the SOP…based on
scientifically sound, supportable principles. The number
should not be adjusted depending on the results obtained.
Re-sampling (new sample) appears to be discouraged, the
original sample is supposed to be large enough to permit retesting
3
OOS GUIDANCE – WHAT IS THE ISSUE?
o
Ignores statistical considerations

o
o
“In cases where … some of the individual results are
OOS, some are within specification, ... the passing
results are no more likely to represent the true value
for the sample than the OOS results…and treat the
reportable average of these values as an OOS result,
even if that average is within specification. … every
individual application of the official test should be
expected to produce a result that meets specifications.”
This language is in conflict with the basic principle
that the essential quality statement is captured in
the ‘average’ and variability between dose units is
captured in a separate protocol
An operational root cause will not be found for
some or many OOSs, and this does not necessarily
indicate a product or process flaw
4
QUALITY BY DESIGN –AN OPPORTUNITY TO
REVISIT OOS MANAGEMENT?
o
Improved Process Knowledge
Systematic Development approach
 Formulation understanding
 Process understanding
 Packaging understanding

o
o
o
Identify Critical to Quality Attributes
(CQAs)
Process Understanding as input to risk
management - Quality risk management
Control what is critical - Advanced control
strategy
5
A PROCESS IS WELL UNDERSTOOD
WHEN…
o
All critical sources of variability are
identified and explained
 OOSs
o
WILL happen occasionally
Variability is managed by the process
 How
do we integrate occurrence of OOS’s in a
QbD framework
o
Product quality attributes can be
accurately and reliably predicted over the
design space
 Basic
Principle : Occurrence of isolated OOSs
does not contradict quality in product
manufacture
6
Tim Schofield (Specs Workshop, 2005)
o
o
o
Multiple repeated stability determinations of the same lot on
stability necessarily lead to an increasing chance of an OOS
Such OOSs require context and mitigation
Statistical mitigation is completely consistent with QbD concepts
of advanced risk management and control strategies
7
OOS’S VIEWED THROUGH THE PRISM OF
REGRESSION TO THE MEAN IN A QbD CONTEXT
o
o
o
o
An isolated OOS can be thought of as a random
outcome related to incipient sources of variability
and hence should be viewed as a normal part of
data collection and testing
Compliance considerations have clouded the true
understanding of the occurrence of OOSs
Under QbD, control strategies related to
statistical considerations can be brought to bear
to understand and assess the ‘truth’ of an OOS
OOSs can be mitigated through the application of
a Regression to the Mean approach
8
REGRESSION TO THE MEAN (RtM)
Regression to the Mean
(RtM) is a statistical
phenomenon that occurs when
repeated measurements are
made on the same subject or
unit of observation.
http://www.socialresearchmethods.net/kb/regrmean.php
During development studies or
commercial studies where
numerous analytical
determinations are being
carried out on the same batch
or process, isolated unusual
observations can be expected
and understood as a ‘pretest’
to be mitigated through a
9
‘posttest’.
CASE STUDY OF A RTM EXAMPLE
IN RELEASE TESTING OF
CONTENT UNIFORMITY AND HOW
IT CAN BE APPLIED TO AN
ADVANCED
10
CONTROL STRATEGY
HARMONIZED CONTENT UNIFORMITY
TEST - USP <905> (JANUARY, 2007)
Stage 1, 10 Tablets
| M  X |  k  s  15, k  2.4
Noneout side (0.75M,1.25M)
M  X if 98.5  X  101.5
No
Stage 2, additional 20 Tablets
M  98.5 if X  98.5
M  101.5 if X  101.5
Yes
PASS
| M  X |  k  s  15, k  2.0
Noneout side (0.75M,1.25M)
M  X if 98.5  X  101.5
No
FAIL
M  98.5 if X  98.5
M  101.5 if X  101.5
Yes
PASS
11
DATA DESCRIPTION AND GOAL
o
Data

2,000 marketed lots from 12 different groups of sizes



New USP test criteria applied to all 2000 lots


o
1,997 lots passed new USP test <905>,
3 lots failed (X, Y, Z) Stage 1, Stage 2 not available
Issue

o
Manufactured prior to 2007 and tested under old USP
method
6 lots (A – F) with Stage 2 CU data
Lots (X, Y, Z) failed new Stage 1 test and no Stage 2
data
Objective

Predict Stage 2 test likely outcome based on a RtM
analysis
12
STAGE 2 RtM TRAJECTORIES IN 6 LOTS
S2 (n=30) test criteria
3 lots without
S2 samples
S1 (n=10) test criteria
13
MODEL ASSUMPTIONS AND FIT RESULTS
o
o
o
2,000 lots with Stage 1 and Stage 2 (only 6 lots)
data were used for analysis
A mixed effects model was fit to the data with
fixed effects for group and random effects due to
lot-to-lot within group, stage-to-stage test
(analytical run-to-run) within lot and residuals
(dose unit, measurement error, etc.)
The estimated variance components are as
follows:
( b2 ) = 2.8

Var(Lot)

Var(Stage) (2 ) = 7.1

Var(Resid) (w2 ) = 3.8
14
MULTIVARIATE NORMAL DISTRIBUTION
Considering the following two equations
X 1    b   1  w1
X 2i    b   2  w2i , i  1,,20
where b , 1 , 2, w1 , w2i are independent and
b  Norm al( 0 ,  ) ,  1  Norm al( 0 ,   )
2
b
2
 2  Norm al( 0 ,  2 ) , w1  Norm al( 0 ,  w2 /10 )
and w2i  Norm al( 0 ,  w2 ).
15
THEORETICAL RTM EFFECT
Given X 1  x1 ,
X 2i | X 1  x1
 (   b   2  w2i ) | X 1  x1
 (   b | X 1  x1 )   2  w2i
And the conditional distribution of +b given X 1  x1 is

2
  b | X 1  x1  Norm al    ( x1   ) ,  22 (1   2 )
1




where
        / 10 ,    , and  
2
1
2
b
2
2
w
2
2
2
b
 b2
 12 22
16
SIMULATION OF USP STAGE 2 TEST
- ADDITIONAL 20 SAMPLES GIVEN STAGE 1 MEAN

2
Norm al     ( x1   ) ,  22 (1   2 )
1


Adding variability
due to stage-to-stage

Normal 0 ,  2





Adding 20
residual
errors


Norm al(0,  w2 )1 , Norm al(0,  w2 ) 2 , Norm al(0,  w2 ) 20
2
E ( X 2 | X 1  x1 )    
( x1   )
1
RtM
Effect
17
EMPIRICAL RTM EFFECT

The empirical RtM effect can be estimated by a
linear regression of second stage mean (n=20) on
the first stage mean (n=10) from those 6 lots with
stage 2 results as follows:
Obs.S2mean – Group Mean
=  × (Obs.S1mean – Group Mean) + error
18
X 2( n20)   vs. X 1( n10)  
Theoretical RtM
Empirical RtM
19
10,000 SIMULATED S2 RESULTS LOT A
Observed S1
result (n=10)
Observed S2
result (n=30)
Simulated S2
results (n=30)
20
21
SUMMARY RESULTS



Statistical evaluation of 3 lots
reporting outlying Stage 1 results
shows that on simulated retest,
all 3 would pass Stage 2 criterion
(> 99%) as a consequence of
Regression to the Mean Principle
with high certainty
Caveat : Limited information to
estimate RTM effect but justified
by the overwhelmingly stable
behavior of the process
This case study shows the
application of the RtM principle
to OOS mitigation
22
RECOMMENDATIONS AND FUTURE RESEARCH
o
o
o
o
o
As a consequence of the Clinical Drug
Development paradigm, Quality of Materials
should be related to the Average as the essential
quality statement
Content uniformity should not be inferred from
isolated indeterminate OOSs
Re-sampling should be part of a control strategy
to mitigate OOS and exploit the RtM principle to
achieve a more representative sample for
calculating the lot Average
The RtM principle can serve as a basis for
pursuing advanced control strategies for OOS
mitigation
Directions for future research
Bayesian approach – in a commercial context could be easier
to justify priors
 Errors-in-variable adjustment

23