Trend Data
Lynn Torbeck
Torbeck and Assoc.
Evanston, IL
June 28, 2007
1
Overview
OOT vs. OOS
Why trend?
How to get started
Types of trends with examples
OOT is relative
Graphical tools
Tend limits
June 28, 2007
2
Why Trend Data?
Good business practice.
Early warning of possible Out Of
Specification (OOS) results.
Gain process understanding.
Minimize risk of potential failures of
product in the market.
Find the “gold in the hills” for process
improvements.
June 28, 2007
3
Regulatory Basis for Trending
No specific regulation requirement
211.180(e) Annual Reviews
FDA Form 483 for observations
Establishment Inspection Reports
Warning letters
FDA presentations at conferences
June 28, 2007
4
OOS Guidance Footnote
“Although the subject of this document
is OOS results, much of the guidance
may be useful for examining results
that are out of trend (OOT).”
How is OOT different than OOS?
How is OOT the same as OOS?
June 28, 2007
5
Out Of Specification - OOS
OOS is the comparison of one result
versus a predetermined specification
criteria.
OOS investigations focus on
determining the truth about that one
value.
Is the OOS result confirmed or not?
June 28, 2007
6
Out Of Trend - OOT
OOT is the comparison of many
historical data values versus time.
OOT investigations focus on
understanding non-random changes.
Is the non-random change confirmed or
not?
June 28, 2007
7
OOS Guidance
Taking into account the differences
between OOS and OOT, the guidance
does provide a framework for OOT
investigations:
Responsibilities
Philosophical basis
General principles of investigations
June 28, 2007
8
1. How to get started
Select the variable to be studied:





June 28, 2007
Potency
Yield
Impurities
Hardness
Bioburden
9
2. How to get started
Select a time period:
At least one year if possible.
More than two preferred.
Do not go past a major change in the
process. Use process knowledge to
advantage.
Use the reportable result, the value
compared to the specifications.
June 28, 2007
10
3. How to get started
Enter the data into analysis software:







Excel
Minitab
Sigma Plot
JMP
StatGraphics
Northwest Analytical
SAS
June 28, 2007
11
4. How to get started
Plot the data vs. time or lot sequence.
Look for non-random changes over
time.
Determine if they are of practical
importance.
Statistical significance is insufficient.
Do an impact and risk assessment.
June 28, 2007
12
What is Trending?
The several activities of:








Collecting data,
Recording it,
Documenting it,
Storing it,
Monitoring it,
Fitting models to it
Evaluating it, and
Reporting it.
June 28, 2007
13
What is a trend?
Any non-random pattern.
Short and long term patterns in data
over time that are of practical
importance.
June 28, 2007
14
Beneficial Trends
Desirable patterns in the data series.
Examples:




A move toward the target or center of the
specification.
More consistent with less variation.
Less likelihood of an OOS value.
A benefit to SSQuIP.
June 28, 2007
15
Beneficial Trend
1.2
mg/mL
1
0.8
0.6
0.4
0.2
0
12/10/200 6/28/2003 1/14/2004 8/1/2004 2/17/2005 9/5/2005 3/24/2006 10/10/200
2
6
Date
June 28, 2007
16
No Trend
June 28, 2007
104
103
102
Normal
Easier to define
what a trend is not.
Random data
Noise
Stationary
No ups, no downs
No cycles
No outliers
101
100
99
98
97
Index
100
200
300
400
500
17
Neutral or No Trend
Neither beneficial or adverse
Examples:



Results that are always the same.
Stability data with a slope of zero.
Data in a state of “statistical control” on a
control chart.
June 28, 2007
18
Process Control
Statistical Process Control, SPC


Normal random data over time
Due to common causes only
Engineering Process Control, EPC



Estimate departures from target
Feedback to control point
Physical changes to the process
June 28, 2007
19
Adverse Trends
Undesirable patterns in the data series.
Examples:





A movement away from the target.
Increased variability.
Increased probability of OOS.
An unexplained change to a beneficial
trend.
A challenge to SSQuIP.
June 28, 2007
20
Out-of-Trend (OOT)
A change from an established pattern
that has the potential of an adverse
effect on SSQuIP or of becoming OOS.
Must be large enough to be of practical
significance.
Statistical significance is insufficient to
determine OOT.
June 28, 2007
21
Long Term Change
June 28, 2007
Increasing Trend
.1 per step after 50
107
106
105
104
Yields
Not stationary
around a fixed value
Increasing or
decreasing average.
Apparently will
continue to get
worse (or better)
unless action is
taken.
103
102
101
100
99
98
Index
10 20 30 40 50 60 70 80 90 100
22
The Aberrant Outlier
June 28, 2007
An outlier
Mu=100, Sigma=1.0
105
104
103
102
Yield%
Stationary and
random but with
one very large value
that could be a
statistical outlier.
Generally assumed
to be due to a
“special cause.”
101
100
99
98
97
Index
10 20 30 40 50 60 70 80 90 100
23
Shift in the Average
Mean Shift
Mu=100 to 104 Sigma=1.0
106
Yield
Here the mean has
increased from 100
to 104 at sample 51.
No other changes
were made.
Variability is the
same.
101
96
Index
June 28, 2007
10 20 30 40 50 60 70 80 90 100
24
Variation Change
Increasing Variability
Mu=100, Sigma=1.0, 2.0, 3.0 & 4.0
110
Yield %
This is stationary
around a fixed mean
of 100%.
But, the standard
deviation increased
from 1.0 to 4.0.
100
90
Index
June 28, 2007
10 20 30 40 50 60 70 80 90 100
25
Cycles
Cycles
104
103
102
Cycles
A reoccurring cycle.
Stationary about a
fixed mean.
The data are not
independent.
101
100
99
98
97
96
Index
June 28, 2007
10 20 30 40 50 60 70 80 90 100
26
Autocorrelated
Autocorrelated
105
104
103
AutoCorr
Data are correlated
with the previous
data.
Not stationary.
Check different time
lags, 1,2, ….
102
101
100
99
98
Index
June 28, 2007
10 20 30 40 50 60 70 80 90 100
27
OOT is Relative
Stationary White Noise
mu=100%, S=1%
Yield %
110
100
90
Index
June 28, 2007
10
20
30
40
50
60
70
80
90
100
28
OOT is Relative
The importance of a trend is its size
relative to the specification criteria.
A state of Statistical Control is desired
but not necessary.
A state of Engineering Control is
necessary to meet specifications.
Success is a marriage of the two.
June 28, 2007
29
A Little Humor (Very Little)
Lottery: A tax on the statisticallychallenged.
If you want three opinions, just
ask two statisticians.
Statistics means never having to
say you're certain.
http://www.keypress.com/x2815.xml
June 28, 2007
30
Trend Fitting
“The general process of representing
the trend component of a time series.”
A Dictionary of Statistical Terms. Marriott
Depends very much on the type of data
and the subject matter being studied.
Need to adapt the tools and techniques
to our specific data and issues.
June 28, 2007
31
Tools of Trending
Summary statistics


Averages, Medians
Ranges, Standard Deviations, %RSD
Graphical plots
Distribution analysis - Histograms
Outlier determination
Regression analysis
June 28, 2007
32
Graphic Tools
Line Plots vs. time.
Shewhart Control Charts.
Histograms.
Sector chart
June 28, 2007
33
Line Plots vs. Time
June 28, 2007
102
101
Yield %
Response on the
vertical axis.
Time or batch # on
the horizontal axis.
Usually connect the
data points with a
line, but optional.
Stationary Time Series
Mu=100, Sigma=1.0
100
99
98
Index
10
20
30
40
50
60
70
80
90 100
34
Control Chart
I and MR Chart for Yield %
June 28, 2007
Individual Value
103.5
UCL=103
102.5
101.5
100.5
Mean=100
99.5
98.5
97.5
LCL=97
96.5
Subgroup
4
Moving Range
Add ‘natural process
limits’ to the line
plot.
±3
A chart for the
response.
A chart for the
variability.
0
50
100
UCL=3.686
3
2
1
R=1.128
0
LCL=0
35
Control Chart Family
Individuals
Averages
Medians
Standard deviations
Ranges
Number of defectives
Fraction defectives
Defects per units
Number of defects
June 28, 2007
36
Variation Change
I and MR Chart for Yield %
110
Individual Value
A control chart will
detect change in the
variation.
1
1
1
1 11 1
1
1
UCL=103
100
1
11
1
1
90
Subgroup
Mean=100
LCL=97
11 1 1
1
1 1
1
0
1
1
50
1
100
Moving Range
1
1
1 1
1
5
0
June 28, 2007
1
10
1 1
1 111 1 1
11
1
11
1 1
11
111
1
1
1
11
11 1 1
UCL=3.686
R=1.128
LCL=0
37
The Outlier
I Chart for Yield%
106
1
105
104
Individual Value
A control chart finds
values outside the
natural limits of the
data.
The value is larger
than would be
expected by chance
alone.
103
UCL=103
102
101
100
Mean=100
99
98
97
LCL=97
96
0
50
100
Observation Number
June 28, 2007
38
“Western Electric” Rules
1. One value outside 3 S limits.
2. Nine values in a row on one side of
the average.
3. Six values in a row all increasing or
decreasing.
4. 14 values in a row alternating up and
down.
June 28, 2007
39
“Western Electric” Rules
5. Two of three values greater than 2 S
from the average.
6. Four of five values greater than 1 S
from the average.
7. 15 values in a row within 1 S of the
average.
8. Eight values in a row greater than 1 S.
June 28, 2007
40
Histogram
20
Frequency
Show the ‘shape’ of
the distribution of
data.
In this case it is
Normally distributed.
10
0
96
97
98
99
100
101
102
103
104
Yield %
June 28, 2007
41
The Outlier
The outlier is clearly
seen in the
histogram.
Variation Change
Frequency
20
10
0
97
98
99 100 101 102 103 104 105 106
Yield%
June 28, 2007
42
Outlier Determination
Reference:





USP 30 NF 25
Chapter <1010>
“Analytical Data – Interpretation and
Treatment”
Page 392 “Outlying Results”
Appendix C: Examples of Outlier Tests for
Analytical Data.
June 28, 2007
43
Regression Analysis
99% Prediction Interval
220000
200000
Value
180000
160000
140000
120000
100000
0
June 28, 2007
5
10
Months
15
20
44
Trend Limits
Numeric (or non-numeric) criteria, that if
exceeded, indicates that an out-of-trend
change has occurred.
Usually the ‘natural process’ variation
AKA “Alert limits”
Use Statistical Tolerance Limits
See USP <1010> Appendix E
June 28, 2007
45
Here, Trend This
Var 1
40
30
20
Index
June 28, 2007
100
200
300
46
A New Engineering Chart
Brings together for the first time:



Comparison to the specification limits in
place of the probability limits
Divides the specification range into equal
zones in place of 1, 2, & 3 sigma areas
Uses cumulative scores
Pharmaceutical Technology, April 2005
June 28, 2007
47
The New “Sector Chart”
SIALIC ACID EXAMPLE
Fail
3.915 3.695 3.298 4.04 3.87 4.147 3.938 4.167 3.9 3.927 3.81 3.9 4.033 3.853 4.142 3.958 3.77
Sector Weight
F 10
D
2
C
1
B
0
A
0
A
0
B
0
C
1
D
2
F 10
Low High
4.1
4
3.9
3.8
3.7
3.6
3.5
3.4
4.2
4.099
3.999 0
3.899
3.799
3.699
3.599
3.499
June 28, 2007
Batch
2
2
2
1
1
0
0
0
0
0
0
0
0
0
0
10
1
2
3
4
5
6
7
8
9
10
11 12 13
14
15
4816
17
The New “Sector Chart” Rules
The first batch tally takes the weight of the
sector it is in.
Subsequent batches have a cumulative tally
of the previous tally plus the current sector
weight.
If the tally reaches a value of, say, 10, an
alert is given.
If the batch enters the A or B sectors, the
tally is reset to zero.
June 28, 2007
49
The New “Sector Chart” Rules
Sectors A and B cover the center 50%
of the specification range.
Sector F is outside the current
specification.
Other weights can be set to fit the
process and the degree of sensitivity
needed.
June 28, 2007
50
Advantages of Sector Chart
No minimum sample size. Can start with
one data point.
No assumptions about the data at all.
Identifies beneficial and adverse trends.
Weights and tally total are selected by
scientific and empirical knowledge.
A decision is made with each new point.
Alerts quickly if a problem exists.
June 28, 2007
51
Justification for Sector Chart
If the process is well inside the
specification, it need not be in a state of
statistical control.
The focus is on OOT and SSQuIP not
being out of “statistical” control.
Sensitivity of the chart is adjustable.
Can be use in parallel with other charts.
June 28, 2007
52
That’s All Folks
1.
2.
3.
4.
5.
Summary Points:
OOT is not OOS
OOT is non-random changes over time
OOT is a statistical and graphical issue
OOT is relative. Statistical significance
is not sufficient.
Trend limits = Natural Limits
June 28, 2007
53
References
Graphics:


http://www.edwardtufte.com/tufte/
http://www.itl.nist.gov/div898/handbook/e
da/section3/eda34.htm
Statistics

http://www.itl.nist.gov/div898/handbook/in
dex.htm
June 28, 2007
54
Software References
http://www.minitab.com/
http://www.systat.com/products/sigma
plot/
http://www.nwasoft.com/
http://www.jmp.com/
http://www.statgraphics.com/
http://www.sas.com/
June 28, 2007
55