Statistical intervals (STT Consulting) March 2010
Statistical Intervals
STT Consulting
March 2010
dietmar@stt-consulting.com
www.stt-consulting.com
1
Statistical intervals (STT Consulting) March 2010
Confidence Interval
It is often desirable to express or quantify the degree of uncertainty for some estimate of an
unknown population parameter. The most common type of interval estimate is a confidence
interval (CI). The CI is essentially an estimate for an unknown population parameter expressed
as a range of values with some specified level of confidence. The level of confidence describes
the probability that the “interval will capture the true parameter in repeated samples”. CI’s
should be calculated for every estimate of a parameter (mean, SD, r, slope, power, etc).
See also: CI of percentiles >Tolerance Interval
Calculation of confidence intervals
Many of the more simple confidence intervals can be calculated with EXCEL and applying its
“INVERSE” functions (NORSMINV, TINV, CHIINV, FINV: www.stt-consulting.com >Statistics
>Excel Statistical Functions).
Example: confidence interval of the mean
 = xbar ± tP,df * s/SQRT(n) Note: s/SQRT(n) = standard error
R-script {TechingDemos} : Distribution of confidence intervals
ci.examp(mean.sim = 100, sd = 3, n = 9, reps = 100, conf.level = 0.95,
method = "t")
Red vertical lines: z value (units of standard error, here 1.96 * SE)
Pink horizontal lines: CI’s that do not contain the parameter = 100 (5 expected).
2
Statistical intervals (STT Consulting) March 2010
Example: confidence interval of the mean
 = xbar ± tP,df * s/SQRT(n) Note: s/SQRT(n) = standard error
R-script : Distribution of t-values
n=9
loop=500000
t=c(rep(0,loop))
for(i in 1:loop)
{
x=rnorm(n, mean = 0, sd = 3)
xstd=sqrt(var(x))
xbar=mean(x)
se=xstd/sqrt(n)
t[i]=(0-xbar)/se
}
hist(t,breaks=550, ylim=c(0,0.5), xlim=c(-6,6), freq=FALSE, lty=0)
lines(density(t), col="red", lwd=2)
quantile(t,probs=c(0.025,0.5,0.975)) #t = qt(0.975,8) = 2.306
3
Statistical intervals (STT Consulting) March 2010
Prediction interval
Assume, you have estimated mean (xbar) and standard deviation (s) from n results (for
example, 20). Now, you want to predict the upper limit for the next result with 95% probability
(also denoted as (100)%;  0.95).
Prediction interval, next n + 1
xi = xbar + kP*s, with
kP = t(p=2sided 95%,df=19)*SQRT(1+1/20) = 2.14 (see Table)
Assume, you want to predict the upper limit to contain ALL next m results with 95% probability,
or to contain ALL of m-k out of m additional observations. These calculations are complex and
one has to consult tables or use software.
Prediction interval, ALL next m-k out of m
Xmk = xbar + kP*s, with
kP: Consult tables (Odeh 1990) or use software
Factor = 3.16, for n = 20, next 9 of 9
4
Statistical intervals (STT Consulting) March 2010
Odeh RE. Two-sided prediction intervals to contain at least k out of m future observations from
a normal distribution. Technometrics 1990;32:203-16.
Table 1.A.0.1: Factors for (100)% two-sided prediction intervals, based on a sample of
size n, to contain all of m out of m additional observations
 = .95
m
n
1
2
3
4
5
6
7
8
9
8
2.51
2.97
3.24
3.43
3.57
3.69
3.79
3.88
3.95
9
2.43
2.86
3.11
3.29
3.43
3.54
3.63
3.71
3.78
10
2.37
2.79
3.02
3.19
3.32
3.43
3.51
3.59
3.66
11
2.33
2.72
2.95
3.11
3.24
3.34
3.42
3.49
3.56
12
2.29
2.68
2.90
3.05
3.17
3.27
3.35
3.42
3.48
15
2.22
2.57
2.78
2.92
3.03
3.12
3.19
3.26
3.31
20
2.14
2.48
2.67
2.80
2.90
2.98
3.05
3.11
3.16
25
2.10
2.43
2.61
2.73
2.83
2.91
2.97
3.03
3.08
30
2.08
2.39
2.57
2.69
2.78
2.86
2.92
2.97
3.02
40
2.05
2.35
2.52
2.64
2.73
2.80
2.86
2.91
2.95
60
2.02
2.31
2.48
2.59
2.67
2.74
2.80
2.85
2.89
120
1.99
2.27
2.43
2.54
2.62
2.68
2.74
2.79
2.83
240
1.97
2.25
2.41
2.51
2.59
2.66
2.71
2.76
2.80
480
1.97
2.25
2.40
2.50
2.58
2.64
2.70
2.74
2.78
Infinite
1.96
2.24
2.39
2.49
2.57
2.63
2.68
2.73
2.77
Note, infinitely, these intervals limit to the 97.5% quantile of the distribution of the maximum
values sampled from a Normal Distribution with m results (for example, 2.73 for m = 8). This
shall be demonstrated with an R-script.
5
Statistical intervals (STT Consulting) March 2010
R-script: Distribution of maximum values of a sample with size n taken from a Normal
Distribution with  = 0 and  = 1
n=8
loop=500000
Maximum=c(rep(0,loop))
for(i in 1:loop)
{
x=rnorm(n, mean = 0, sd = 1)
Maximum[i]=max(x)
}
quantile(Maximum, probs = c(0.025,0.5,0.975))
hist(Maximum,breaks=60,xlim=c(-2,5))
Quantiles
2.5%
50%
97.5%
15000 30000
of Maximum
0
Frequency
0.3340892 1.3849190Histogram
2.7326133
-2
-1
0
1
2
3
4
5
Maximum
The distribution is slightly skewed and its 97.5% quantile is at 2.73 (also consult the Table
above).
6
Statistical intervals (STT Consulting) March 2010
Prediction interval, high number of next m
The more “next” results we want to be included in the prediction
interval, the greater the k-factor becomes (see Table).
Odeh RE. Two-sided prediction intervals to contain at least k out of m future observations from
a normal distribution. Technometrics 1990;32:203-16.
Table 1.A.0.1: Factors for (100)% two-sided prediction intervals, based on a sample of
size n, to contain all of m out of m additional observations
 = .95
m
n
10
15
20
30
40
50
60
80
100
8
4.02
4.28
4.46
4.70
4.88
5.01
5.11
5.28
5.40
9
3.85
4.09
4.25
4.49
4.65
4.77
4.87
5.02
5.14
10
3.72
3.94
4.10
4.32
4.47
4.59
4.68
4.83
4.94
11
3.62
3.83
3.98
4.19
4.34
4.45
4.54
4.68
4.79
12
3.53
3.74
3.89
4.09
4.23
4.34
4.42
4.56
4.66
15
3.36
3.56
3.69
3.87
4.00
4.10
4.18
4.30
4.40
20
3.21
3.38
3.50
3.67
3.79
3.88
3.95
4.06
4.15
25
3.12
3.28
3.40
3.56
3.67
3.75
3.82
3.93
4.01
30
3.06
3.22
3.33
3.48
3.59
3.67
3.73
3.84
3.92
40
2.99
3.14
3.25
3.39
3.49
3.57
3.63
3.73
3.80
60
2.93
3.07
3.17
3.30
3.40
3.47
3.53
3.62
3.69
120
2.86
3.00
3.09
3.22
3.31
3.38
3.43
3.52
3.58
240
2.83
2.96
3.05
3.18
3.26
3.33
3.38
3.46
3.53
480
2.81
2.94
3.03
3.16
3.24
3.31
3.36
3.44
3.50
Infinite
2.80
2.93
3.02
3.14
3.22
3.28
3.33
3.41
3.47
For high m, we therefore may consider another interval, the Tolerance Interval.
7
Statistical intervals (STT Consulting) March 2010
Tolerance interval
Assume, you have estimated mean (xbar) and standard deviation (s) from n results (for
example, 20). Now you want to know in which interval you will find a certain percentage (100)% of future measurements with a certain prabability (100-)% (or confidence). For example, a
tolerance interval with 95% confidence for 90% of distribution implies 95% of intervals cover at
least 90% of distribution.
A tolerance interval can also be viewed as a confidence interval for the pth centile (κp = μ +
Zp*σ, for example, z = 1.65, 1.96). The definition of a tolerance interval, unfortunately, is not
used consistently in the literature. Also, several oversimplified formulas exist for its calculation
(t/sqrt(n)*sqrt(1+nz2(Cn2-1)) * s; t*sqrt(1/n+z2*Cn2*1/2n) * s). The publication “Chakraborti S, Li
J. Confidence interval estimation of a normal percentile. Amer Statistician 2007;61:331-6” gives
a very good insight into the statistics of the confidence interval of a percentile/tolerance interval.
Other important references are:
A. Wald, J. Wolfowitz (1946), Tolerance limits for a normal distribution, Annals of
Mathematical Statistics 17, 208-215.
R. E. Odeh, D. B. Owen (1980), Tables for Normal Tolerance Limits, Sampling Plans, and
Screening, Marcel Dekker, New York.
The 2 different approaches can be expressed in the formulas
TI = xbar + kT*s
p= xbar + Cn*Zp*s ± CI
(Cn is a factor needed because s is a biased estimator; see
Chakraborti)
Note: finally, one arrives at the same numbers!
8
Statistical intervals (STT Consulting) March 2010
Calculation of kT
The calculation of kT requires values of the non-central t-distribution. Most easily, the
calculations can be performed with “R” and the package {tolerance} installed (http://cran.rproject.org/web/packages/tolerance/tolerance.pdf). Otherwise, tables should be consulted.
Note, the package {spc} gives slightly different values; {tolerance} with “WBE” (WeissbergBeatty) is preferred by myself because it gives values consistent with ISO 3207 and many other
documents.
Howe, W. G. (1969), Two-Sided Tolerance Limits for Normal Populations - Some
Improvements, Journal of the American Statistical Association, 64, 610–620.
Weissberg, A. and Beatty, G. (1969), Tables of Tolerance Limit Factors for Normal
Distributions, Technometrics, 2, 483–500.
R-script
K.factor(n = seq(5,20,1), alpha = 0.05, P = 0.95, side = 2, method = "WBE")
(ignore the warning messages).
n=5
6
7
8
9
10
11
12
5.08
4.41
4.01
3.73
3.53
3.38
3.26
3.16
n = 13
14
15
16
17
18
19
20
3.08
3.01
2.95
2.90
2.86
2.82
2.78
2.75
Note, the formulas given in http://www.itl.nist.gov/div898/handbook/prc/section2/prc263.htm
work well for the 2-sided case, but not so good for the 1-sided case.
Other useful resources
Statistical Intervals: A Guide for Practitioners. Gerald J. Hahn and William Q. Meeker. John
Wiley & Sons, 1991.
Software related to the book: http://www.public.iastate.edu/~wqmeeker/stint.html
Note: the prediction interval (npik) is an approximation, only!
http://www.public.iastate.edu/~wqmeeker/anonymous/Papers/HahnMeekerTAS1993.pdf
9
Statistical intervals (STT Consulting) March 2010
Applications (for the medical laboratory)
Confidence intervals
Confidence intervals are generally applied and relate to statistical null-hypothesis significance
testing.
Prediction intervals
Prediction intervals are seldom used in the medical laboratory, except in regression analysis.
However, they are typically not utilized. One exception is their application for commutability
testing in the CLSI EP14 document.
Tolerance intervals
Tolerance intervals are applied for the limit of the blank (LoB), the Bland-Altman limits of
agreement, and for the reference interval (also IQC limits). Typically, however, they are not
named tolerance intervals, but rather seen as confidence intervals of percentiles.
10