DEFINITIONS AND FORMULAE WITH
STATISTICAL TABLES
FOR ELEMENTARY STATISTICS AND
QUANTITATIVE METHODS COURSES
Department of Statistics
University of Oxford
October 2012
Contents
1 Laws of Probability
1
2 Theoretical mean and variance for discrete distributions
1
3 Mean and variance for sums of Normal random variables
1
4 Estimates from samples
1
5 Two common discrete distributions
1
6 Standard errors
2
7 95% confidence limits for population parameters
2
8 z-tests
3
9 t-tests
3
10 The χ2 -test
3
11 Correlation and regression
4
12 Analysis of variance
4
13 Median test for two independent samples
5
14 Rank sum test or Mann-Whitney test
5
15 Sign test for matched pairs
5
16 Wilcoxon test for matched pairs
5
17 Kolmogorov-Smirnov test
5
18 Kruskal-Wallis test for several independent samples
6
19 Spearman’s Rank Correlation Coefficient
6
20 TABLE 1 : The Normal Integral
7
21 TABLE 2 : Table of t
TABLE 3 : Table of χ2
8
22 TABLE 4 : Table of F for P = 0.05
9
23 TABLE 5 : Critical values of R for the Mann-Whitney rank-sum test
9
24 TABLE 6 : Critical values for T in the Wilcoxon Matched-Pairs Signed-Rank test . 10
1
Laws of Probability
Multiplication law
Addition law
Bayes’ Rule
B1
A1
B2
B1
A2
B2
2
P (A and B) = P (A)P (B|A) = P (B)P (A|B)
P (A or B) = P (A) + P (B) − P (A and B)
P (B1 ) = P (B1 |A1 )P (A1 ) + P (B1 |A2 )P (A2 )
Theoretical mean and variance for discrete distributions
σ 2 = Σ(x − µ)2 p(x)
µ = Σxp(x)
3
P (B1 |A1 )P (A1 )
P (B1 )
P (A1 |B1 ) =
Mean and variance for sums of Normal random variables
P
If Xi ∼ N(µi , σi2 ), i = 1, . . . , n and let Y = ni=1 ai Xi then
µY =
n
X
σY2 =
ai µ i
i=1
4
n
X
a2i σi2
i=1
Estimates from samples
Ungrouped data:
Σx
sample mean x̄ = n i
sample variance s2 =
5
Σx2i −nx̄2
n−1
2
for x in a sample of size n ,
x
sample proportion p̂ = n
Two common discrete distributions
(i) Binomial
p(x) =
n
(ii) Poisson
Cx px q n−x , x = 0, 1, . . . , n
µ = np, σ 2 = npq, σ =
n
=
Σf (x−x̄)2
s = Σf −1
Σf x
Grouped data: x̄ = Σf
Counted events:
Σ(xi −x̄)2
n−1
√
p(x) =
npq
e−λ λx
x! ,
µ = λ,
n!
Cx = x!(n−x)!
1
x = 0, 1, . . . , ∞
σ 2 = λ,
σ=
√
λ
6
Standard errors
Single sample of size n
SE(x̄) = √σn or, if σ unknown, √sn
q
p pq
SE(p̂) =
with q = 1 − p ,
n
or, if p unknown,
p̂(1−p̂)
n
Sampling without replacement
When n individuals are sampled from a population of N without replacement, the standard
error is reduced. The standard error for no replacement SENR is related to the standard error
with replacement SEWR by the formula
s
s
σ
n−1
n−1
=√
,
1−
1−
SENR = SEWR
N −1
N −1
n
where σ is the known standard deviation of the whole population.
Two independent samples of sizes, n1 and n2
q
SE(x̄1 − x̄2 ) =
σ12
n1
+
σ22
n2
q
or, if σ1 and σ2 unknown and different,
For common but unknown σ ,
q
SE(p̂1 − p̂2 ) =
p1 q1
n1
+
p2 q2
n2 ,
SE(x̄1 − x̄2 ) = s
q
p̂1 q̂1
n1
7
+
1
n2
with s2 =
s22
n2
.
(n1 −1)s21 +(n2 −1)s22
n1 +n2 −2
p̂2 q̂2
n2
+
For common but unknown p , SE(p̂1 − p̂2 ) =
n1 p̂1 +n2 p̂2
n1 +n2
1
n1
+
or,
if p1 and p2 unknown and unequal,
p defined as p̂ =
q
s21
n1
q
p̂q̂( n11 +
1
n2 )
where p̂ is a pooled estimate of
and q̂ = 1 − p̂.
95% confidence limits for population parameters
Mean:
when σ known use
when σ unknown use
x̄ ± 1.96 √σn
x̄± t √sn
where t is the tabulated two-sided 5% level value with degrees of freedom, d.f. = n − 1
p
Proportion:
p̂ ± 1.96 p̂q̂/n
2
8
z-tests
x̄−µ
z = σ/√n
Single sample test for population mean µ (known σ):
p̂−p
z = √ pq
Single sample test for population proportion p:
n
z=
Two sample test for difference between two means (known σ1 and σ2 ):
Two sample test for difference between two proportions :
z =q
where p̂ is a pooled estimate of p defined as p̂ =
9
2
2
σ1
σ2
n1 + n2
pˆ1 −pˆ2
p̂q̂( n1 + n1 )
1
n1 p̂1 +n2 p̂2
n1 +n2
rx̄1 −x̄2
2
and q̂ = 1 − p̂
t-tests
Population variance σ 2 unknown and estimated by s2
x̄−µ
Single sample test for population mean µ
t = √ with d.f.= n − 1
s/ n
Paired samples : test for zero mean difference, using n pairs (x, y), d = x − y
d¯√
t=
with d.f.= n − 1, where d¯ and sd are the mean and standard deviation of d.
sd / n
Independent samples test for difference between population means µx and µy using nx x’s
and ny y’s. Provided that s2x and s2y are similar values, use the pooled variance estimate,
(nx −1)s2x +(ny −1)s2y
x̄−ȳ
2
s =
with d.f = nx + ny − 2
,
and t = q 1
nx +ny −2
s
+1
nx
10
ny
The χ2 -test
(Note that the two tests in this section are nonparametric tests. There are χ2 tests of variances, not included here, that are parametric.)
χ2 Goodness-of-fit tests using k groups have
d.f.=(k − 1) − p where p is the number of independent parameters estimated
and used to obtain the (fitted) expected values.
χ2 Contingency table tests on two-way tables with r rows and c columns have
d.f.= (r − 1)(c − 1)
For both tests, χ2 = Σ
expected frequency
(O−E)2
E
where O is an observed frequency and E is the corresponding
3
11
Correlation and regression
For n pairs
(xi , yi ), with sample variances s2x and s2y as in section 3, define sample covariance,
P
P
−x̄)(yi −ȳ)
1
n
sxy = (xi(n−1)
. May be computed as sxy = n−1
xi yi − n−1
x̄ȳ.
2
2
Computed from the sample variances sx+y of x + y and sx−y of x − y as sxy = 14 (s2x+y − s2x−y ).
Sample product-moment correlation coefficient,
sxy
r= s s
x y
Test the significance of
√ the correlation coefficient, ρ = 0, or equivalently, of the regression
r n−2
slope β = 0:
t =√
with d.f.= n − 2
1−r2
Linear Regression of y on x. Equation y = α + βx with α and β estimated by a and b.
sxy
a = ȳ − bx̄.
Estimates:
b = s2
x
√
Root-mean-square error of regression prediction given by sy 1 − r2 .
√
b √1−r2
b
Test significance of regression as above or t= SE(b) with d.f.= n − 2 where SE(b) =
r n−2
b ± t.SE(b), where t is the tabulated two-sided
5% level value with d.f.= n − 2
95% confidence limits for the slope β are:
12
Analysis of variance
Single factor or One-way analysis for a completely randomized design.
The test statistic F is calculated as a ratio of two mean squares. If the numbers in the k groups
are n1 , n2 , . . . , nk then the total sample size is Σni = n. Calculate the “total sum of squares”,
T SS = (n − 1)s2T , where s2T is variance of all n observations. Calculate the sample means and
sample variances of the k groups by x̄1 , x̄2 , . . . , x̄k and s21 , s22 , . . . , s2k and then the “within groups
sum of squares” (also known as the“error sum of squares”), ESS = Σ(ni P
−1)s2i . The “between
groups sum of squares” may be computed in two different ways: BSS = ni (x̄i − x̄T )2 , where
x̄T is the mean of all n observations; or BSS = T SS − ESS.
These together with their degrees of freedom are entered into the ANOVA table:
Source of variation
Degrees
of freedom
Sum of
squares
Between samples
k−1
BSS
BM S =
BSS
k−1
Within samples
n−k
ESS
EM S =
ESS
n−k
Total
n−1
T SS
4
Mean
square
F
BM S
EM S
13
Median test for two independent samples
For two independent samples, sizes n1 and n2 , the median of the whole sample of n = n1 + n2
observations is found. The number in each sample above this median is counted and expressed
as a proportion of that sample size. The two proportions are compared using the Z-test as in §8.
14
Rank sum test or Mann-Whitney test
For two independent samples, sizes n1 and n2 , ranked without regard to sample, call the sum
of the ranks in the smaller sample R. If n1 ≤ n2 ≤ 10 referq
to Table 5, otherwise use a Z
n1 n2 (n1 +n2 +1)
test with z = (R − µ)/σ where µ = 21 n1 (n1 + n2 + 1) and σ =
, assuming
12
n1 ≤ n2 . In case of ties, ranks are averaged.
15
Sign test for matched pairs
The number of positive differences from the n pairs is counted. This number is binomially
distributed with p = 21 , assuming a population zero median difference. So apply the Z test
for a binomial proportion with p = 21 .
16
Wilcoxon test for matched pairs
Ignoring zero differences, the differences between the values in each pair are ranked without
regard to sign and the sums of the positive ranks, R+ and of the negative ranks, R− , are
calculated. (Check R+ + R− = 12 n(n + 1), where n is the number of nonzero differences). The
smaller of R+ and R− is called T and may be compared with the critical values in Table 6 for
a two-tailed test. (For one-tailed tests, use R− and R+ with the same table, remembering to
halve P .) In case of ties, ranks are averaged.
17
Kolmogorov-Smirnov test
Two samples of sizes n1 and n2 are each ordered along a scale. At each point on the scale
the empirical cumulative distribution function is calculated for each sample and the difference
between the pairs are recorded as Di . The largest absolute value of the Di is called Dmax and
this value is compared with the 5% one-tailed value
r
n1 + n2
Dcrit = 1.36
.
n1 n2
Single sample version, compares sample with theoretical distribution,
r
1
Dcrit = 1.36
.
n
Should only be used with no ties, but it commonly is used otherwise. With ties, the value of
Dmax tends to be too small, so that the p-value is an overestimate.
5
18
Kruskal-Wallis test for several independent samples
(Analysis of variance for a single factor). For k samples of sizes n1 , n2 , ..nk , comprising a total
of n observations, all values are ranked without regard to sample, from 1 to n. The rank sums
for the samples are calculated as R1 , R2 , .., Rk . (Check ΣRi = 21 n(n + 1). The test statistic is
R2
12
H = n(n+1) Σ ni
i
h
i
−3(n + 1),
which is compared to χ2 table with d.f. = k − 1
19
Spearman’s Rank Correlation Coefficient
If x and y are ranked variables the Spearman Rank Correlation Coefficient is just the sample
product moment correlation coefficient between the pairs of ranks, rs , which may also be
computed by
rs = 1−
6Σd2
n(n2 −1)
where d is the difference
x − y, and n is the number of pairs (x, y).
√
r√
n−2
with d.f.= n − 2
Test rs using t = s
1−rs2
6
20
TABLE 1 : The Normal Integral
Tabled value is P
2P ! 1
P
z
0
!z
0
1!P
1!P
!z 0
z
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
z
0
z
0.00
0.5000
0.5398
0.5793
0.6179
0.6554
0.6915
0.7257
0.7580
0.7881
0.8159
0.8413
0.8643
0.8849
0.9032
0.9192
0.9332
0.9452
0.9554
0.9641
0.9713
0.01
5040
5438
5832
6217
6591
6950
7291
7611
7910
8186
8438
8665
8869
9049
9207
9345
9463
9564
9649
9719
0.02
5080
5478
5871
6255
6628
6985
7324
7642
7939
8212
8461
8686
8888
9066
9222
9357
9474
9573
9656
9726
0.03
5120
5517
5910
6293
6664
7019
7357
7673
7967
8238
8485
8708
8907
9082
9236
9370
9484
9582
9664
9732
0.04
5160
5557
5948
6331
6700
7054
7389
7704
7995
8264
8508
8729
8925
9099
9251
9382
9495
9591
9671
9738
0.05
5199
5596
5987
6368
6736
7088
7422
7734
8023
8289
8531
8749
8944
9115
9265
9394
9505
9599
9678
9744
0.06
5239
5636
6026
6406
6772
7123
7454
7764
8051
8315
8554
8770
8962
9131
9279
9406
9515
9608
9686
9750
0.07
5279
5675
6064
6443
6808
7157
7486
7794
8078
8340
8577
8790
8980
9147
9292
9418
9525
9616
9693
9756
0.08
5319
5714
6103
6480
6844
7190
7517
7823
8106
8365
8599
8810
8997
9162
9306
9429
9535
9625
9699
9761
0.09
5359
5753
6141
6517
6879
7224
7549
7852
8133
8389
8621
8830
9015
9177
9319
9441
9545
9633
9706
9767
2.0 0.9772
3.0 0.9987
0.1
9821
9990
0.2
9861
9993
0.3
9893
9995
0.4
9918
9997
0.5
9938
9998
0.6
9953
9998
0.7
9965
9999
0.8
9974
9999
0.9
9981
9999
7
21
1!
2
TABLE 3 : Table of χ2
TABLE 2 : Table of t
P
P
P
1!
2
!t
0
t
X2
Probability P of a value of χ2
greater than:
Probability P of lying outside ±t
d.f.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
40
60
∞
P=0.10
6.31
2.92
2.35
2.13
2.02
1.94
1.90
1.86
1.83
1.81
1.80
1.78
1.77
1.76
1.75
1.75
1.74
1.73
1.73
1.73
1.72
1.72
1.71
1.71
1.71
1.71
1.70
1.70
1.70
1.70
1.68
1.67
1.65
P=0.05
12.71
4.30
3.18
2.78
2.57
2.45
2.37
2.31
2.26
2.23
2.20
2.18
2.16
2.15
2.13
2.12
2.11
2.10
2.09
2.09
2.08
2.07
2.07
2.06
2.06
2.06
2.05
2.05
2.05
2.04
2.02
2.00
1.96
P=0.02
31.82
6.96
4.54
3.75
3.36
3.14
3.00
2.90
2.82
2.76
2.72
2.68
2.65
2.62
2.60
2.58
2.57
2.55
2.54
2.53
2.52
2.51
2.50
2.49
2.49
2.48
2.47
2.47
2.46
2.46
2.42
2.39
2.33
P=0.01
63.7
9.93
5.84
4.60
4.03
3.71
3.50
3.36
3.25
3.17
3.11
3.06
3.01
2.98
2.95
2.92
2.90
2.88
2.86
2.85
2.83
2.82
2.81
2.80
2.79
2.78
2.77
2.76
2.76
2.75
2.70
2.66
2.58
8
d.f
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
40
60
P=0.05
3.84
5.99
7.81
9.49
11.07
12.59
14.07
15.51
16.92
18.31
19.68
21.03
22.36
23.68
25.00
26.30
27.59
28.87
30.14
31.41
32.67
33.92
35.17
36.42
37.65
38.89
40.11
41.34
42.56
43.77
55.76
79.08
P=0.01
6.63
9.21
11.34
13.28
15.09
16.81
18.48
20.09
21.67
23.21
24.73
26.22
27.69
29.14
30.58
32.0
33.41
34.81
36.19
37.57
38.93
40.29
41.64
42.98
44.31
45.64
46.96
48.28
49.59
50.90
63.69
88.38
22
TABLE 4 : Table of F for P = 0.05
P = 0.05
F
Variance ratio F = s21 /s22 with ν1 and ν2 degrees of freedom respectively.
ν1
ν2
6
8
10
12
14
16
18
20
30
40
60
∞
23
1
5.99
5.32
4.96
4.75
4.60
4.49
4.41
4.35
4.17
4.08
4.00
3.84
2
3
5.14
4.46
4.10
3.89
3.74
3.63
3.55
3.49
3.32
3.23
3.15
3.00
4.76
4.07
3.71
3.49
3.34
3.24
3.16
3.10
2.92
2.84
2.76
2.60
4
4.53
3.84
3.48
3.26
3.11
3.01
2.93
2.87
2.69
2.61
2.53
2.37
5
6
4.39
3.69
3.33
3.11
2.96
2.85
2.77
2.71
2.53
2.45
2.37
2.21
4.28
3.58
3.22
3.00
2.85
2.74
2.66
2.60
2.42
2.34
2.25
2.10
8
4.15
3.44
3.07
2.85
2.70
2.59
2.51
2.45
2.27
2.18
2.10
1.94
12
4.00
3.28
2.91
2.69
2.53
2.42
2.34
2.28
2.09
2.00
1.92
1.75
∞
24
3.84
3.12
2.74
2.51
2.35
2.24
2.15
2.08
1.89
1.79
1.70
1.52
3.67
2.93
2.54
2.30
2.13
2.01
1.92
1.84
1.62
1.51
1.39
1.00
ν2
6
8
10
12
14
16
18
20
30
40
60
∞
TABLE 5 : Critical values of R for the Mann-Whitney rank-sum test
The pairs of values below are approximate critical values of R for two-tailed tests at levels
P = 0.10 (upper pair) and P = 0.05 (lower pair). (Use relevant P = 0.10 entry for one-tailed
test at level 0.05).
smaller sample
size n1
4
5
6
7
larger sample size, n2
4
5
6
7
12,24 13,27 14,30 15,33
11,25 12,28 12,32 13,35
19,36 20,40 22,43
18,37 19,41 20,45
28,50 30,54
26,52 28,56
39,66
37,68
8
9
10
9
8
16,36
14,38
23,47
21,49
32,58
29,61
41,71
39,73
52,84
49,87
9
17,39
15,41
25,50
22,53
33,63
31,65
43,76
41,78
54,90
51,93
66,105
63,108
10
18,42
16,44
26,54
24,56
35,67
33,69
46,80
43,83
57,95
54,98
69,111
66,114
83,127
79,131
24
TABLE 6 : Critical values for T in the Wilcoxon Matched-Pairs Signed-Rank test .
The values below are the approximate critical values of T for two-tailed tests at level P. For
a significant result, the calculated T must be less than or equal to the tabulated value.
(Values of P are halved for one-tailed tests using R− and R+ .)
n
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
P = 0.10
2
2
3
5
8
10
14
17
21
26
30
36
41
47
53
60
67
75
83
91
100
10
P = 0.05
0
2
3
5
8
10
13
17
21
25
29
34
40
46
52
58
65
73
81
89