The Bonferroni Confidence Rectangle
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
1 / 23
Suppose
y = Xβ + ε,
ε ∼ N(0, σ 2 I).
Let c01 β, . . . , c0m β be m estimable functions.


c01 β
 . 
. 
We want to find an m-dimensional rectangle that contains 
 .  with
c0m β
probability at least 1 − α.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
2 / 23
We may prefer a rectangular confidence region to an ellipsoidal region
for ease of interpretation.
α2−α3
(−1.6,8.6)
(10.4,5.4)
(6,1)
(0,0)
α1−α2
(1.6,−3.4)
c
Copyright 2012
Dan Nettleton (Iowa State University)
(13.6,−6.6)
Statistics 611
3 / 23
Let Ej (α) denote the event
q
q
c0j β ∈ c0j β̂ − tn−r,α/2 σ̂ 2 c0j (X0 X)− cj , c0j β̂ + tn−r,α/2 σ̂ 2 c0j (X0 X)− cj .
From our previous result, we know
P[Ej (α)] = 1 − α
c
Copyright 2012
Dan Nettleton (Iowa State University)
∀ j = 1, . . . , m.
Statistics 611
4 / 23
Note that
P(∩m
j=1 Ej (α)) ≤ 1 − α
with strict inequality in most cases.
(An exception: c1 = · · · = cm ).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
5 / 23
We will show that
P(∩m
j=1 Ej (α/m)) ≥ 1 − α.
∴ (l1 , u1 ) × · · · × (lm , um ) is a rectangular confidence region with the
desired coverage probability, where
q
α
lj = c0j β̂ − tn−r, 2m
σ̂ 2 c0j (X0 X)− cj
q
α
uj = c0j β̂ + tn−r, 2m
σ̂ 2 c0j (X0 X)− cj
c
Copyright 2012
Dan Nettleton (Iowa State University)
and
∀ j = 1, . . . , m.
Statistics 611
6 / 23
We say the Bonferroni intervals
(lj , uj ) j = 1, . . . , m
have simultaneous coverage probability at least 1 − α ∵
P[c0j β ∈ (lj , uj )
c
Copyright 2012
Dan Nettleton (Iowa State University)
∀ j = 1, . . . , m] ≥ 1 − α.
Statistics 611
7 / 23
A collection of subsets of a sample space S is called a sigma algebra,
denoted B, if
(i) ∅ ∈ B
(ii) A ∈ B =⇒ Ac ∈ B
(iii) A1 , A2 , . . . ∈ B =⇒ ∪∞
j=1 Aj ∈ B.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
8 / 23
Given a sample space S and an associated sigma algebra B, a
probability function with domain B satisfies
(i) P(A) ≥ 0
∀A∈B
(ii) P(S) = 1
(iii) A1 , A2 , . . . ∈ B pairwise disjoint =⇒ P(∪∞
j=1 Aj ) =
c
Copyright 2012
Dan Nettleton (Iowa State University)
P∞
j=1 P(Aj ).
Statistics 611
9 / 23
It follows that ∀ set A ∈ B,
P(A) + P(Ac ) = P(A ∪ Ac ) = P(S) = 1.
∴ ∀ A ∈ B, P(A) = 1 − P(Ac ).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
10 / 23
Boole’s inequality:
For any sets A1 , . . . , Am ∈ B,
P(∪m
j=1 Aj ) ≤
c
Copyright 2012
Dan Nettleton (Iowa State University)
m
X
P(Aj ).
j=1
Statistics 611
11 / 23
Proof:
Let
B1 = A1
B2 = Ac1 ∩ A2
B3 = Ac1 ∩ Ac2 ∩ A3
..
.
Bm = Ac1 ∩ · · · ∩ Acm−1 ∩ Am .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
12 / 23
Then
Bj ⊆ Aj
∀ j = 1, . . . , m.
B1 , . . . , Bm are pairwise disjoint, and
m
∪m
j=1 Bj = ∪j=1 Aj .
Thus,
m
P(∪m
j=1 Aj ) = P(∪j=1 Bj )
m
X
=
P(Bj )
c
Copyright 2012
Dan Nettleton (Iowa State University)
j=1
≤
m
X
P(Aj ).
j=1
Statistics 611
13 / 23
Bonferroni’s Inequality:
For any sets A1 , . . . , Am ∈ B
P(∩m
j=1 Aj )
≥1−
m
X
P(Acj ).
j=1
Prove this inequality using basic results from probability theory.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
14 / 23
Proof:
m
c
P(∩m
j=1 Aj ) = 1 − P((∩j=1 Aj ) )
c
Copyright 2012
Dan Nettleton (Iowa State University)
c
= 1 − P(∪m
j=1 Aj )
m
X
≥ 1−
P(Acj ).
j=1
Statistics 611
15 / 23
Now prove that
P[∩m
j=1 Ej (α/m)] ≥ 1 − α.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
16 / 23
Proof:
P[∩m
j=1 Ej (α/m)] ≥ 1 −
c
Copyright 2012
Dan Nettleton (Iowa State University)
= 1−
= 1−
= 1−
m
X
j=1
m
X
j=1
m
X
j=1
m
X
P[Ej (α/m)c ]
(1 − P[Ej (α/m)])
(1 − (1 − α/m))
α/m
j=1
= 1 − mα/m = 1 − α.
Statistics 611
17 / 23
Under what circumstances does the Bonferroni confidence rectangle


c01 β
 . 
. 
(l1 , u1 ) × · · · × (lm , um ) give exact 1 − α coverage of 
 . ?
c0m β
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
18 / 23
Looking back at the proof of Bonferroni’s inequality, we have
P(∩m
j=1 Aj )
=1−
m
X
P(Acj )
j=1
iff
c
P(∪m
j=1 Aj ) =
c
Copyright 2012
Dan Nettleton (Iowa State University)
m
X
P(Acj ).
j=1
Statistics 611
19 / 23
c
P(∪m
j=1 Aj )
⇐⇒
P(Aci
=
∩
m
X
P(Acj )
j=1
Acj ) =
0 ∀ i 6= j.
Thus, we have
P[∩m
j=1 Ej (α/m)] = 1 − α
⇐⇒ P[Ei (α/m)c ∩ Ej (α/m)c ] = 0
∀ i 6= j.
This says that the Bonferroni confidence rectangle is exact iff the
failure of any one interval to cover its
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
20 / 23
target parameter implies that all other intervals cover their target
parameters with probability 1; i.e.,
c0j β ∈
/ (lj , uj ) =⇒ c0i β ∈ (li , ui ) ∀ i 6= j.
Thus, in practice
P[∩m
j=1 Ej (α/m)] > 1 − α.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
21 / 23
Note that if
E1 (α/m), . . . , Em (α/m)
are independent events, then
P[∩m
j=1 Ej (α/m)] =
c
Copyright 2012
Dan Nettleton (Iowa State University)
=
m
Y
P[Ej (α/m)]
j=1
m
Y
(1 − α/m) = (1 − α/m)m
j=1
> 1−α
for m > 1.
Statistics 611
22 / 23
Under independence,
1/m
P[∩m
)] =
j=1 Ej (1 − (1 − α)
c
Copyright 2012
Dan Nettleton (Iowa State University)
m
Y
P[Ej (1 − (1 − α)1/m )]
j=1
= [(1 − α)1/m ]m
= 1 − α.
Statistics 611
23 / 23