Use: Find c1 , c2 s.t. P (c1 ≤ Q ≤ c2 ) = p, based).
Mapping: Likelihood ↔ Confidence
then algebraically isolate θ to get CI.
These replace θ in the SE with θ̂ (plug-in A 100p% likelihood interval corresponds to
Point Estimate (θ̂): a fixed number from Table 4.8.2: Gaussian & Exp CIs
an approximate 100q% confidence interval:
principle).
Model
Unknown
Pivotal Q
100p% CI
p
observed data (e.g. ȳ).
Confidence
Interval
Interpretation
G(µ, σ)
Ȳ −µ
√ ∼ G(0, 1)
µ
ȳ ± a √σn
σ/ n
Correct: “If we repeated this experiment q = P (W ≤ −2 ln p) = 2Φ
−2 ln p − 1
Point Estimator (θ̃): a random variable, σ known
G(µ, σ)
Ȳ −µ
√ ∼ t(n−1)
µ
ȳ ± b √sn
many times, approximately 100p% of the conS/ n
the rule (e.g. Ȳ ).
σ unknown
2
q
G(µ, σ)
Y
−
Ȳ
1
∗
structed intervals would contain the true θ.” where W ∼ χ (1), Φ is the standard normal
√
Sampling Distribution: the probability µ, σ unk
Y (pred)
∼ t(n−1)
ȳ ± bs 1+ n
S 1+1/n
h
i
Also: “We are 100p% confident that θ ∈ CDF.
G(µ, σ)
(n−1)S
(n−1)s
distribution of θ̃.
σ2
∼ χ2 (n−1)
, (n−1)s
σ
d
c
µ unknown
Reverse: Given q, find p:
q
q
[·, ·].”
h
i
Unbiased: E(θ̃) = θ.
G(µ, σ)
2
n−1
P
σ
same as above
s n−1
d , s
c
, then p = e−a /2 .
Set a = Φ−1 1+q
1
µ unknown
2
2
2
WRONG:
“P
(µ
∈
[L,
U
])
=
0.95.”
h
i
2
• S = n−1 (Yi − Ȳ ) is unbiased for σ :
2nȳ 2nȳ
2nȲ
2
Exp(θ)
θ
θ ∼ χ (2n)
d , c
Once data is observed, [L, U ] is fixed. θ is Lookup Table
√
E(S 2 ) = σ 2 .
How to Look Up Critical Values
LI p ln p
−2 ln p CI q
P
1
a
constant.
The
probability
is
0
or
1.
The
2
2
• MLE σ̃ = n (Yi − Ȳ ) is biased: (1) a: P (Z ≤ a) = 1+p where Z ∼ G(0, 1).
50%
−0.693
1.177
76%
2
randomness is in the interval, not θ.
2
E(σ̃ 2 ) = n−1
n σ .
26%
−1.353
1.645
90%
).
In R: a = qnorm( 1+p
Interval Coverage Probability
2
MLE σ̂ divides by n. Sample s divides
1+p
If
L,
U
are
r.v.’s
with
L
≤
U
,
and
15%
−1.897
1.948
95%
(2) b: P (T ≤ b) = 2 where T ∼ t(n−1).
by n−1.
P
(L
≤
θ)
=
1
−
a,
P
(U
≥
θ)
=
1
−
b,
then:
1+p
10%
−2.303
2.146
97%
In R: b = qt( 2 , n−1).
Computing ȳ and s from Sums
P 2
P
1−p
4%
−3.319
2.576
99%
P 2 ( yi )
(3) c and d: P (W ≤ c) =
yi
1
2 ,
yi −
ȳ = n , s2 = n−1
1%
−4.605
3.035
99.8%
n
P (L ≤ θ ≤ U ) = 1 − a − b
P (W ≤ d) = 1+p
2
Reading R(θ) Plots
2
Reference Distributions
where W ∼ χ (n−1). In R:
θ̂ = peak where R(θ) = 1.
Chi-Squared χ2 (k)
1−p
c
=
qchisq( 2 , n−1),
d
= Since {L > θ} and {U < θ} are mutually 100p% LI = set of all θ where R(θ) ≥ p.
E = k, Var = 2k. Support: (0, ∞).
qchisq( 1+p
exclusive.
2
2
Draw horizontal line at R = p, read L/R
2 , n−1).
Z
Pn∼ χ 2(1) if2Z ∼ G(0, 1).
Hypothesis Testing
(4) c1 , d1 : same but with χ2 (2n).
intersections.
5-Step
Template
Z
∼
χ
(n)
for
IID
Z
∼
G(0,
1).
i
i=1 i
c
=
qchisq( 1−p
d1
=
Larger n ⇒ narrower curve ⇒ more precise.
2 , 2n),
Additive: W1 ∼ χ2 (k1 ), W2 ∼ χ2 (k2 ) indep 1
1.
State
H
:
µ
=
µ
vs
H
:
µ
=
6
µ
.
0
0
A
0
Reading r(θ) = ln R(θ) Plots
qchisq( 1+p
2 , 2n).
=⇒ W1 + W2 ∼ χ2 (k1 + k2 ).
2.
Test
statistic
+
distribution
under
H
:
0
θ̂ = peak where r(θ) = 0.
χ2 Interval Inversion
2
Ȳ
−µ
Special: χ (2) ∼ Exp(2), so P (W ≥ w) =
σ known: D = σ/√n0 ∼ G(0, 1)
2
100p% LI: draw line at r = ln p, read endBecause
χ
is
not
symmetric,
d
(larger
cutoff)
−w/2
e
.
√
√ 0 ∼ t(n−1)
σ unknown: D = ȲS/−µ
points.
n
If W ∼ χ2 (1): P (W ≥ w) = 2[1 − Φ( w)]. goes in the lower bound denominator, and
ȳ−µ
0
E.g.:
30% LI → r = ln(0.3) = −1.204.
c
(smaller
cutoff)
goes
in
the
upper
bound
√
Student’s t(k)
3. Compute observed value: d = s/ n
Z
2
15%
LI
(≈95% CI) → r = ln(0.15) = −1.897.
denominator.
√
where Z ∼ G(0, 1), U ∼ χ (k), Key Critical Values
T =
4. p-value (two-sided): p = 2P (T ≥ |d|), CI Length → Sample Size
U/k
T ∼ t(n−1)
independent.
z.975 = 1.960, z.995 = 2.576, z.95 = 1.645
For G(µ,
√ σ) with σ known, CI length =
Symmetric about 0. Fatter tails than G(0, 1). t.975 (11) = 2.201, t.975 (15) = 2.131, R: 2*(1 - pt(abs(d), n-1))
2aσ/ n.
5. Conclusion statement (full sentence!):
As k → ∞, t(k) → G(0, 1).
t.975 (18) = 2.101
If LI has observed length U − L, equate:
k
“Since
p = [value] is [large/small], there is U − L = 2aσ/√n =⇒ n = (2aσ/(U − L))2
E(T ) = 0 (if k > 1), Var(T ) = k−2 (if t.975 (29) = 2.045, t.975 (∞) = 1.960
[no/strong] evidence against H0 . We [cannot where a = √−2 ln p (from LI cutoff p).
CI Length (Gaussian, σ known)
k > 2).
σ
Constructing Distributions from IID G(0, 1)
Length = 2a √n . Halve it to get margin of conclude / conclude] that µ differs from µ0 .” Sample Size for Binomial
Key Rules
error.
Given Z1 , . . . , Zn+1 IID G(0, 1):
Goal: P (|θ̃ − θ| ≤ d) ≥ p.
2aσ 2
2
2
2
2
Solve for n: n = L
where L is the de- • When σ is unknown, always use t(n−1),
• χ (n) = Z1 + Z2 + · · · + Zn
∗ 2
z
never G(0, 1).
Zn+1
sired length.
n≥
θ(1 − θ)
• t(n) = q P
Prediction vs Confidence
•
Two-sided:
multiply
one-tail
probability
by
d
n
1
2
Z
i=1 i
n
PI is always wider than CI because it ac- 2.
∗
√
• G(0, n) = Z1 + Z2 + · · · + Zn (or n · Z1 ) counts for both estimation uncertainty and • Missing conclusion sentence = lost marks. z = qnorm((1 + p)/2).
Worst case (θ unknown): θ(1−θ) ≤ 0.25:
• G(µ, σ 2 ) = µ + σZ1
individual observation variability.
• CI–Test link: µ0 ∈ CI ⇐⇒ fail to reject
2
For t(n): The Z in the numerator must be Table 4.8.1: Asymptotic CIs
H0 .
1 z∗
n
≥
Dist
θ̂
θ̃
Pivotal
Approx
CI
independent of those in the denominator.
• µ0 ∈
/ CI ⇐⇒ reject H0 at that significance
q
4 d
θ̂(1−θ̂)
Y
θ̂−θ
Need n+1 total.
√
level.
Bin(n, θ) ny
θ̂
±
a
n
n
θ̂(1−θ̂)/n
Width constraint: If width ≤ 2d, then
q
Likelihood-Based CIs
For χ2 (n): Use exactly n of them squared.
θ̂
Likelihood Ratio Statistic
√θ̂−θ
Poi(θ)
ȳ
Ȳ
θ̂
±
a
margin = d.
n
Pivotal Quantities
θ̂/n
Λ(θ)
=
−2
ln
R(θ)
=
−2r(θ)
Definition: A function Q(Y ; θ) of data and Exp(θ)
Always
round n UP.
θ̂−θ
√
ȳ Ȳ
θ̂ ± a √θ̂n
θ̂/ n
Asymptotically: Λ(θ) ∼ χ2 (1) (single param- If a prior estimate θ̂ is available, use θ̂(1−θ̂)
parameter whose distribution is completely
eter).
known (no unknown parameters).
a = qnorm( 1+p
instead of 0.25.
2 ). All are approximate (CLTSTAT 231 QUIZ 2 — CHAPTER 4 FOCUS
Estimates, Estimators, Sampling Dists
∗
2
2
2
2
1
1
2
P (c ≤ Q ≤ d) = p: c ≤ (n−1)S
≤d
σ2
mean, sd, var, median
For any model with E(Y ) = g(θ), Var(Y ) = Independent experiments, same θ:
(n−1)s2
(n−1)s2
fitdistr(x, "dist") (MASS), nlm(f, p)
2
≤σ ≤
=⇒
d
c
L(θ) = L1 (θ) · L2 (θ), ` = `1 + `2 .
h(θ):
E.g.:
95% CI (σ unk, n=16): qt(.975, 15)
p
Inversion flips: larger d goes in lower bound.
By CLT: Ȳ ≈ G(g(θ), h(θ)/n) for large n. Re-derive MLE from combined `.
=
2.131.
Censored Data
Binomial: θ̂ = (y1 + y2 )/(n1 + n2 ).
Asymptotic pivotal quantity:
Some obs censored at c (survived past c): 90% CI for σ 2 (n=19): qchisq(.05, 18)
P (1) P (2)
Poisson: θ̂ = ( yi + yi )/(n1 + n2 ). Failed (yi ≤ c): contribute f (yi ; θ) to L.
= 9.39,
Ȳ − g(θ)
≈ G(0, 1)
Z=p
Exact vs Approximate CIs
Censored (yi > c): contribute P (Y > c; θ) qchisq(.95, 18) = 28.87.
h(θ)/n
Exact (distribution of pivotal known ex- to L.
Ch1/Ch3 Quick
(midterm sheet)
P 2Ref P
Q 1 −yi /θ Q
1
2
actly):
−c/θ
y
−
(
yi )2 /n
s
=
Two approaches to build CI:
i
n−1
Exp: L(θ) = fail θ e
· cens e
E(aY +b) = aE(Y )+b, V (aY +b) = a2 V (Y )
(1) Algebraic: Solve −a ≤ Z ≤ a for θ • G(µ, σ) for µ: exact t(n−1) CI.
Central Limit Theorem
2
2
2
• G(µ, σ) for σ : exact χ (n−1) CI.
Y1 , . . . , Yn IID, E(Y ) = µ, V (Y ) = σ < ∞: E(Ȳ ) = µ, V (Ȳ ) = σ 2 /n
directly.
2
Quantile: k = (n+1)p. Int → y(k) . Else avg
This gives an exact (possibly asymmetric) • Exp(θ): exact χ (2n) CI.
Approximate (CLT-based, large n):
Ȳ − µ d
neighbors.
interval.
√ −
→ G(0, 1) as n → ∞
68-95-99.7: P (µ±σ) ≈ .683, P (µ±2σ) ≈
(2) Plug-in (Slutsky): Replace θ in the SE • Binomial, Poisson, Exp (asymptotic).
σ/ n
•
Custom
distributions
via
CLT.
c
.954.
with θ̂, then θ̂ ± a · SE.
Justifies
all
asymptotic
pivotal
quantities.
•
Likelihood-based
CIs.
Gaussian: IQR = 1.349σ. Exp: median
Simpler but approximate.
When σ depends on θ: standardize as
Always prefer exact when available.
= 0.693θ.
2
E.g.: If E(Y ) = 4θ, V (Y ) = 10θ , θ̂ = ȳ/4: Invariance Property of MLEs
Ȳ −g(θ)
√
Z
=
≈
G(0,
1),
solve
for
θ.
Linear: ū = a+bȳ, su = |b|sy , g2 invariant.
Ȳ −4θ
h(θ)/n
Z= √
≈ G(0, 1)
MLE of g(θ) is g(θ̂). Just plug in θ̂!
θ 10/n
Study types: Survey, Obs (assoc only),
Distribution Quick Reference
c = √ȳ
Poi: P\
(Y =0) = e−ȳ , SD
Exp (causation OK).
ȳ
ȳ
Dist
E V
θ̂
√
√
Algebraic: θ ∈
,
−a/ȳ d
2
\
4+a 10/n 4−a 10/n
Errors:
Study (study6=target pop),
Exp:
P
(Y
>a)
=
e
,
Var
=
ȳ
Bin(n, p) np npq y/n
p
Sample
(sample6
=study), Measurement
\
Plug-in: θ̂ ± aθ̂ 10/n/4
G(0, σ): P (Y >a) = 1 − Φ(a/σ̂)
Poi(λ)
λ
λ
ȳ
(measured6
=
true).
2
Multinomial MLE (Ch 2.4)
Relative Likelihood R(θ)
P
Geo(p) 1/p q/p
1/ȳ
n!
θi = 1.
θy1 · · · θkyk ,
f (y; θ) = y1 !···y
Confounding: Z affects both X and Y .
R(θ) = L(θ)/L(θ̂) = e`(θ)−`(θ̂) .
k! 1
NB(k, p) k/p kq/p2 k/ȳ
Fix: randomize.
2
Unconstrained: θ̂i = yi /n.
Exp(θ) θ
θ
ȳ
Always 0 ≤ R ≤ 1, R(θ̂) = 1.
Traps Checklist
Restricted Parameter Space (Genetics)
2
R(θ) Formulas
G(µ, σ) µ
σ
µ̂=ȳ
y n−y
• σ unknown ⇒ use t(n−1), never G(0, 1)
If θ1 = α2 , θ2 = 2α(1−α), θ3 = (1−α)2 :
2
1−θ
χ (k)
k
2k
—
Binom: θθ̂
• χ2 CI: d (larger) in LOWER denom, c in
y2 +2y3
2y1 +y2
1−θ̂
(1−α)
L(α) ∝ α
k
t(k)
0
—
nθ̂
UPPER
k−2
` = (2y1 +y2 ) ln α + (y2 +2y3 ) ln(1−α)
Common Critical Values
en(θ̂−θ)
Poisson: θ̂θ
• CI: “confident”, not “P (θ ∈)”
2y1 +y2
y2 +2y3
S(α) = α − 1−α = 0
G(0, 1): z.90 =1.282, z.95 =1.645, z.975 =1.960,
n
• t(n) requires n+1 independent Zi ’s
z.99 =2.326, z.995 =2.576
Exp: θ̂θ en(1−θ̂/θ)
• HT: MUST write full conclusion sentence
2y1 + y2
n
α̂ =
df t.975 t.995 t.95
• p-value: ×2 for two-sided
G(0, σ): σσ̂ exp n2 (1 − σ̂ 2 /σ 2 )
2n
5 2.571 4.032 2.015
Proof R(θ) ≤ 1
• Sample size: always round UP
Verify max: `00 (α) < 0 since both terms are θ̂ maximizes L(θ), so L(θ) ≤ L(θ̂)
10 2.228 3.169 1.812
• PI wider than CI
−(·)/(·)2 .
11 2.201 3.106 1.796
• χ2 (2) ∼ Exp(2), P (W ≥ w) = e−w/2
=⇒ R(θ) = L(θ)/L(θ̂) ≤ 1. Binomial Group Testing
15
2.131
2.947
1.753
• MLE σ̂ uses 1/n; s uses 1/(n−1)
Derivation: Exp CI
Pool k people per sample, n pools total.
18 2.101 2.878 1.734
2Yi /θ ∼ χ2 (2).
• Obs study = association only
Pool neg ⇐⇒ all k healthy: pneg = (1−θ)k . Yi ∼ Exp(θ). Know
20 2.086 2.845 1.725
Additivity: Q = 2nθȲ ∼ χ2 (2n).
• Binom edge: y=0 or y=n, MLE at boundPool pos: ppos = 1 − (1−θ)k .
29 2.045 2.756 1.699
ary
Y (# positive pools) ∼ Binom(n, 1−(1−θ)k ). P (c1 ≤ Q ≤ d1 ) = p:
∞ 1.960 2.576 1.645
• “Protocol” = how units selected
c1 ≤ 2nθȲ ≤ d1
MLE of pool prob: p̂ = Y /n.
R Commands
2nȳ
2nȳ
• r ≈ 0 6= no relationship
k
=⇒
≤
θ
≤
(1 − θ̂) = 1 − Y /n
Quantile fns:
qnorm(p), qt(p, df),
d1
c1
• Invariance: MLE of g(θ) = g(θ̂)
2
Derivation: Gaussian σ CI
qchisq(p, df)
1/k
(n−1)S 2
Y
Yi ∼ G(µ, σ). Pivotal: Q =
∼ CDF
fns:
pnorm(q),
pt(q, df),
σ2
θ̂ = 1 − 1 −
n
χ2 (n−1).
pchisq(q, df)
Custom Distribution CI via CLT
Combining Likelihoods