STAT 421 – Formula Sheet (By Lecture)
Lecture 1 – Introduction & Preliminaries
f_X(x; θ)
Identifiability: f_X(x; θ■) = f_X(x; θ■) ⇒ θ■ = θ■
Parameter space: Θ
Model: Y■ = µ + ε■, ε■ ∼ N(0, σ²)
Expected value: E[X] = ∫ x f_X(x) dx
Variance: Var(X) = E[X²] − (E[X])²
Lecture 2 – Parametric Families & Transformations
Location–Scale Family: f(x; µ, σ) = (1/σ) f■((x − µ)/σ)
Regular Exponential Class (REC): f(x; θ) = exp{p(θ)K(x) + H(x) + q(θ)}
Transformation Method: f_Y(y) = f_X(g■¹(y)) |d/dy g■¹(y)|
Multivariate Normal (MVN): f_X(x) = (2π)^(−p/2)|Σ|^(−1/2) exp[−½(x − µ)■Σ■¹(x − µ)]
Affine Transformation: Y = AX + b ⇒ Y ∼ N■(Aµ + b, AΣA■)
Conditional MVN: X■|X■ = x■ ∼ N(µ■ + Σ■■Σ■■■¹(x■ − µ■), Σ■■ − Σ■■Σ■■■¹Σ■■)
Lecture 3 – Distribution Theory for Statistics
Statistic: T(X■, …, X■)
Sampling PDF/PMF: f_T(t; θ), p_T(t; θ)
Expectation: E_θ[T] = ∫ t f_T(t; θ) dt
Variance: Var_θ(T) = E_θ[(T − E_θ[T])²]
Bias: Bias(T) = E_θ[T] − θ
Unbiasedness: E_θ[T] = θ
MGF: M_T(t) = E[e^{tT}]
Additivity (independent): M_{X+Y}(t) = M_X(t)M_Y(t)
Example: ■ ∼ N(µ, σ²/n)
Lecture 4 – Sufficiency, Completeness & Ancillarity
Factorization Theorem: T(X) sufficient ⇔ f(x; θ) = k■(T(x); θ)k■(x)
Minimal Sufficiency: f(x■; θ)/f(x■; θ) free of θ ⇔ T(x■) = T(x■)
Equivalent Statistics: T■(x■) = T■(x■) ⇔ T■(x■) = T■(x■)
Complete Statistic: E_θ[g(T)] = 0, ∀θ ⇒ g(T) = 0 (a.s.)
Fisher Information: I(θ) = E_θ[((∂/∂θ) ln f(X; θ))²]
Information Matrix: I(θ) = E_θ[−∂²/∂θ² ln f(X; θ)]
Basu’s Theorem: If T is complete and sufficient, it is independent of every ancillary statistic.
Appendix – Common Distributions
Distribution
PDF / PMF
Mean
Variance
MGF
Normal
f(x) = (1/√(2πσ²)) e^{−(x−µ)²/(2σ²)}
µ
σ²
e^{µt + ½σ²t²}
Exponential
f(x) = λe^{−λx}, x>0
1/λ
1/λ²
λ/(λ−t)
Distribution
PDF / PMF
Mean
Variance
MGF
Gamma
f(x) = (β^α x^{α−1} e^{−βx}) / Γ(α)
α/β
α/β²
(β/(β−t))^α
Bernoulli
p(x) = p^x(1−p)^{1−x}
p
p(1−p)
1 − p + pe^t
Binomial
p(x) = (n choose x)p^x(1−p)^{n−x}
np
np(1−p)
(1−p + pe^t)^n
Poisson
p(x) = e^{−λ}λ^x/x!
λ
λ
e^{λ(e^t−1)}
Uniform
f(x) = 1/(b−a), a<x<b
(a+b)/2
(b−a)²/12
(e^{tb}−e^{ta})/(t(b−a))