Asymptotic Properties of MLEs
Main Result1 : Let X1 , X2 , . . . , Xn be iid with common pmf/pdf f (x|θ). Let θ̂n ≡
MLE of θ ∈ Θ ⊂ R based on X1 , X2 , . . . , Xn . Then, under Cramér-Rao-type regularity
conditions: as n → ∞,
1. (Consistency)
p
θ̂n −→ θ
2. (Asymp. normality)
3. (Asymp. efficiency)
√
µ
d
n(θ̂n − θ) −→ N
¶
µ
¶2
1
d log f (X1 |θ)
0,
, I1 (θ) = Eθ
I1 (θ)
dθ
The sequence of estimators {θ̂n } is asymptotically efficient.
Discussion:
Remarks:
• The regularity conditions required for the validity of the Theorem above hold for
distributions in the one-parameter exponential family: Binomial(∗, θ), Negative
Binomial(∗, θ), Poisson(θ), N (∗, θ), N (θ, ∗), Gamma(∗, θ), Gamma(θ, ∗), etc., where
∗ indicates a given/known second parameter value.
• The following “Delta Method” result, combined with the Theorem above, is useful
for finding the distribution of functions of θ̂n , e.g., g(θ̂n ). Because the result has
wide applicability, we state it in a general form.
Delta Method: For a sequence {Tn } of real-valued random variables, suppose it holds
¡
¢
√
d
that n(Tn − a) −→ N 0, σ 2 (a) , for some a ∈ R. Then for any function g : R → R
that is continuously differentiable with derivative g 0 (a) 6= 0 at a, then as n → ∞
´
³
´
√ ³
d
n g(Tn ) − g(a) −→ N 0, [g 0 (a)]2 σ 2 (a)
1
See Bickel and Doksum for details
1
Example: Let X1 , X2 , . . . , Xn be iid Poisson(θ), θ > 0.
1. Show that Tn ≡MLE of γ(θ) = θ/(1 + θ2 ) is consistent.
¢
√ ¡
2. Find the limiting distribution of n Tn − γ(θ) .
2