Asymptotic Efficiency
Recall: For two unbiased estimators T and T ∗ of γ(θ), we compare the variances of the
estimators to judge their relative efficiency, i.e. RE(T, T ∗ , θ) = Varθ (T ∗ )/Varθ (T ).
Similarly, we compare large-sample variances of asymptotically unbiased estimators.
Definitions: Let {Tn∗ } and {Tn } be asymptotically unbiased for γ(θ). Then, define
1. The asymptotic relative efficiency of {Tn } with respect to {Tn∗ } at θ is defined
as
Varθ (Tn∗ )
ARE(Tn , Tn∗ , θ) = lim
, θ∈Θ
n→∞ Varθ (Tn )
2. {Tn∗ } is called asymptotically efficient if ARE(Tn , Tn∗ , θ) ≤ 1, ∨θ ∈ Θ and any
other {Tn } that is asymptotically unbiased for γ(θ).
3. The asymptotic efficiency of {Tn } is defined as AE(Tn , θ) ≡ ARE(Tn , Tn∗ , θ) if
{Tn∗ } is asymptotically efficient.
Previous example. Let X1 , X2 , . . . be iid uniform(0, θ), θ > 0. Recall
• Tn = MME of θ based on X1 , . . . , Xn = 2X̄n
• Tn∗ = MLE of θ based on X1 , . . . , Xn = max1≤i≤n Xi = X(n)
• Eθ Tn = θ
• Eθ Tn∗ =
Varθ (Tn ) =
n
θ
n+1
3θ2
n
Varθ (Tn∗ ) =
nθ2
(n + 1)2 (n + 2)
1
Asymptotic Properties of MLEs
Main Result1 : Let X1 , X2 , . . . , Xn be iid with common pmf/pdf f (x|θ). Let θ̂n ≡
MLE of θ ∈ Θ ⊂ R based on X1 , X2 , . . . , Xn . Then, under Cramér-Rao-type regularity
conditions: as n → ∞,
1. (Consistency)
p
θ̂n −→ θ
2. (Asymp. normality)
3. (Asymp. efficiency)
√
µ
d
n(θ̂n − θ) −→ N
¶
µ
¶2
1
d log f (X1 |θ)
0,
, I1 (θ) = Eθ
I1 (θ)
dθ
The sequence of estimators {θ̂n } is asymptotically efficient.
Discussion:
Remarks:
• The regularity conditions required for the validity of the Theorem above hold for
distributions in the one-parameter exponential family: Binomial(∗, θ), Negative
Binomial(∗, θ), Poisson(θ), N (∗, θ), N (θ, ∗), Gamma(∗, θ), Gamma(θ, ∗), etc., where
∗ indicates a given/known second parameter value.
• The following “Delta Method” result, combined with the Theorem above, is useful
for finding the distribution of functions of θ̂n , e.g., g(θ̂n ). Because the result has
wide applicability, we state it in a general form.
Delta Method: For a sequence {Tn } of real-valued random variables, suppose it holds
¡
¢
√
d
that n(Tn − a) −→ N 0, σ 2 (a) , for some a ∈ R. Then for any function g : R → R
that is continuously differentiable at a with derivative g 0 (a) 6= 0,
´
³
´
√ ³
d
n g(Tn ) − g(a) −→ N 0, [g 0 (a)]2 σ 2 (a)
as n → ∞
1
See Bickel and Doksum for details
2
Example: Let X1 , X2 , . . . , Xn be iid Poisson(θ), θ > 0.
1. Show that Tn ≡MLE of γ(θ) = θ/(1 + θ2 ) is consistent.
¢
√ ¡
2. Find the limiting distribution of n Tn − γ(θ) .
3