# Fisher Information Stat 543 Spring 2005 ```Fisher Information
Stat 543 Spring 2005
Definition 1 We will say that the model for X specified by f (x|θ) (either a probability mass function or a
probability density) is FI (Fisher Information) Regular at θ0 ∈ Θ ⊂ Rk provided there is an open neighborhood
of θ0 , say O, such that
i) f (x|θ) &gt; 0 ∀x and ∀θ ∈ O,
ii) ∀x, f (x|θ) has first&macr; order partials at θ0 , and&macr;
R
P
&macr;
&macr;
iii) 0 = x ∂θ∂ i f (x|θ)&macr;
(or 0 = ∂θ∂ i f (x|θ)&macr;
dx) .
θ=θ0
θ=θ0
Definition 2 If the model for X is FI regular at θ0 and
Eθ0
then the k &times; k matrix
I(θ0 ) =
&Atilde;
&Atilde;
!2
&macr;
&macr;
∂
&macr;
ln f (X|θ)&macr;
&lt; ∞ ∀i ,
∂θi
θ=θ0
!
&macr;
&macr;
&macr;
&macr;
∂
∂
Eθ0
ln f (X|θ)&macr;&macr;
ln f (X|θ)&macr;&macr;
∂θi
θ=θ0 ∂θ j
θ=θ0
is called the Fisher Information Matrix at θ0 .
Theorem 3 Suppose the model for X is FI Regular at θ0 ∈ Θ ⊂ Rk and that I(θ0 ) exists and is nonsingular.
If ∀x f (x|θ) has continuous second order partials in the neighborhood O, and ∀i, j
0=
or
0=
then
Z
&macr;
&macr;
X ∂
X ∂2
&macr;
&macr;
f (x|θ)&macr;
and 0 =
f (x|θ)&macr;
in the discrete case
∂θ
∂θ
∂θ
θ=θ
θ=θ0
0
i
i
j
x
x
Z
&macr;
&macr;
∂
∂2
&macr;
&macr;
f (x|θ)&macr;
dx and 0 =
f (x|θ)&macr;
dx in the continuous case,
∂θi
∂θi ∂θj
θ=θ0
θ=θ0
I(θ0 ) = −Eθ0
&Atilde;
!
&macr;
&macr;
∂2
&macr;
ln f (X|θ)&macr;
∂θi ∂θj
θ=θ0
.
Notation 4 In an iid model, i.e. where X = (X1 , ..., Xn ), let I1 (θ0 ) be the Fisher Information Matrix at
θ0 for a single observation. That is, with marginal distribution specified by f (x|θ),
&Atilde;
!
&macr;
&macr;
&macr;
&macr;
∂
∂
I1 (θ0 ) = Eθ0
ln f (X1 |θ)&macr;&macr;
ln f (X1 |θ)&macr;&macr;
.
∂θi
θ=θ0 ∂θ j
θ=θ0
Theorem 5 Suppose that X1 , X2 , ..., Xn are iid random vectors and f (x|θ) is either the marginal probability
mass function or the marginal probability density. If the model for X1 is FI regular at θ0 and with X =
(X1 , ..., Xn ), I(θ0 ) is the Fisher Information Matrix for X at θ0 ,
I(θ0 ) = nI1 (θ0 ) .
1
```