advertisement

Fisher Information Stat 543 Spring 2005 Definition 1 We will say that the model for X specified by f (x|θ) (either a probability mass function or a probability density) is FI (Fisher Information) Regular at θ0 ∈ Θ ⊂ Rk provided there is an open neighborhood of θ0 , say O, such that i) f (x|θ) > 0 ∀x and ∀θ ∈ O, ii) ∀x, f (x|θ) has first¯ order partials at θ0 , and¯ R P ¯ ¯ iii) 0 = x ∂θ∂ i f (x|θ)¯ (or 0 = ∂θ∂ i f (x|θ)¯ dx) . θ=θ0 θ=θ0 Definition 2 If the model for X is FI regular at θ0 and Eθ0 then the k × k matrix I(θ0 ) = Ã Ã !2 ¯ ¯ ∂ ¯ ln f (X|θ)¯ < ∞ ∀i , ∂θi θ=θ0 ! ¯ ¯ ¯ ¯ ∂ ∂ Eθ0 ln f (X|θ)¯¯ ln f (X|θ)¯¯ ∂θi θ=θ0 ∂θ j θ=θ0 is called the Fisher Information Matrix at θ0 . Theorem 3 Suppose the model for X is FI Regular at θ0 ∈ Θ ⊂ Rk and that I(θ0 ) exists and is nonsingular. If ∀x f (x|θ) has continuous second order partials in the neighborhood O, and ∀i, j 0= or 0= then Z ¯ ¯ X ∂ X ∂2 ¯ ¯ f (x|θ)¯ and 0 = f (x|θ)¯ in the discrete case ∂θ ∂θ ∂θ θ=θ θ=θ0 0 i i j x x Z ¯ ¯ ∂ ∂2 ¯ ¯ f (x|θ)¯ dx and 0 = f (x|θ)¯ dx in the continuous case, ∂θi ∂θi ∂θj θ=θ0 θ=θ0 I(θ0 ) = −Eθ0 Ã ! ¯ ¯ ∂2 ¯ ln f (X|θ)¯ ∂θi ∂θj θ=θ0 . Notation 4 In an iid model, i.e. where X = (X1 , ..., Xn ), let I1 (θ0 ) be the Fisher Information Matrix at θ0 for a single observation. That is, with marginal distribution specified by f (x|θ), Ã ! ¯ ¯ ¯ ¯ ∂ ∂ I1 (θ0 ) = Eθ0 ln f (X1 |θ)¯¯ ln f (X1 |θ)¯¯ . ∂θi θ=θ0 ∂θ j θ=θ0 Theorem 5 Suppose that X1 , X2 , ..., Xn are iid random vectors and f (x|θ) is either the marginal probability mass function or the marginal probability density. If the model for X1 is FI regular at θ0 and with X = (X1 , ..., Xn ), I(θ0 ) is the Fisher Information Matrix for X at θ0 , I(θ0 ) = nI1 (θ0 ) . 1