Problem Sheet VII 1. Let X the random variable that describes the life time of a lamp. It is known that the distribution of X is: p(x, θ) = 1 −x xe θ , x > 0, θ > 0. θ2 X1 , X2 , ..., Xn are the life times of n lamps. (a) Calculate the maximum likelihood estimator of the parameter θ. (b) In a random sample of 10 lamps, the following measurements are obtained: 1.2, 5.7, 6.01, 2.3, 8.4, 3.22, 2.8, 1.51, 7.2, 4.1 Estimate the expected value of the life time. (c) Using the data from (b) and the given distribution, estimate the P (X > 10). 2. Let X1 , . . . Xn be i.i.d. Poisson(λ), that is P(Xi = k) = e−λ λk , k! k = 0, 1, 2, . . . , i = 1, . . . , n. Find the maximum likelihood estimator λ̂ of λ. Is it unbiased? Is it efficient? Is it consistent? 3. Let X1 , . . . , Xn be i.i.d. with p.d.f. 1 f (x; θ) = (1 + θx), 2 x ∈ (−1, 1), θ ∈ (−1, 1). Find a consistent estimator of θ and show it is consistent. 4. Let X1 , . . . , Xn be i.i.d. Bernoulli random variables with P(X1 = 1) = p. Find the maximum likelihood estimator p̂M LE of p. Show that it is consistent, efficient and sufficient. 5. (Optional) Let X1 , . . . , Xn be i.i.d. samples drawn from a distribution with density ( f (x; α) = αxα−1 , 0 ≤ x ≤ 1; 0, otherwise. Find the maximum likelihood estimator α̂ of α. Is it sufficient? 6. (Optional) Let X1 , . . . , Xn be i.i.d. samples drawn from a distribution with density ( f (x; m, α) = 1 m−1 e−xm /α , α mx 0, x > 0; otherwise. Suppose that m is known. Find the maximum likelihood estimator for α and show it is sufficient for α. 1 Solutions VII 1. We have (a) We have n 1 − 1 Pn xi Y θ i=1 e xi , θ2n i=1 L(θ, ; x1 , . . . , xn ) = n n Y 1X xi + ln xi , l(θ) = −2nlnθ − θ i=1 i=1 n d 2n 1 X x̄ l(θ) = − + 2 xi = 0 =⇒ θ = . dθ θ θ i=1 2 x̄ 2 Also for the θ = : d2 8n l(θ) = − 2 < 0. 2 d θ x̄ Hence the MLE of θ is θ = x̄2 . (b) Z ∞ x E(X) = 0 1 1 −x xe θ dx = 2 θ2 θ Z ∞ x x2 e− θ dx y=x/θ = 0 θ2 θ Z ∞ y 3−1 e−y dy = θΓ(3) = 2θ, 0 because Γ(3) = 2! = 2. So the MLE of E(X) is 2θ̂ = 2 x̄2 = x̄ = 4.244. (c) P (X > 10) = Z ∞ 1 10 For θ̂ = x̄ 2 θ x xe− θ dx = 2 1 θ2 Z ∞ x xe− θ dx = 10 Z ∞ ye−y dy = ( 10/θ − 10 = 2.122: P̂ (X > 10) = ( 10 + 1)e θ̂ θ̂ = 0.051. 2. We compute L(λ; x1 , . . . xn ) = e−nλ λnx̄ /(x1 ! · · · xn !) l(λ; x1 , . . . , xn ) = −nλ + nx̄ log(λ) + constants ∂ nx̄ l(λ; x1 , . . . , xn ) = −n + ; ∂λ λ ∂2 nx̄ l(λ; x1 , . . . , xn ) = − 2 . 2 ∂λ λ Equating the first derivative to 0 we find λ̂ = x̄. This is of course unbiased, since E Xi = λ, and its variance is var(λ̂) = var(X1 ) λ = , n n since the variance is also λ. To see why the variance is indeed λ E(X 2 ) = ∞ X k=0 k2 λk e−λ k! 2 10 10 + 1)e− θ . θ = = = = ∞ X k k=1 ∞ X λk e−λ (k − 1)! (k − 1 + 1) k=1 ∞ X (k − 1) k=2 ∞ X λk e−λ (k − 1)! ∞ X λk e−λ λk e−λ + (k − 1)! k=1 (k − 1)! ∞ X λk e−λ λk e−λ + k (k − 2)! k=0 k! k=2 = λ2 ∞ X λk e−λ k! k=0 + EX = λ2 + λ, var(X) = λ2 + λ − λ2 = λ. To find Fischer’s information we use the second derivative formula # " X n ∂2 l(λ; X) = n E 2 = . In (λ) = −n E 2 ∂λ λ λ It is clear the estimator is efficient. Consistency follows easily from the law of large numbers. 3. Here maximum likelihood can run into problems. However a quick calculation shows that 1 1 x(1 + θx)dx 2 −1 Z 1 1 2 = θx d x 2 −1 θ h x3 i1 dx = 2 3 −1 θ2 θ = dx = . 23 3 Z EX = Therefore θ̂ = 3X̄ is consistent, since X̄ is consistent for θ/3. 4. The likelihood is L(p; x1 , . . . , xn ) = l(p; x1 , . . . , xn ) = ∂ l(p; x1 , . . . , xn ) = ∂p ∂2 ∂p2 n Y i=1 n X pxi (1 − p)1−xi xi log p + (1 − xi ) log(1 − p); i=1 n X i=1 l(p; x1 , . . . , xn ) = − xi 1 − xi − ; p 1−p n X xi i=1 1 − xi + . 2 p (1 − p)2 To find the MLE we set n X ∂ xi 1 − xi l(p; x1 , . . . , xn ) = − = 0, ∂p p 1−p i=1 3 where we find that nx̄ n − nx̄ = , p p which is solved by p̂ = x̄. This is of course consistent for p by the LLN since X̄ = X1 + · · · + Xn , n and E[X1 ] = p. To see it is efficient # " ∂2 l(p; X) In (θ) = −n E ∂p2 Xi 1 − Xi = nE 2 + p (1 − p)2 E Xi 1 − E Xi =n + p2 (1 − p)2 1 1−p+p n 1 + =n = . =n p (1 − p) p(1 − p) p(1 − p) On the other hand the variance is var(X̄) = var(X1 ) p(1 − p) 1 = = , n n In (θ) so it is efficient. We know efficient estimators are sufficient so we are done. 5. We compute L(α; x1 , . . . xn ) = αn (x1 · · · xn )α−1 l(α; x1 , . . . , xn ) = n log(α) + (α − 1) X log(xi ) ∂ n l(α; x1 , . . . , xn ) = + log(xi ); ∂α α ∂2 n l(α; x1 , . . . , xn ) = − 2 < 0. 2 ∂α α X Equating the first derivative to 0 we find α̂ = − P n , log(yi ) which is a maximum since the second derivative is always negative. To show sufficiency, notice first that n exp − α̂ = exp − ! n = exp(log(x1 · · · xn )) = x1 · · · xn . − log(x1n···xn ) Therefore f (x1 , . . . , xn ; α) = α n n exp − α̂ (α−1) , which is just a function of the parameters and the statistic so we are done. 4 6. We compute P m 1 n m (x1 · · · xn )m−1 e− xi /α n α 1X m l(α; x1 , . . . , xn ) = −n log(α) − xi + constants; α ∂ n 1 X m xi . l(α; x1 , . . . , xn ) = − + 2 ∂α α α L(α; x1 , . . . xn ) = Equating the derivative to 0 we find that α̂ = 1X m xi . n To see it is sufficient f (x1 , . . . , xn ; α) = α−n e−nα̂/α which is the desired factorisation. 5 mn (x1 · · · xn )m−1 ,