LINKÖPINGS UNIVERSITET
Institutionen för datavetenskap
Statistik, ANd
732A36 THEORY OF STATISTICS, 6 CDTS
Master’s program in Statistics and Data Mining
Spring semester 2011
Suggested solutions to examples of tasks that may appear in exam
Task 1
(a) The Beta distribution is part of the exponential family and the density function may
be written
f (x; a, b) = e(a−1) log(x)+(b−1) log(1−x)−log B(a,b)
Thus the likelihood function is
P
P
L(x; a, b) = e(a−1) log xi +(b−1) log(1−xi )−n log B(a,b)
P
P
⇒ T = ( log xi , log(1 − xi )) is minimal sufficient for (a, b)
(b) The log-likelihood function is
l(a; x) = (a − 1)
X
log xi +
X
log(1 − xi ) − n · log B(a, 2)
First derivative w.r.t. a is
X
dl
=
log xi − n ·
da
Since
Z
B(a, 2) =
d
da
{B(a, 2)}
B(a, 2)
1
xa−1 (1 − x)dx = . . . =
0
1
a(a − 1)
we get
−2a − 1
d
{B(a, 2)} = . . . = 2
da
a (a + 1)2
that gives us
X
(2a + 1)n
dl
=
log xi +
da
a(a + 1)
Further we get
2na(a + 1) − (2a + 1)n(2a + 1)
n(2a2 + 2a + 1)
d2 l
=
=
.
.
.
=
−
da2
a2 (a + 1)2
a2 (a + 1)2
which is always negative since a > 1 > 0. Thus the equation that gives the MLestimator is
X
(2a + 1)n
=−
log xi
a(a + 1)
(c)
d2 l
Ia = E − 2
da
n(2a2 + 1)
= 2
a (a + 1)2
is N µ = a, σ = p
a(a + 1)
Thus the asymptotic distribution of âM L
. Subn(2a2 + 2a + 1)
stituting âM L = 2.5 for a gives the numerical 95% confidence interval as
2.5 · 3.5
2.5 ± 1.96 · p
⇒ 2.5 ± 0.9
20 · (2 · 2.52 + 2 · 2.5 + 1)
1
(d) The likelihood function can be written
7
Y
L(a; x ) =
!
f (xi ; a, 2)
· (P (X > 0.9))3
i=1
Now,
1
Z
P (X > 0.9) =
0.9
We also have that B(a, 2) =
upon simplification
R1
0
1
xa−1 (1 − x)dx
B(a, 2)
xa−1 (1 − x)dx = 1/a − 1/(a + 1) = 1/(a(a + 1)). Thus
1 − 0.92 (0.1a + 1)
a2 (a + 1)2
P (X > 9) =
and the likelihood function simplifies to
1
L(a; x ) = 7
·
a (a + 1)7
7
Y
!a−1
! 3
7
Y
1 − 0.92 (0.1a + 1)
(1 − xi ) ·
a2 (a + 1)2
i
·
zi
1
The log-likelihood function becomes (upon simplification)
l(a; x ) = −13 log a−13 log(a+1)+(a−1)
7
X
7
X
log(1−xi )
log xi +3 log(1−0.9a (0.1a+1)+
1
1
with first derivative
7
X
dl
13
13
0.9a−1 (0.1a2 + a − 0.09)
=− −
+
log xi − 3 ·
da
a
a+1
1 − 0.9a (0.1a + 1)
1
Assuming the stationary point defines a maximum the equation that gives the MLestimate becomes
7
0.9a−1 (0.1a2 + a − 0.09) X
13
+3·
=
log xi ' −3.527
a(a + 1)
1 − 0.9a (0.1a + 1)
1
(e) b = 1 ⇒
f (x; a, 1) =
1
· xa−1 = axa−1
B(a, 1)
a why the equation for finding the MM-estimator is
Thus E(X) = a +
1
a
= x̄
a+1
This gives us
âM M =
x̄
0.6143
=
= 1.59
1 − x̄
1 − 0.6143
2
Task 2
(a) Use Lemma 2.2 on p. 14 in the textbook. The log-likelihood function with normally
distributed data where µ = 2 is
n
n
1 X
n
2
x
(xi − 2)2
l(σ ; ) = − log σ − log(2π) − 2
2
2
σ 1
2
with first derivative
n
dl
n
1 X
=− 2 + 2 2
(xi − 2)2
dσ 2
2σ
(σ ) 1
and second derivative
n
d2 l
n
1 X
(xi − 2)2
2 2 =
2 2 −
2 3
d(σ )
2(σ )
(σ ) 1
The Fisher information is
n
n
n
n
d2 l
1 X 1 X 2
n
2
E (xi − 2) =
σ =
Iσ2 = E −
=
2 2
2 2−
2 3
2 2−
2 3
d(σ )
2(σ ) (σ ) 1
2(σ ) (σ ) 1
2(σ 2 )2
and thus the first derivative may be written
n
n
dl
·
2 =
dσ
2(σ 2 )2
1X
(xi − 2)2 − σ 2
n 1
!
1 Pn (x − 2)2 is unbiased this shows that this estimator is the one that
Since σb2 = n
i
1
4
attains the CR bound and thus s2 does not attain it. However since Var (s2 ) = n2σ
−1
4
2σ
2
2
b
and Var (σ ) = n both vanishes when n → ∞ we may condlude that s will attain
the CR-bound when n → ∞. (The variances may be easily obtained by using the fact
2
c2
is χ2n−1 -distributed and nσσ2 is χ2n -distributed.)
that for normally distributed data (n−1)s
σ2
(b)
2
MSE (σb2 ) = Var (σb2 ) + Bias(σb2 ) =
2
2
= Var (c(n − 1)s2 ) + Bias(c(n − 1)s2 ) = c2 (n − 1) · 2σ 4 + σ 2 (c(n − 1) − 1) =
· · · = c2 (2σ 2 (n − 1) + σ 4 (n − 1)2 ) − c · 2σ 4 (n − 1) + σ 4 = φ(c)
Minimise:
dφ
= 2cσ 4 (2(n − 1) + (n − 1)2 ) − 2σ 4 (n − 1)
dc
3
dφ
1
=0⇒c=
dc
n+1
d2 φ
= 2σ 4 (2(n − 1) + (n − 1)2 ) > 0
dc2
Thus the smallest MSE is obtained when c = 1/(n + 1).
Task 3
(a) The likelihood function is
P
λ
L(λ; x ) = Q
xi
xi !
· e−nλ
Thus,
L(λ1 ; x )
=
L(λ0 ; x )
λ1
λ0
P xi
· e−n(λ1 −λ0 ) ≥ A
Taking logarithms:
(log λ1 − log λ0 )
X
xi − n(λ1 − λ0 ) ≥ B
With λ1 < λ0 we get the form of the best test as
X
xi ≤ C =
B + n(λ1 − λ0 )
log λ1 − log λ0
P
P
(b) Critical region is xi ≤ C. P
With size 5% we get the equation P ( xi ≤ C | λ = 2) =
0.05. Now we can use that
xi is P o(nλ) (since it is a sum of independent Poisson
variates, all with mean λ). This gives the following equation for finding the critical
region:
C
X
(2n)i −2n
·e
= 0.05
i!
i=0
(c) No, since the form of the best test is
P
xi ≤ C for λ1 < λ0 but is
P
xi ≥ C for λ0 < λ1
maxλ≤1 {L(λ; x )}
. We find the maximum values by investigating
max {L(λ;
P x )}
Q
the log-likelihood: l(λ; x ) = ( xi ) log λ − log xi ! − nλ. This function is maximised
for λ = 3 and is thus increasing when λ approaches 3 from the left.⇒
(d) Test statistic is Λ =
P
1
max {L(λ; x )} = L(1; x ) = Q
λ≤1
xi !
P
3
max {L(λ; x )} = L(3; x ) = Q
4
xi
xi
xi !
· e−n·1 =
e−3
2! · 4! · 3!
· e−n·3 =
39 · e−9
2! · 4! · 3!
e−3
' 0.002
39 · e−9
Thus −2 log Λ ' 7.8 that is compared with the χ21 -distribution to get the P -value
approx. 0.0052.
n
o
2
P
d
l
(e) Iλ = E − 2 = 12 E ( xi ) = n ⇒ Iλ̂M L = n . But λ̂M L = x̄ and thus the Wald
λ
dλ
λ
λ̂M L
n(x̄ − 1)2
test is (x̄ − 1) · n
·
(x̄
−
1)
≥
C
or
equivalently
≥ C.
x̄
x̄
⇒Λ=
Task 4
(a) Prior: N (3, 2) ⇒ φ = 3, τ = 2. Data: N (µ, 2) ⇒ σ = 2. See the textbook on p. 125.
The posterior is


s
!
r
2
2
2 2
φσ
+
nx̄τ
σ
τ
3
·
4
+
nx̄
·
4
4
·
4
=N
N 2
,
,
σ + nτ 2
σ 2 + nτ 2
4+n·4
4+n·4
Here n = 3 and x̄ = 3.4 ⇒ posterior is N (3.3, 1). Under absolute error loss µ̂B is the
median of the posterior distribution, which here coincides with the mean, i.e. 3.3.
0 if x < 3
(b) LS (µ, δ1 ) =
and LS (µ, δ2 ) ≡ 0. Thus the risk functions are
6 if x ≥ 3
Z ∞
1
3−µ
− 81 ·(x−µ)2
6· √ e
R(µ, δ1 ) =
dx = 6 · P (X ≥ 3) = 6 · 1 − Φ
2
2 2π
3
R(µ, δ2 ) ≡ 0
Thus δ2 is minimax since it always minimises the risk
R∞
1
2
· √1 e− 8 ·(x−µ) dµ and for δ2 RB ≡ 0.
(c) For δ1 RB = −∞ 6 · 1 − Φ 3−µ
2
2 2π
Task 5
50 20
1
(a) The posterior q(π; x ) is proportional to
π (1 − π)30 ·
· π(1 − π)4 ∝
B(2, 5)
20
π 21 (1 − π)34 and is thus Beta(22,35). To find the limits a and b of the credible interval
(a, b)we need to solve the equations
Z 1
1
π 21 (1 − π)34 dπ = 0.95
l B(22, 35)
Z
u
1
1
π 21 (1 − π)34 dπ = 0.05
B(22, 35)
5
34
· 134−k · (−π)k we come (upon some algebra)
By using the result (1 − π) = k=0
k
to the following defining equations for the two limits:
34
P34
34 X
(−1)k
34
·
· (1 − a22+k ) = 0.95 · B(22, 35)
k
22
+
k
k=0
34 X
(−1)k
34
·
· (1 − b22+k ) = 0.05 · B(22, 35)
k
22
+
k
k=0
(b) Use Theorem 7.3 in the textbook ⇒
q(π | x , H1 )
B = lim
= lim
π→0.4 p(π | H1 )
π→0.4
1
· π 21 (1 − π)34
B(22,35)
1
· π ( 1 − π)4
B(2,5)
=
B(2, 5)
· 0.420 · 0.630
B(22, 35)
Now B(2, 5) ' 0.0333 and B(22, 35) ' 2.12 · 10−17 (can be tedious to calculate though,
not the best illustration of what you’re supposed to cope with during the exam), and
this gives us B ' 3.8. Thus the posterior odds becomes 3.8 · 0.1 = 0.38 or approx 1
against 2.63.
R1
(c) g(xn+1 | x ) = 0 f (xn+1 ; π)q(π | x )dπ =
R1
1
· π 21 (1 − π)34 dπ =
= 0 π xn+1 (1 − π)1−xn+1 ·
B(22, 35)
B(22 + xn+1 , 26 − xn+1 )
=
B(22, 35)
Task 6
(a) Ties: In Jar 7 Brand 2 has exactly 5 hours longer time of growth so this observation
is discarded. This leaves us with 14 jars disposable. For x = 9 jars of these Brand
2 has less than 5 hours longer time of growth. H0 says that the median growth time
difference is less than or equal to 5. Under this hypothesis x should follow a Bi(15, 0.5)distribution. Since 9 is larger than 14/2=7 we have no evidence for rejecting H0 .
(b) Compute the differences (Brand 2 − (Brand 1 + 5)). Analogously with a) this gives
a zero difference for jar 7 and that one is discarded. The rank sum for the negative
differences is 74, and although there are many ties we compute the z-score for this
value to be 1.35. Thus we have no evidence for rejecting H0 here either.
(c) Well, this is up to you to discuss.
6