LINKÖPINGS UNIVERSITET
Institutionen för datavetenskap
Statistik, ANd
732A36 THEORY OF STATISTICS, 6 CDTS
Master’s program in Statistics and Data Mining
Spring semester 2011
Suggested solutions to Exam Aug 11, 2011
Task 1
(a) The likelihood function can be written
1
L(x; λ) = e− λ
⇒T =
P
P
xi −4n log λ+3
P
log xi −n log 3!
xi is minimal sufficient for λ
(b) The log-likelihood function is
X
1X
xi − 4n log λ + 3
log xi − n log 3!
λ
First derivative w.r.t. λ is
P
dl
xi 4n
= 2 −
dλ
λ
λ
P
1
that takes the value of zero when λ = 4n xi . The second derivative w.r.t. λ is
P
2 xi 4n
d2 l
=−
+ 2
dλ2
λ3
λ
P
1
that can be shown to be negative when λ = 4n xi . Thus the ML-estimate of λ is
P
1
xi
λ̂M L = 4n
l(λ; x) = −
Task 2
∞
Z
θ−2 e−y/θ dy = . . . = θ−1 e−x/θ = e−x/θ−log θ = eA(θ)B(x)+D(θ)
(a) f (x; θ) =
x
Z
(b) f (y; θ) =
y
θ−2 e−y/θ dx = . . . = θ−2 ye−y/θ (This is a gamma distribution). Find the
0
ML-estimator. The log-likelihood function is
X
1X
l(θ; x ) = −2n log θ +
log yi −
yi
θ
with first derivative
dl
2n
1 X
=− + 2
yi
dθ
θ
θ
P
1
attaining the value zero when θ = 2n
yi . The second derivattive is
d2 l
2n
2 X
=
−
yi
dθ2
θ2
θ3
P
1
that can be shown to be negative when θ = 2n
yi . Thus the
Z ∞ML-estimate is θ̂M L =
n
o
P
E(ȳ)
E(y)
ȳ
1
yi = 2 . Now, E θ̂M L = 2 = 2 and E(y) =
yθ−2 ye−y/θ dy = . . . =
2n
0
2θ. Hence θ̂M L is unbiased. The Fisher information is
2 dl
2n
2 X
2n
Iθ = E − 2 = − 2 + 3
E(yi ) = 2
dθ
θ
θ
θ
θ2
Thus Var (θ̂M L ) ≥ 2n
1
2
θ
Thus
(c) The asymptotic distribution of θ̂M L is N (θ, Iθ−1 ) = N θ, 2n
P
!
θ̂M L − θ
√
−1.96 <
< 1.96 ' 0.95
θ/ 2n
Re-arranging the terms within the inequality gives the following approximate 95%
confidence interval:
θ̂M L
θ̂M L
√ <θ<
√
1 + 1.96/ 2n
1 − 1.96/ 2n
Numerically with ȳ = 3 ⇒ θ̂M L = 1.5 and n = 100 we get the interval 1.32 < θ <
1.74. Alternatively we may also approximate further and evaluate the interval as
θ̂M L ± 1.96 · θ̂√M2nL which numerically gives the interval 1.29 < θ < 1.71.
Task 3
(a) The likelihood function is
P
L(π; x ) = (1 − π)
xi −n n
π =
π
1−π
n
P
(1 − π)
xi
The Neyman-Pearson lemma then gives the best test as
n
P
π1
xi
(1
−
π
)
1
1−π1
L(π1 ; x )
n
=
≥A
P
π0
L(π0 ; x )
(1 − π ) xi
0
1−π0
Taking logarithms and simplifying gives:
n(log π1 + log(1 − π0 ) − log π0 − log(1 − π1 )) + (
X
xi ) · (log(1 − π1 ) − log(1 − π0 )) ≥ B
Now π1 > π0 ⇒ log(1 − π1 ) − log(1 − π0 ) < 0 and the form of the best test is
X
xi ≤ C
P
P
(b) Critical region is xi ≤ C. With size 5% we get the equation P ( xi ≤ C | π0 = 0.2) =
0.05. Since the sum has the negative binomial distribution the equation for finding the
critical limit is
C X
x+n−1
(1 − 0.2)n · 0.2x = 0.05
x
i=0
L(0.2; x )
. We find the denominator
maxπ {L(π; x )}
P
by investigating the log-likelihood: l(π;P
x ) = n(log π − log(1 − π) + ( xi ) log(1 − π).
xi
dl
n
Its first derivative is dπ
= πn + 1−π
− 1−π
that attains its maximum at π = x̄. Its
(c) Since H0 is simple the test statistic is Λ =
2
2
n
d l
second derivative is dπ
2 = − π2 −
maxπ {L(π; x )} = L(x̄; x ) and
P
xi −n
(1−π)2
which is < 0 since x ≥ 1 ⇒
P
xi ≥ n. Thus
P
0.2 n
xi
(1
−
0.2)
0.8
P
n
x̄
(1 − x̄) xi
1−x̄
Λ=
P
For an symptotic comparison we may use −2 log Λ = ( xi ) · (log 0.8 − log(1 − x̄)) −
n(log x̄ − log(1 − x̄)) − n log 4 and compare with a χ21 -distribution.
Task 4
(a) f (x; θ) = e−θx+log θ and therefore belongs to the exponential family. Thus a conjuugate
family of prior distributions can be chosen as
p(θ) = eθ·α1 +α2 ·log θ+K(α1 ,α2 )
Now, if we set α1 = 1/4, α2 = 0 and K(α1 , α2 ) = log(1/4) we obtain p(θ) =
e−θ/4+log(1/4) = (1/4)e−θ/4 = g(θ)
P
P
x) is proportional to p(θ)L(θ; x) = g(θ)·e−θ xi +n log θ ∝ e−θ(1/4+ xi )+n log θ =
(b) The posterior
q(θ|x
P
xi )
θn e−θ(1/4+
. This is a gamma distribution with
P
P parameters α = n + 1 and β =
1/4 + xi and the mean is thus (n + 1)/(1/4 + xi ). Hence the Bayes’ estimator
P
x) = (n + 1)/(1/4 + xi ) = 4/12.25 ' 0.33.
under quadritic loss is θ̂B = E(θ|x
Task 5
(a) Prior: N (2, 1) ⇒ φ = 2, τ = 1. Data: N (µ, 1) ⇒ σ 2 = 1. See the textbook on p. 125.
The posterior is


s
!
r
2
2
2 2
σ
τ
2
·
1
+
10
·
2.7
·
1
1
·
1
φσ
+
nx̄τ
=N
,
,
=
N 2
σ + nτ 2
σ 2 + nτ 2
1 + 10 · 1
1 + 10 · 1
=N
29 1
,
11 11
Let L denote the lower limit of the 95% credible interval and U the corresponding
upper limit. Then
L − 29/11
P (µ < L | x ) = Φ
= 0.05
1/11
P (µ ≤ U | x ) = Φ
which yields L ' 2.49 and U ' 2.79
3
U − 29/11
1/11
= 0.95
(b) Under H1 the prior for µ is p(µ|H1 ) is N (2, (1.5)2 ) (instead of N (2, 1)). This gives the
posterior
s
!
62.75 2.25
2 · 1 + 10 · 2.7 · (1.5)2
1 · (1.5)2
,
=N
,
q (µ | x , H1 ) = N
1 + 10 · (1.5)2
1 + 10 · (1.5)2
23.5 23.5
Now use Theorem 7.3 on page 162 in the textbook:
q (µ | x , H1 )
=
µ→2 p (µ | H1 )
√
B = lim
1
e−
(2.25/23.5)·2π
1
√
1.5 2π
−
e
(2−62.75/23.5)2
2·2.25/23.5
' 0.46
(2−2)2
2·(1.5)2
Hence the posterior odds becomes Q∗ = B · Q = 0.46 · 1 = 0.46 or 1 against 2.15.
Task 6
Order the samples, put out ranks:
Obs.:
Sample:
Rank:
10
x
1
11
x
2
12
x
3
13
x
4
15
x
5
16
x
6.5
16
y
6.5
18
y
8
22
x
25
y
10
28
y
11
29
y
12
30
y
13
32
y
14
Wx = 1 + 2 + 3 + 4 + 5 + 6.5 + 9 = 30.5 ⇒ T2 = 30.5 − 7 · 8/2 = 2.5
The Z score statistics then becomes
2.5 − 7 · 7/2
Z=p
' −2.81
7 · 7 · (7 + 7 + 1)/12
Hence the hypothesis that the two samples come from populations with equal locations can
be rejected at reasonable levels (5%, 1%)
4