Stat 643 Problems, Fall 2000
Assignment 1
1. Let (H,T ) be a measurable space. Suppose that ., T and U are 5-finite positive
measures on this space with T ¥ U and U ¥ ..
.U
.T
a) Show that T ¥ . and .T
. . œ .U † . . a.s. ..
b) Show that if . ¥ U, then
..
.U
œ Š .U
.. ‹
•"
a.s. ..
2. (Cressie) Let H œ Ò • -ß -Ó for some - ž !, T be the set of Borel sets and T be
Lebesgue measure divided by 2-. For a subset of H, define the symmetric set for E as
• E œ Ö • =l= - E× and let V œ ÖE - T |E œ • E×Þ
a) Show that V is a sub 5-algebra of T .
b) Let \ be an integrable random variable. Find EÐ\lVÑÞ
3. Let H œ Ò!ß "Ó# , T be the set of Borel sets and T œ "# ? € "# . for ? a distribution
placing a unit point mass at the point Ð "# ß "Ñ and . 2-dimensional Lebesgue measure.
Consider the variable \Ð=Ñ œ =" and the sub 5-algebra of T generated by \ , V.
a) For E - T , find EÐME lVÑ œ T ÐElVÑ.
b) For ] Ð=Ñ œ =# , find EÐ] lVÑ.
4. Suppose that \ œ Ð\" ß \# ß ÞÞÞß \8 Ñ has independent components, where each \3 is
generated as follows. For independent random variables [3 µ normalÐ.ß "Ñ and
^3 µ Poisson Ð.Ñ, \3 œ [3 with probability : and \3 œ ^3 with probability " • :.
Suppose that . - Ò!ß_Ñ . Use the factorization theorem and find low dimensional
sufficient statistics in the cases that:
a) : is known to be "# , and
b) : - Ò!ß "Ó is unknown.
(In the first case the parameter space is @ œ Ö "# × ‚ Ò!ß_Ñ , while in the second it is
Ò!ß "Ó ‚ Ò!ß_Ñ.)
5. (Ferguson) Consider the probability distributions on Ðe" ß U" Ñ defined as follows. For
) œ Ð)ß :Ñ - @ œ e ‚ Ð!ß "Ñ and \ µ T) , suppose
T) Ò\ œ BÓ œ œ
Ð" • :Ñ:B•)
!
for B œ )ß ) € "ß ) € #ß ÞÞÞ
otherwise .
Let \" ß \# ß ÞÞÞß \8 be iid according to T) Þ
a) Argue that the family ÖT)8 ×)-@ is not dominated by a 5-finte measure, so that
the factorization theorem can not be applied to identify a sufficient statistic here.
b) Argue from first principles that the statistic X Ð\Ñ œ Ðmin\3 ß !\3 Ñ is
sufficient for the parameter ) .
1
c) Argue that the factorization theorem can be applied if ) is known (the first
factor of the parameter space @ is replaced by a single point) and identify a
sufficient statistic for this case.
d) Argue that if : is known, min\3 is a sufficient statistic.
6. Suppose that \ w is exponential with mean -•" (i.e. has density
0- ÐBÑ œ -expÐ • -BÑMÒB !Ó with respect to Lebesgue measure on e" , but that one
only observes \ œ \ w MÒ\ w ž "Ó. (There is interval censoring below B œ ".)
a) Consider the measure . on k œ Ö!× • Ð"ß_Ñ consisting of a point mass of 1
at 0 plus Lebesgue measure on (1,_). Give a formula for the R-N derivative of
T-\ wrt . on k .
b) Suppose that \" ß \# ß ÞÞÞß \8 are iid with the marginal distribution T-\ . Find a
2-dimensional sufficient statistic for this problem and argue that it is indeed
sufficient.
c) Argue carefully that your statistic from b) is minimal sufficient.
7. As a complement to Problem 6, consider this "random censoring" model. Suppose
that \ w is as in Problem 6. Suppose further that independent of \ w , ^ is exponential with
mean ". We will suppose that with [ œ mina\ w ß ^ b one observes
\ œ aMÒ[ œ \ w Óß [ b (one sees either \ w or a random censoring time less than \ w ). The
family of distributions of \ is absolutely continuous with respect to the product of
counting measure and Lebesgue measure on k œ Ö!ß "× ‚ e" (call this dominating
measure .).
a) Find an R-N derivative of T-\ wrt . on k .
b) Supose that \" ß \# ß ÞÞÞß \8 are iid with the marginal distribution T-\ . Find a
minimal sufficient statistic here. (Actually carefully proving minimal sufficiency
looks like it would be hard. Just make a good guess and say what needs to be
done to finish the proof.)
8. Schervish Exercise 2.8. And show that ^ is ancillary.
9. Schervish Exercise 2.13. (I think you need to assume that ÖB3 × spans e5 .)
10. Schervish Exercise 2.18.
11. Schervish Exercise 2.22. What is a first order ancillary statistic here?
12. Schervish Exercise 2.27.
2
Assignment 2
13. Schervish Exercise 2.34 (consider 8 iid observations).
14. Schervish Exercise 2.36.
15. Schervish Exercise 2.42.
16. Schervish Exercise 2.45.
17. Schervish Exercise 2.46.
18. Prove the following version of Theorem 24 (that Vardeman gave only a discrete case
argument for) À
Suppose that \ œ ÐX ß WÑ and the family c of distributions of \ on k œ g ‚ f is
dominated by the measure . œ / ‚ # (a product of 5-finite measures on g and f
)
respectively). With .T
. . Ð>ß =Ñ œ 0) Ð>ß =Ñ, let
1) Ð>Ñ œ ( 0) Ð>ß =Ñ.# Ð=Ñ and 0) Ð=l>Ñ œ
f
0) Ð>ß =Ñ
1) Ð>Ñ
Suppose that c is FI regular at )! and that 1w) Ð>Ñ œ 'f 0)w Ð>ß =Ñ. # Ð=Ñ a>. Then
M\ Ð)! Ñ
M X Ð )! Ñ
19. Let T! and T" be two distributions on k and 0! and 0" be densities of these with
respect to some dominating 5-finite measure .. Consider the parametric family of
distributions with parameter ) - Ò!ß "Ó and densities wrt . of the form
0) ÐBÑ œ Ð" • )Ñ0! ÐBÑ € )0" ÐBÑ ß
and suppose that \ has density 0) .
a) If KÐÖ!×Ñ œ KÐÖ"×Ñ œ "# , find the posterior distribution of )l\ .
b) Suppose now that K is the uniform distribution on Ò!ß "Ó. Find the posterior
distribution of )l\ .
i) What is the mean of this posterior distribution (one Bayesian way of
inventing a point estimator for :)?
ii) Consider the partition of @ into @! œ Ò!ß Þ&Ó and @" œ ÐÞ&ß "Þ!Ó. One
Bayesian way of inventing a test for H! :) - @! is to decide in favor of @! if
the posterior probability assigned to @! is at least .5. Describe as explicitly as
you can the subset of k where this is the case.
3
20. Consider the situation of problem 6 and maximum likelihood estimation of -.
a) Show that with Q œ #Ò\3 's equal to !Ó, in the event that Q œ 8 there is no
MLE of -, but that in all other cases there is a maximizer of the likelihood. Then
^ ,
argue that for any - ž !, with T- probability tending to 1, the MLE of -, say 8
exists.
b) Give a simple estimator of - based on Q alone. Prove that this estimator is
consistent for -. Then write down an explicit one-step Newton modification of
your estimator from a).
c) Discuss what numerical methods you could use to find the MLE from a) in the
event that it exists.
d) Give two forms of large sample confidence intervals for - based on the MLE
^ and two different approximations to M Ð-ÑÞ
8
"
21. Consider the situation of Problems 6 and 20. Below are some data artificially
generated from an exponential distribution.
.24, 3.20, .14, 1.86, .58, 1.15, .32, .66, 1.60, .34,
.61, .09, 1.18, 1.29, .23, .58, .11, 3.82, 2.53, .88
a) Plot the loglikelihood function for the uncensored data (the Bw values given
above). Give approximate 90% two-sided confidence intervals for - based on the
asymptotic ;# distribution for the LRT statistic for testing H! À - œ -! vs
H" À - Á -! and based on the asymptotic normal distribution of the MLE.
Now consider the censored data problem where any value less than 1 is reported as 0.
Modify the above data accordingly and do the following.
b) Plot the loglikelihood for the censored data (the derived B) values. How does
this function of - compare to the one from part a)? It might be informative to plot
these on the same set of axes.
c) It turns out (you might derive this fact) that
M" Ð- Ñ œ
expÐ • -Ñ
"
#
Œ
•ˆ" € - • expÐ • -щ Þ
#
- " • expÐ • -Ñ
Give two different approximate 90% confidence intervals for - based on the
asymptotic distribution of the MLE here. Then give an approximate 90% interval
based on inverting the LRTs of H! À - œ -! vs H" À - Á -! Þ
22. Suppose that \" ß \# ß \$ and \% are independent binomial random variables,
\3 µ binomial Ð8ß :3 Ñ. Consider the problem of testing H! À :" œ :# and :$ œ :% against
the alternative that H! does not hold.
a) Find the form of the LRT of these hypotheses and show that the log of the LRT
statistic is the sum of the logs of independent LRT statistics for H! À :" œ :# and
H! À :$ œ :% (a fact that might be useful in directly showing the ;# limit of the
LRT statistic under the null hypothesis).
b) Find the form of the Wald tests and show directly that the test statistic is
asymptotically ;# under the null hypothesis.
4
c) Find the form of the ;# (score) tests of these hypotheses and again show
directly that the test statistic is asymptotically ;# under the null hypothesis.
23. Suppose that \" ß \# ß ÞÞÞ are iid, each taking values in k œ Ö!ß "ß #× with R-N
derivative of T) wrt to counting measure on k
0) ÐBÑ œ
expÐB)Ñ
.
" € exp) € expÐ#)Ñ
a) Find an estimator of ) based on 8! œ ! MÒ\3 œ !Ó that is È8 consistent (i.e.
8
for which È8Ðs
) • )Ñ converges in distribution).
b) Find in more or less explicit form a "one step Newton modification " of your
estimator from a).
c) Prove directly that your estimator from b) is asymptotically normal with
variance "ÎM" Ð)Ñ. (With )^8 the estimator from a) and )˜ 8 the estimator from b),
3œ"
Pw8 Ðs
)8Ñ
˜)8 œ s
,
)8 •
s
ww
P8 Ð) 8 Ñ
‡
‡
and write Pw8 Ðs
) 8 Ñ œ Pw8 Ð)Ñ € Ðs
) 8 • )ÑPww8 Ð)Ñ € "# Ðs
) 8 • )Ñ# Pwww
8 Ð)8 Ñ for some )8
between s
) 8 and ).)
– - Ð!ß #Ñ the loglikelihood has a maximizer
d) Show that provided B
– • " € È • $B
– # € 'B
–€"
B
s
s
) 8 œ logŽ
• Þ
–
#Ð# • BÑ
s
– - Ð!ß #Ñ will be asymptotically
Prove that an estimator defined to be s
) 8 when B
normal with variance "ÎM" Ð)Ñ.
e) Show that the "observed information" and "expected information"
approximations lead to the same large sample confidence intervals for ). What do
s
these look like based on, say, s
)8?
By the way, a version of nearly everything in this problem works in any one parameter
exponential family.
24. Prove the discrete \ version of Theorem 30.
5
Assignment 3
25. (Optional) Consider the problem of Bayesian inference for the binomial parameter :.
In particular, for sake of convenience, consider the Uniform Ð!ß "Ñ (BetaÐ!ß " Ñ for
! œ " œ ") prior distribution.
(a) It is possible to argue from reasonably elementary principles that in this
binomial context, where @ œ Ð!ß "Ñ, the Beta posteriors have a consistency
property. That is, simple arguments can be used to show that for any fixed :! and
any % ž 0, for \8 µ binomial Ð8ß :! Ñ, the random variable
]8 œ (
:! €%
:! •%
"
:!€\8 •" Ð" • :Ñ" €Ð8•\8 Ñ•" .:
FÐ! € \8 ß " € Ð8 • \8 ÑÑ
(which is the posterior probability assigned to the interval Ð:! • %ß :! € % Ñ)
converges in :! probability to 1 as 8 p _. This problem is meant to lead you
through this argument. Let % ž 0 and $ ž 0.
i) Argue that there exists 7 such that if 8
7, ¹ B88 •
!€B8
!€" €8 ¹
•
%
3
aB8 œ !ß "ß ÞÞÞß 8Þ
8 ÑÐ" €8•B8 Ñ
w
ii) Note that the posterior variance is Ð!€Ð!"€B
€8Ñ# Ð!€" €8€"Ñ Þ Argue there is an 7
such that if 8 7w the probability that the posterior assigns to
% !€B8
%
8
Š !!€€B
" €8 • 3 ß !€" €8 € 3 ‹is at least " • $ aB8 œ !ß "ß ÞÞÞß 8.
iii) Argue that there is an 7ww such that if 8
¸ \88 • :! ¸ • $% is at least " • $ .
7ww the :! probability that
Then note that if 8 maxÐ7ß 7w ß 7ww Ñ i) and ii) together imply that the posterior
probability assigned to ˆ B88 • #$% ß B88 € #$% ‰is at least " • $ for any realization B8 .
Then provided ¸ B88 • :! ¸ • $% the posterior probability assigned to
Ð:! • %ß :! € %Ñ is also at least " • $ Þ But iii) says this happens with :!
probability at least " • $ . That is, for large 8, with :! probability at least " • $ ,
]8 " • $ Þ Since $ is arbitrary, (and ]8 Ÿ ") we have the convergence of ]8 to
1 in :! probability.
(b) Corollary 45 suggests that in an iid model, posterior densities for large 8 tend
to look normal (with means and variances related to the likelihood material). The
posteriors in this binomial problem are Beta Ð! € B8 ß " € Ð8 • B8 ÑÑ (and we can
think of \8 µ binomial Ð8ß :! Ñ as derived as the sum of 8 iid Bernoulli Ð:! Ñ
variables). So we ought to expect Beta distributions for large parameter values to
look roughly normal. To illustrate this (and thus essentially what Corollary 45
says in this case), do the following. For 3 œ Þ$ (for example ... any other value
would do as well), consider the Beta Ð! € 83ß " € 8Ð" • 3ÑÑ (posterior)
distributions for 8 œ "!ß #!ß %! and "!!. For :8 µ Beta Ð! € 83ß " € 8Ð" • 3ÑÑ
plot the probability densities for the variables
6
8
Ê 3Ð" • 3Ñ a:8 • 3b
on a single set of axes along with the standard normal density. Note that if [ has
‰
pdf 0 Ð † Ñ, then +[ € , has pdf 1Ð † Ñ œ +" 0 ˆ †•,
+ . (Your plots are translated and
rescaled posterior densities of : based on possible observed values B8 œ Þ$8.)
If this is any help in doing this plotting, I tried to calculate values of the Beta
function using MathCad and got the following:
ÐFÐ%ß )ÑÑ•" œ "Þ$# ‚ "!$ ß ÐFÐ(ß "&ÑÑ•" œ )Þ"% ‚ "!& ß
ÐFÐ"$ß #*ÑÑ•" œ #Þ#*" ‚ "!"" and ÐFÐ$"ß ("ÑÑ•" œ #Þ*'( ‚ "!#( .
26. Problem 12, parts a)-c) Schervish, page 534.
27. Consider the following model. Given parameters -" ß ÞÞÞß -R variables \" ß ÞÞÞß Þ\R
are independent Poisson variables, \3 µ PoissonÐ-3 ÑÞ Q is a parameter taking values in
Ö"ß #ß ÞÞÞß R × and if 3 Ÿ Q ß -3 œ ." , while if 3 ž Q ß -3 œ .# Þ (Q is the number of
Poisson means that are the same as that of the first observation.) With parameter vector
) œ ÐQß ." ß .# Ñ belonging to @ œ Ö"ß #ß ÞÞÞß R × ‚ Ð!ß_Ñ‚Ð!ß_Ñ we wish to make
inference about Q based on \" ß ÞÞÞß Þ\R in a Bayesian framework.
As matters of notational convenience, let W7 œ ! \3 and X œ W R .
7
3œ"
a) If, for example, a prior distribution K on @ is constructed by taking Q uniform
on Ö"ß #ß ÞÞÞß R × independent of ." exponential with mean 1, independent of .#
exponential with mean 1ß it is possible to explicitly find the (marginal) posterior
of Q given that \" œ B" ß ÞÞÞß \R œ BR . Don't actually bother to finish the
somewhat messy calculations needed to do this, but show that this is possible
(indicate clearly why appropriate integrals can be evaluated explicitly).
b) Suppose now that K is constructed by taking Q uniform on Ö"ß #ß ÞÞÞß R ×
independent of Ð." ß .# Ñ with joint density 1Ð † ß † Ñ wrt Lebesgue measure on
(0,_) ‚ (0,_). Describe in as much detail as possible a SSS based method for
approximating the posterior of Q , given \" œ B" ß ÞÞÞß \R œ BR . (Give the
necessary conditionals up to multiplicative constants, say how you're going to use
them and what you'll do with any vectors you produce by simulation.)
28. Consider the simple two-dimensional discrete distribution for ) œ Ð)" ß )# Ñ given in
the table below.
7
)#
)"
4
3
2
1
1
0
0
.2
.2
2
0
0
.1
.1
3
.2
.05
0
0
4
.1
.05
0
0
a) This is obviously a very simple setup where anything one might wish to know
or say about the distribution is easily derivable from the table. However, for sake
of example, suppose that one wished to use SSS to approximate
U œ T Ò)" Ÿ #Ó œ Þ'. Argue carefully that Gibbs Sampling (SSS) will fail here to
produce a correct estimate of U. What is it about the transition matrix that
describes SSS here that "causes" this failure?
b) Consider the very simple version of the Metropolis-Hastings algorithm where
the matrix X œ Ð>)w ß) Ñ is ") 1 (for 1 an ) ‚ ) matrix of "'s) and ) w and ) are vectors
for which the corresponding probabilities in the table above are positive. Find the
transition matrix T and verify that it has the distribution above as it invariant
distribution.
29. Suppose for sake of argument that I am interested in the properties of the discrete
distribution with probability mass function
0 ÐBÑ œ œ
|
5 |sinB
#B
!
for B œ "ß #ß á
otherwise
but, sadly, I don't know 5.
a) Describe a MCMC method I could use to approximate the mean of this
distribution.
(b) One doesn't really have to resort to MCMC here. One could generate iid
observations from 0 ÐBÑ (instead of a MC with stationary distribution 0 ÐBÑ) using
the "rejection method." (And then rely on the LLN to produce an approximate
mean for the distribution.) Prove that the following algorithm (based on iid
Uniform Ð!ß "Ñ random variables and samples from a geometric distribution) will
produce \ from the this distribution.
Step 1 Generate \ ‡ from the geometric distribution with probability mass
function
2ÐBÑ œ œ #B
!
"
for B œ "ß #ß á
otherwise
Step 2 Generate Y that is Uniform Ð!ß "Ñ
B|
Step 3 Note that 2ÐBÑ |sin
#B . If Y 2ÐBÑ •
return to Step 1.
|sinB|
#B
set \ œ \ ‡ . Otherwise
8
30. Suppose that f is a convex subset of Ò!ß_Ñ 5 . Argue that there exists a finite @
decision problem that has f as its set of risk vectors. (Consider problems where \ is
degenerate and carries no information.)
31. Consider the two state decision problem with @ œ Ö"ß #×, T" the Bernoulli ˆ "% ‰
distribution and T# the Bernoulli ˆ "# ‰distribution, T œ @ and PÐ)ß +Ñ œ MÒ) Á +ÓÞ
a) Find the set of risk vectors for the four nonrandomized decision rules. Plot
these in the plane. Sketch f , the risk set for this problem.
b) For this problem, show explicitly that any element of W‡ has a corresponding
element of W‡ with identical risk vector and vice versa.
c) Identify the set of all admissible risk vectors for this problem. Is there a
minimal complete class for this decision problem? If there is one, what is it?
d) For each : - Ò!ß "Ó, identify those risk vectors that are Bayes versus the prior
1 œ Ð:ß " • :Ñ. For which priors are there more than one Bayes rule?
e) Verify directly that the prescription "choose an action that minimizes the
posterior expected loss" produces a Bayes rule versus the prior 1 œ Ð "# ß "# ÑÞ
32. Consider a two state decision problem with @ œ Ö"ß #×, where the observable
\ œ Ð\" ß \# Ñ has iid Bernoulli ˆ "% ‰coordinates if ) œ " and iid Bernoulli ˆ "# ‰coordinates
if ) œ #. Suppose that T œ @ and PÐ)ß +Ñ œ MÒ) Á +Ó. Consider the behavioral decision
rule 9B defined by
9B ÐÖ"×Ñ œ "
if B" œ !, and
"
"
9B ÐÖ"×Ñ œ # and 9B ÐÖ#×Ñ œ #
if B" œ ".
a) Show that 9B is inadmissible by finding a rule with a better risk function. (It
may be helpful to figure out what the risk set is for this problem, in a manner
similar to what you did in problem 31.)
b) Use the construction from Result 61 and find a behavioral decision rule that is
a function of the sufficient statistic \" € \# and is risk equivalent to 9B Þ (Note
that this rule is inadmissible.)
33. Consider the squared error loss estimation of : - Ð!ß "Ñ, based on \ µ binomial
Ð8ß :Ñ, and the two nonrandomized decision rules $" ÐBÑ œ 8B and $# ÐBÑ œ "# ˆ 8B € #" ‰Þ Let
< be a randomized decision function that chooses $" with probability "# and $# with
probability "# Þ
a) Write out expressions for the risk functions of $" , $# and <.
b) Find a behavioral rule that is risk equivalent to <.
c) Identify a nonrandomized estimator that is strictly better than < or 9.
34. Suppose that @ œ @" ‚ @# and that a decision rule 9 is such that for each )# ß 9 is
admissible when the parameter space is @" ‚ Ö)# ×. Show that 9 is then admissible when
the parameter space is @.
9
35. Suppose that AÐ)Ñ ž ! a)Þ Show that 9 is admissible with loss function PÐ)ß +Ñ iff it
is admissible with loss function AÐ)ÑPÐ)ß +Ñ.
36. Problem 9, page 209 of Schervish.
10
Assignment 4
37. (Ferguson) Prove or give a counterexample: If V" and V# are complete classes of
decision rules, then V" • V# is essentially complete.
38. (Ferguson and others) Suppose that \ µ binomial Ð8ß :Ñ and one wishes to estimate
: - Ð!ß "Ñ. Suppose first that PÐ:ß +Ñ œ :•" Ð" • :Ñ•" Ð: • +Ñ# Þ
a) Show that \8 is Bayes versus the uniform prior on Ð!ß "Ñ.
b) Argue that \8 is admissible in this decision problem.
c) Show that \8 is minimax and identify a least favorable prior
Now consider ordinary squared error loss, PÐ:ß +Ñ œ Ð: • +Ñ# .
d) Apply the result of problem 35 and prove that \8 is admissible under this loss
function as well.
39. Consider a two state decision problem where @ œ T œ Ö!ß "×, T0 and T" have
respective densities with respect to a dominating 5-finite measure ., 0! and 0" and the
loss function is PÐ)ß +Ñ.
a) For K an arbitrary prior distribution, find a formal Bayes rule versus KÞ
b) Specialize your result from a) to the case where PÐ)ß +Ñ œ MÒ) Á +Ó. What
connection does the form of these rules have to the theory of simple versus simple
hypothesis testing?
40. Suppose that \ µ Bernoulli Ð:Ñ and that one wishes to estimate : with loss
PÐ:ß +Ñ œ l: • +l. Consider the estimator $ with $ Ð!Ñ œ "% and $ Ð"Ñ œ $% .
a) Write out the risk function for $ and show that VÐ:ß $ Ñ Ÿ "% Þ
b) Show that there is a prior distribution placing all its mass on Ö!ß "# ß "× against
which $ is Bayes.
c) Prove that $ is minimax in this problem and identify a least favorable prior.
41. Consider a decision problem where T) is the Normal Ð)ß "Ñ distribution on k =e" ,
T œ Ö!ß "× and PÐ)ß +Ñ œ MÒ+ œ !ÓMÒ) ž &Ó € MÒ+ œ "ÓMÒ) Ÿ &Ó.
a) If @ œ Ð • _ß &Ó • Ò'ß_Ñ guess what prior distribution is least favorable, find
the corresponding Bayes decision rule and prove that it is minimax.
b) If @ œ e" , guess what decision rule might be minimax, find its risk function
and prove that it is minimax.
42. (Ferguson and others) Consider (inverse mean) weighted squared error loss
estimation of -, the mean of a Poisson distribution. That is, let A œ Ð!ß_Ñß T œ Aß Tbe the Poisson distribution on k œ Ö!ß "ß #ß $ß ÞÞÞ× and PÐ-ß +Ñ œ -•" Ð- • +Ñ# Þ Let
$Ð\Ñ œ \Þ
a) Show that $ is an equalizer rule.
b) Show that $ is generalized Bayes versus Lebesgue measure on A.
c) Find the Bayes estimators wrt the >Ð!," Ñ priors on A.
d) Prove that $ is minimax for this problem.
11
43. Problem 14, page 340 Schervish.
44. Let R œ ÐR" ß R# ß ÞÞÞß R5 Ñ be multinomial Ð8ß :" ß :# ß ÞÞÞß :5 Ñ where !:3 œ "Þ
a) Find the form of the Fisher information matrix based on the parameter :.
b) Suppose that :3 Ð)Ñ for 3 œ "ß #ß ÞÞÞß 5 are differentiable functions of a real
parameter ) - @, and open interval, where each :3 Ð)Ñ ! and !:3 Ð)Ñ œ "Þ
Suppose that 2ÐC" ß C# ß ÞÞÞß C5 Ñ is a continuous real-valued function with continuous
first partial derivatives and define ;Ð)Ñ œ 2Ð:" Ð)Ñß :# Ð)Ñß ÞÞÞß :5 Ð)ÑÑ. Show that
the information bound for unbiased estimators of ;Ð)Ñ in this context is
5
Ð; w Ð)ÑÑ#
.
where M" Ð)Ñ œ ":3 Ð)ÑŒ log:3 Ð)Ñ• Þ
8M" Ð)Ñ
.)
3œ"
#
45. Show that the C-R inequality is not changed by a smooth reparameterization. That is,
suppose that c œ ÖT) × is dominated by a 5-finite measure . and satisfies
i) 0) ÐBÑ ž ! for all ) and B,
ii) for all B, ..) 0) ÐBÑ exists and is finite everywhere on @ § e" and
iii) for any statistic $ with E) l$ Ð\Ñl • _ for all ), E) $ Ð\Ñ can be differentiated
under the integral sign at all points of @.
Let 2 be a function from @ to e" such that 2w is continuous and nonvanishing on @. Let
( œ 2Ð)Ñ and define U( œ T) . Show that the information inequality bound obtained
from ÖU( × evaluated at ( œ 2Ð)Ñ is the same as the bound obtained from c .
46. As an application of Lemma 88 (or 88w ) consider the following. Suppose that
\ µ UÐ)" ß )# Ñ and consider unbiased estimators of # Ð)" ß )# Ñ œ )" €#)# . For )" • )"w • )#
and )" • )#w • )# , apply Lemma 88 with 1" ÐBÑ œ " •
0Ð)w ß)# Ñ ÐBÑ
"
0Ð)" ß)# Ñ ÐBÑ
and 1# ÐBÑ œ " •
0Ð)" ß)w Ñ ÐBÑ
0Ð)" ß)# Ñ ÐBÑ Þ
#
What does Lemma 88 then say about the variance of an unbiased estimator of # ()" ß )# Ñ?
(If you can do it, sup over values of )"w and )#w . I've not tried this, so am not sure how it
comes out.)
47. Problem 11, page 389 Schervish.
48. Consider the estimation problem with 8 iid T) observations, where T) is the
exponential distribution with mean ). Let Z be the group of scale transformations on
k œ Ð!ß_Ñ8 ß Z œ Ö1- l- ž !× where 1- ÐBÑ œ -BÞ
a) Show that the estimation problem with loss PÐ)ß +Ñ œ ÐlogÐ+Î)ÑÑ# is invariant
under Z and say what relationship any equivariant nonrandomized decision rule
must satisfy.
b) Show that the estimation problem with loss PÐ)ß +Ñ œ Ð) • +Ñ# Î) # is invariant
under Z , and say what relationship any equivariant nonrandomized estimator must
satisfy.
12
c) Find the generalized Bayes estimator of ) in situation b) if the "prior" has
density wrt Lebesgue measure 1Ð)Ñ œ )•" MÒ) ž !Ó. Argue that this estimator is
the best equivariant estimator in the situation of b).
13
Assignment 5
49. Problem 19, page 341 Schervish. Also, find the MRE estimator of ) under squared
error loss for this model.
50. Let 0 Ð?ß @Ñ be the bivariate probability density of the distribution uniform on the
square in Ð?ß @Ñ-space with corners at ÐÈ#ß !Ñß Ð!ß È#Ñß Ð • È#ß !Ñ and Ð!ß • È#Ñ.
Suppose that \ œ Ð\" ß \# Ñ has bivariate probability density
0 ÐBl)Ñ œ 0 ÐB" • )ß B# • ) Ñ. Find explicitly the best location equivariant estimator of )
under squared error loss. (It may well help you to visualize this problem to "sketch" the
joint density here for ) œ "(.)
51. Consider again the scenario of problem 31. There you sketched the risk set f . Here
sketch the corresponding set i .
52. Prove the following "filling-in" lemma:
Lemma Suppose that 1! and 1" are two distinct, positive probability densities defined on
an interval in e" Þ If the ratio 1" Î1! is nondecreasing in a real-valued function X ÐBÑ, then
the family of densities Ö1! | ! - Ò!ß "Ó× for 1! œ !1" € Ð" • !Ñ1# has the MLR property
in X ÐBÑ.
53. Consider the two distributions T! and T" on k œ Ð!ß "Ñ with densities wrt Lebesgue
measure
0! ÐBÑ œ " and 0" ÐBÑ œ $B# .
a) Find a most powerful level ! œ Þ# test of H! À \ µ T! vs H" À \ µ T" .
b) Plot both the !-" loss risk set f and the set i œ Ö"9 Ð)! Ñß " Ð)" Ñ× for the simple
versus testing problem involving 0! and 0" . What test is Bayes versus a uniform
prior? This test is best of what size?
c) Take the result of Problem 52 as given. Consider the family of mixture
distributions c œ ÖT) × where for ) - Ò!ß "Ó, T) œ Ð" • )ÑT! € ) T" . In this
family find a UMP level ! œ Þ# test of the hypothesis H! À ) Ÿ Þ& vs H" À ) ž Þ&
based on a single observation \ . Argue carefully that your test is really UMP.
54. Consider a composite versus composite testing problem in a family of distributions
c œ ÖT) × dominated by the 5-finite measure ., and specifically a Bayesian decisiontheoretic approach to this problem with prior distribution K under !-" loss.
a) Show that if KÐ@! Ñ œ ! then 9ÐBÑ » " is Bayes, while if KÐ@1 Ñ œ ! then
9ÐBÑ » 0 is Bayes.
b) Show that if KÐ@! Ñ ž ! and KÐ@1 Ñ ž ! then the Bayes test against K has
Neyman-Pearson form for densities 1! and 1" on k defined by
14
1! ÐBÑ œ
' 0) ÐBÑ.KÐ)Ñ
@!
KÐ@! Ñ
and 11 ÐBÑ œ
' 0) ÐBÑ.KÐ)Ñ
@1
KÐ@1 Ñ
.
c) Suppose that \ is Normal (),1). Find Bayes tests of H! À ) œ ! vs H" À ) Á !
i) supposing that K is the Normal Ð!ß 5# Ñ distribution, and
ii) supposing that K is "# ÐR € ?Ñ, for R the Normal Ð!ß 5# Ñ distribution,
and ? a point mass distribution at 0.
!
55. Fix ! - Ð!ß Þ&Ñ and - - Ð #•#
! ß !Ñ. Let @ œ Ö • "× • Ò!ß "Ó and consider the discrete
distributions with probability mass functions as below.
B
•#
•"
!
"
#
!
"
"
!
) œ •"
!
#
# •!
# •!
#
"•- ‰ˆ "
"•- ‰
"•- ‰ˆ "
‰
ˆ
ˆ
‰
a
) Á • " )- ˆ "•
•
!
!
•
!
"
•
)b!
#
"•!
"•!
#
Find the size ! likelihood ratio test of H! À ) œ • " vs H" À ) Á • ". Show that the test
9ÐBÑ œ MÒB œ !Ó is of size ! and is strictly more powerful that the LRT whatever be ).
(This simple example shows that LRTs need not necessarily be in any sense optimal.)
56. Suppose that \ is Poisson with mean ). Find the UMP test of size ! œ Þ!& of
H! À ) Ÿ % or ) 10 vs H" À % • ) • "!. The following table of Poisson probabilities
T) Ò\ œ BÓ will be useful in this enterprise.
B
)œ%
) œ "!
B
)œ%
) œ "!
!
Þ!")$
Þ!!!!
"$
Þ!!!#
Þ!(#*
"
Þ!($$
Þ!!!&
"%
Þ!!!"
Þ!&#"
#
Þ"%'&
Þ!!#$
"&
Þ!!!!
Þ!$%(
$
Þ"*&%
Þ!!('
"'
%
Þ"*&%
Þ!")*
"(
&
Þ"&'$
Þ!$()
")
'
Þ"!%#
Þ!'$"
"*
(
Þ!&*&
Þ!*!"
#!
)
Þ!#*)
Þ""#'
#"
*
Þ!"$#
Þ"#&"
##
"!
Þ!!&$
Þ"#&"
#$
""
Þ!!"*
Þ""$(
#%
"#
Þ!!!'
Þ!*%)
#&
Þ!#"(
Þ!"#)
Þ!!("
Þ!!$(
Þ!!"*
Þ!!!*
Þ!!!%
Þ!!!#
Þ!!!"
Þ!!!!
57. Suppose that \ is exponential with mean -•" Þ Set up the two equations that will
have to be solved simultaneously in order to find UMP size ! test of H! À - Ÿ Þ& or
- 2 vs H" À - - ÐÞ&ß #Ñ.
58. Consider the situation of problem 23. Find the form of UMP tests of H! À ) Ÿ )! vs
H" À ) ž )! Þ Discuss how you would go about choosing an appropriate constant to
produce a test of approximately size !. Discuss how you would go about finding an
optimal (approximately) 90% confidence set for ).
59. (Problem 37, page 122 TSH.) Let \" ß ÞÞÞß \8 and ]" ß ÞÞÞß ]7 be independent samples
from NÐ0ß "Ñ and NÐ(ß "Ñ, and consider the hypotheses H! À ( Ÿ 0 and H" À ( ž 0Þ Show
–
–
that there exists a UMP test of size !, and it rejects H! when ] • \ is too large. (If
0" • (" is a particular alternative, the distribution assigning probability 1 to the point
with ( œ 0 œ Ð70" € 8(" ÑÎÐ7 € 8Ñ is least favorable at size !.)
60. Problem 57 Schervish, page 293.
15