University of Edinburgh
School of Mathematics
Statistical Methodology
Workshop 2: MLE, Tests, CI and numerical methods for a single parameter
Semester 1, 2024-25
Cauchy distribution
Consider a random sample y1 , y2 , . . . , yn from a Cauchy distribution with probability density function
1
(−∞ < y < ∞),
f (y; θ) =
π{1 + (y − θ)2 }
where θ is an unknown parameter. θ is a location parameter.
1. ML equation
b =
Find an equation satisfied by the maximum likelihood estimator (MLE) of θ (i.e. U (θ)
0). Can you find the MLE, θb analytically?
2. Asymptotic variance
Show that the expected information (Fisher information) based on the sample is n2 . Deduce an expression for the asymptotic variance of the maximum likelihood estimator of θ
and hence its estimated standard error (ESE).
Hint. You can use
Z ∞
1
1
dy = ,
2
2
2
−∞ π(1 + (y − θ) )
Z ∞
3
1
dy = ,
2
3
8
−∞ π(1 + (y − θ) )
Z ∞
1
2(y − θ)2
dy = ,
2
3
4
−∞ π(1 + (y − θ) )
i.e. for Yi ∼ f (y; θ),
1
1
2(Yi − θ)2
1
3
1
E
= , E
= , E
= .
2
2
2
2
2
(1 + (Yi − θ) )
2
(1 + (Yi − θ) )
8
(1 + (Yi − θ) )
4
3. Iterative MLE
A random sample is available:
y1 = 1.09, y2 = −0.23, y3 = 0.79, y4 = 2.31, y5 = −0.81.
1. Determine the maximum likelihood estimate of θ by the Fisher’s method of scoring (a suggested initial value is θ0 = 0). Use the stopping criterion |U (θr )−0| ≤ ϵ for
some small ϵ, e.g. ϵ = 10−6 . Write R code for the method. Plot the log-likelihood
function as a function of θ to check that the identified MLE corresponds to the
maximum. Advice. Plot ℓ(θ). This will give you theoretical insights and practical
intuitions.
2. You may also like to investigate other initial values, such as θ0 = 5, −5, ....
3. Discuss the (possible) use of Newton-Raphson to find the MLE; in R, you can use
function maxLik to run this method.
4. Confidence interval
Provide 95% and 99% confidence intervals for θ, based on the Wald test statistic:
q
b
b θb − θ) = θ − θ ∼ N (0, 1).
I(θ)(
√1
b
I(θ)
Note that z0.025 = 1.96 and z0.005 = 2.579.