Spring 2009
Statistics 580
Assignment No.3 (40 points)
1. Use the R function steepest() (supplied on the Downloads page in the file steepest.R)
to find the minimum of the function
f (x) = (1.5 − x1 (1 − x2 ))2 + (2.25 − x1 (1 − x22 ))2 + (2.625 − x1 (1 − x32 ))2
using the starting values x0 = (1.0, 1.0)T . Note that the function golden() (that
you must write) must be called to perform the linear search instead of the R function
optimize() as used in this class example. Obtain a perspective plot of this function in
the region defined by x1 = (−4, 4) x2 = (−4, 4) . Turn in the functions you write, the
plot, and the results. Discuss the convergence properties of this algorithm in relation
to this problem. If the Hessian was available how would you suggest we should proceed
using the same starting point? (Hint: You could just try the objective function from
the example in the optimization class notes first to familiarize yourself on how steepest
descent works compared to Newton-Raphson in the class example.)
2. Write an R function nr(derivs, x0, eps,...) to perform minimization of a multivariable function using the Newton-Raphson algorithm, where derivs is an R function
derivs used by nr that you must write as specified below, x0 is a vector of starting
values, and eps is a value for convergence criterion check. Print the current iteration
number, current iterates and the function value at each iteration. Check whether the
Euclidean distance between two succesive iterates is less than a specified tolerance as
well as that the gradient is close to zero as the convergence criterion. Use nr() to find
the minimum of the function given in Problem 1 starting at x0 = (1, 1)T . Obtain a
contour plot of this function in the region defined by x1 = ((0.0, 4.0) x2 = (−1.0, 2.0) .
Turn in the functions you write, the result and the contour plot.
derivs - is a function that you must write as follows:
derivs ← f unction(x){
f←
*evaluate function value at x*
g←
*evaluate gradient at x*
h←
*evaluate Hessian at x*
list(f, g, h)
return objects of computed values as
components of a list
}
Hint: Study the functions steepest() from Problem#1 and the function newton()
used for deriving the mle’s of parameters in Rao’s genetics problem in class notes.
Within your nr() function, extract values from the list returned from derivs() as
appropriate R objects for use locally. Your nr() function must be executable in another
R workspace.
1
3. Consider the two-parameter Weibull density
f (x; λ, k) =
k−1
k x
λ λ
k
e−(x/λ) ,
x ≥ 0, k > 0, λ > 0.
In practice this distribution is used to model censored failure time data. In this problem, for simplicity, consider only complete data i.e., the case where all units are measured until failure. A data set given in Lawless (1982) gives the number of days it took
rats painted with a carcinogen to develop carcinoma:
143 164 188 188 190 192 206 209 213 216 220 227 230 234 246 265 304
where all observations are uncensored. Use your function nr() from Problem#2 and
an appropriate derivs() function, to obtain maximum likelihood estimates of k and
λ using this data set. Give an estimate of the asymptotic covariance matrix of ( k̂, λ̂).
Use the method of moments or other method to obtain starting values (k(0) , λ(0) ).
Notes for Problem 3:
(a) Remember that we need to specify the negative of the likelihood function and its
derivatives when using nr().
(b) Since the evaluation of the likelihood function involves the data vector, we need
to use the ... parameter in both nr() and derivs() to pass this additional information.
4. Use the Nelder-Mead direct search algorithm to minimize
f (x) = 1 − 2x1 − 2x2 − 4x1 x2 + 10x21 + 2x22
starting at (4, 5) using the R function optim(). Use the default tolerance values for
convergence checking and the default values for the scaling parameters alpha, beta, and
gamma set in the control list.
5. For the minimization of the function in the previous problem, perform about 30 steps
of the Nelder-Mead algorithm as described in class by hand. Draw a graph as in the
class example to showing a trace of the progress of the algorithm for about 30 steps
(i.e. upto about T25 or so) on a contour plot of the function starting with the simplex
(4, 5), (5, 5) and (5, 6). (See notes on how to obtain a contour plot with a grid for
the purpose of tracing the path using R. You may do the plotting using R as well, but
this is not required.)
6. (Bonus Problem for additional 5 points) Use the steepest descent algorithm to
minimize
f (x) = (1.5 − x1 (1 − x2 ))2 + (2.25 − x1 (1 − x22 ))2 + (2.625 − x1 (1 − x32 ))2
starting at (1, 1), with tol set to 0.01 and step size to .1 as the line search parameters,
using a C program and the GSL function gsl multimin fdfminimizer steepest descent.
Experiment with suitable values for epsabs for the gradient convergence criterion to
obtain at least 6 decimal digits of accuracy for the minima.
Due Tuesday, March 31, 2009
2