Spring 2009 Statistics 580 Assignment No.3 (40 points) 1. Use the R function steepest() (supplied on the Downloads page in the file steepest.R) to find the minimum of the function f (x) = (1.5 − x1 (1 − x2 ))2 + (2.25 − x1 (1 − x22 ))2 + (2.625 − x1 (1 − x32 ))2 using the starting values x0 = (1.0, 1.0)T . Note that the function golden() (that you must write) must be called to perform the linear search instead of the R function optimize() as used in this class example. Obtain a perspective plot of this function in the region defined by x1 = (−4, 4) x2 = (−4, 4) . Turn in the functions you write, the plot, and the results. Discuss the convergence properties of this algorithm in relation to this problem. If the Hessian was available how would you suggest we should proceed using the same starting point? (Hint: You could just try the objective function from the example in the optimization class notes first to familiarize yourself on how steepest descent works compared to Newton-Raphson in the class example.) 2. Write an R function nr(derivs, x0, eps,...) to perform minimization of a multivariable function using the Newton-Raphson algorithm, where derivs is an R function derivs used by nr that you must write as specified below, x0 is a vector of starting values, and eps is a value for convergence criterion check. Print the current iteration number, current iterates and the function value at each iteration. Check whether the Euclidean distance between two succesive iterates is less than a specified tolerance as well as that the gradient is close to zero as the convergence criterion. Use nr() to find the minimum of the function given in Problem 1 starting at x0 = (1, 1)T . Obtain a contour plot of this function in the region defined by x1 = ((0.0, 4.0) x2 = (−1.0, 2.0) . Turn in the functions you write, the result and the contour plot. derivs - is a function that you must write as follows: derivs ← f unction(x){ f← *evaluate function value at x* g← *evaluate gradient at x* h← *evaluate Hessian at x* list(f, g, h) return objects of computed values as components of a list } Hint: Study the functions steepest() from Problem#1 and the function newton() used for deriving the mle’s of parameters in Rao’s genetics problem in class notes. Within your nr() function, extract values from the list returned from derivs() as appropriate R objects for use locally. Your nr() function must be executable in another R workspace. 1 3. Consider the two-parameter Weibull density f (x; λ, k) = k−1 k x λ λ k e−(x/λ) , x ≥ 0, k > 0, λ > 0. In practice this distribution is used to model censored failure time data. In this problem, for simplicity, consider only complete data i.e., the case where all units are measured until failure. A data set given in Lawless (1982) gives the number of days it took rats painted with a carcinogen to develop carcinoma: 143 164 188 188 190 192 206 209 213 216 220 227 230 234 246 265 304 where all observations are uncensored. Use your function nr() from Problem#2 and an appropriate derivs() function, to obtain maximum likelihood estimates of k and λ using this data set. Give an estimate of the asymptotic covariance matrix of ( k̂, λ̂). Use the method of moments or other method to obtain starting values (k(0) , λ(0) ). Notes for Problem 3: (a) Remember that we need to specify the negative of the likelihood function and its derivatives when using nr(). (b) Since the evaluation of the likelihood function involves the data vector, we need to use the ... parameter in both nr() and derivs() to pass this additional information. 4. Use the Nelder-Mead direct search algorithm to minimize f (x) = 1 − 2x1 − 2x2 − 4x1 x2 + 10x21 + 2x22 starting at (4, 5) using the R function optim(). Use the default tolerance values for convergence checking and the default values for the scaling parameters alpha, beta, and gamma set in the control list. 5. For the minimization of the function in the previous problem, perform about 30 steps of the Nelder-Mead algorithm as described in class by hand. Draw a graph as in the class example to showing a trace of the progress of the algorithm for about 30 steps (i.e. upto about T25 or so) on a contour plot of the function starting with the simplex (4, 5), (5, 5) and (5, 6). (See notes on how to obtain a contour plot with a grid for the purpose of tracing the path using R. You may do the plotting using R as well, but this is not required.) 6. (Bonus Problem for additional 5 points) Use the steepest descent algorithm to minimize f (x) = (1.5 − x1 (1 − x2 ))2 + (2.25 − x1 (1 − x22 ))2 + (2.625 − x1 (1 − x32 ))2 starting at (1, 1), with tol set to 0.01 and step size to .1 as the line search parameters, using a C program and the GSL function gsl multimin fdfminimizer steepest descent. Experiment with suitable values for epsabs for the gradient convergence criterion to obtain at least 6 decimal digits of accuracy for the minima. Due Tuesday, March 31, 2009 2