MATH 565X Supplementary Homework
Spring 2013
9. Carry out two iterations of the Nelder-Mead simplex method for the problem of minimizing
f (x) = cos2 x in R1 , with initial simplex vertices ∆0 = {1, 2}.
10. Carry out two iterations of the golden section search method for the problem of minimizing f (x) = cos2 x in R1 , with initial bracketing triplet {1, 1.5, 2}.
11. An important problem in signals processing amounts to finding parameters c1 , . . . cn ,
λ1 , . . . λn such that
n
X
ci e−λi t ≈ f (t)
i=1
for a given ’signal’ function f (t). Here the ≈ symbol means that the sense in which the
finite exponential sum fits the given f (t) has yet to be specified – we can’t expect equality
to hold for every t, no matter what choice of parameters is made, but we might be able to
get a good approximation over some suitable range of t values. One approach is to set this
up as a specific nonlinear least squares problem as follows. Let
x = (x1 , . . . x2n ) = (c1 , . . . cn , λ1 , . . . λn )
be the vector of parameters to be determined, let sj = f (tj ) be given ’samples’ of f for
j = 1, . . . 2n, and set
rj (x) =
n
X
−λi tj
ci e
− sj =
i=1
n
X
xi e−xi+n tj − sj
i=1
We then obtain the parameters as the solution of the nonlinear least square problem
min ||r(x)||2
x∈R2n
It is known that there is at most one solution of r(x) = 0.
a) Find a general expression for J(x).
b) Let n = 2, t = (0, .3, .6, .9), s = (2.7, 1.48, .819, .458) and use starting guess x0 =
(1, 1, 1, 2) to do one of the following.
1. Carry out two iterations of the Gauss-Newton method by paper/calculator and report
the results, including value of the residual at each step, or
2. Program the Gauss-Newton method to carry out these iterations, and report the result
and number of iterations required to reduce the residual to below 10−4 .
Many other techniques have been developed for this problem.