DCP3352 Numerical Methods Final Exam, 06/09/2014
Please write you name and student ID on your answer sheet!
Exam time: 10:10 AM 11:59 AM
1. Please derive a two-point Gaussian quadrature rule on the interval
[−1, 1]:
Z1
f (x)dx ≈ w1 f (x1 ) + w2 f (x2 )
−1
(a) (6%) Write down the nonlinear equations w1 , w2 , x1 , x2 of use the
method of undetermined coefficients
Solution:
Z1
3
f (x) = x :
x3 dx = 0 = w1 x31 + w2 x32
−1
Z1
2
f (x) = x :
x2 dx=
2
= w1 x21 + w2 x22
3
−1
Z1
f (x) = x :
xdx = 0 = w1 x1 + w2 x2
−1
Z1
f (x) = 1 :
dx
= 2 = w1 + w2
−1
(b) (9%) Solve the nonlinear system in (a)
Solution:
Multiplying the third equation by x21 , and subtracting from the
first, we have
0 = 0 + w2 (x32 − x2 x21 )
= w2 (x2 )(x2 − x1)(x2 + x1)
1
Therefore, w2 = 0, x2 = 0, x1 = x2 , orx1 = −x2
If w2 = 0, then w1 = 2. Plugging w1 , w2 into the third equation,
and we get x1 = 0 which contradicts the second equation.
2
If x2 = 0, then w1 x21 = and w1 x1 = 0, contradiction.
3
If x1 = x2 , then w1 x1 + w2 x2 = x1 (w1 + w2 ) = 0. We get x1 =
x2 = 0 which contradicts the second equation.
If x1 = −x2 , then we get w1 − w2 = 0 from the second equation.
With the fourth equation, we get w1 = w2 = 1. Plugging w1 , w2
2
into the third equation, and we get 2x21 = with x1 = −x2 . It
3
r
1
= −x2 .
implies x1 =
3
Finally, we find that
w1 = w2 =1
r
x1 = −x2 =
1
3
2. (5%) For each of the following properties, state which type of quadrature, Newton-Cotes or Gaussian, more accurately fits the description:
(a) Easier to compute nodes and weights
(b) Has maximal polynomial degree for the number of nodes
(c) Nodes easy to reuse as order of rule changes
(d) Easier to apply for a general interval [a, b]
(e) More accurate for the same number of nodes
Solution:
(a) Newton (b) Gaussian (c) Newton (d) Newton (e) Gaussian
3. (5%) Relative to other methods for numerical quadrature, why is the
Monte Carlo method more effective in higher dimensions than in low
dimensions?
Solution:
Efficiently sampling the domain of integral function.
4. Solve y 00 + 3y 0 + 2y = 0 using the backward (implicit) Eulers method.
2
(a) (5%) Express the second order ODE as a first-order system of
ODEs
Solution:
Assume y = y1 , y 0 = y2 , and we get
0 y1
y2
=
y20
−3y2 − 2y1
(b) (5%) Carry out one iteration of backward Eulers method applied
to the system of ODEs with initial conditions y(0) = 1, y(0) = 2
and step size h = 0.01.
Solution:
x1 = x0 + hf (x1 )
y1
y1 (0)
y2
=
+h
y2
y2 (0)
−3y2 − 2y1
1
y2
=
+ 0.01
2
−3y2 − 2y1
y1
1
y2
=
+ 0.01
y2
2
−3y2 − 2y1
5250 1.01922
y1
= 5151
9900 =
1.92196
y2
5151
5. (6%) For each of the following properties, state which type of ODE
method, multistep or classical Runge-Kutta, more accurately fits the
description
(a) Self starting
(b) More efficient in attaining high accuracy
(c) Can be efficient for stiff problems
(d) Easier to program
(e) Easier to change stepsize
3
(f) Easier to obtain a local error estimate
Solution:
(a) RK (b) Multistep (c) Multistep (d) RK (e) RK (f) Multistep
6. Please derive the 3rd-step Adams method for solving the ODE
dx(t)
= f (x, t)
dt
(a) (4%) Formulate the ODE solving problem as a numerical quadrature problem
Solution:
dx
= f (x, t)
dt
⇒ dx = f (x, t)dt
tZn+1
⇒
dx = x(tn+1 ) − x(tn )
tn
tZn+1
=
f (x, t)dt
tn
(b) (10%) Apply the method of undetermined coefficients to (a) to
obtain the following
xn+1 − xn ≈
h
(5fn−2 − 16fn−1 + 23fn )
12
Solution:
tZn+1
f (x, t)dt ≈ c0 f (xn−2 , tn−2 ) + c1 f (xn−1 , tn−1 ) + c2 f (xn , tn )
tn
x is a function of t
tZn+1
f (t)dt ≈ c0 fn−2 + c1 fn−1 + c2 fn
tn
4
Interpolate 3 points (0, fn ),(−h, fn−1 ),(−2h, fn−2 )
Zh
f (t) = 1 :
dt =h =c0 (1) + c1 (1) + c2 (1)
0
Zh
f (t) = t :
tdt =
h2
=c0 (−2h) + c1 (−h) + c2 (0)
2
t2 dt=
h3
=c0 (−2h)2 + c1 (−h)2 + c2 (0)
3
0
f (t) = t2 :
Zh
0

   
h
1
1 1
c0
−2h −h 0 c1  =  h2 
2
h3
4h2 h2 0 c2
3
Solving linear system
   5h 
c0
12
c1  =  −4h 
3
23h
c2
12
xn+1 − xn ≈ c0 fn−2 + c1 fn−1 + c2 fn
h
= (5fn−2 − 16fn−1 + 23fn )
12
7. (10%) Suppose we want to use the finite difference method to solve the
following boundary value problem:
u00 − (1 − x5 )u = x with boundary conditions u(1) = 2 and u(3) = −1
with four subintervals (h=0.5). Please list the system equation for the
by replacing the second-order and first-order derivative with the finite
2
i +ui−1
difference approximation, u00 = ddxu2 = ui+1 −2u
and u0 = du
=
h2
dx
ui+1 −ui−1
2h
Solution:
Label the nodes as x1 = 1.5,x2 = 2.0,x3 = 2.5. The endpoints are x0
and x4 . The equation becomes
ui+1 − 2ui + ui−1
xi
− (1 − )ui = xi
2
h
5
5
which i = 1, 2, 3 can be rearranged into
ui−1 − (2 + h2 (1 −
xi
))ui + ui+1 = h2 xi
5
Substitute h = 0.5, substitute the x-values at the nodes, and substitute
the u-values at the endpoints and arrange in matrix form, which gives

  

−2.175
1
0
u1
−1.625
 1
−2.150
1  u2  =  0.5 
0
1
−2.125 u3
1.625
8. (15%) Consider the function: f (x, y) = 4x + 2y + x2 − 2x4 + 2xy + 3y 2 .
If the steepest descent method is used from the starting point (x, y) =
(0, −2), in what direction does it move (5%) and to what point before
changing direction (5%)? What is the direction of the second movement (5%)?
Solution:
∂f ∂x
∂f
∂y
5f =
4 + 2x − 8x3 + 2y
=
2 + 2x + 6y
(a)
4 + 0 − 0 + 2(−2)
− 5 f (0, −2) = −
2 + 0 + 6(−2)
0
=
10
(b)
0
0
x1 =
+α
−2
10
0
=
−2 + 10α
6
4 + 2(−2 + 10α)
5f (x1 ) =
2 + 6(−2 + 10α)
20α
=
−10 + 60α
5f (x1 )T · 5f (x0 ) = 0
10(−10 + 60α) = 0
1
α=
6
0
x1 =
−2 + 10α
0
= −1
3
(c)
−1
4 + 0 − 0 + 2( −1
)
3
− 5 f (0,
)=−
)
2 + 0 + 6( −1
3
3
−10 = 3
0
9. (15%) Let f : Rn → R be given by f (x) = 21 xT Ax − xT b + c, where A
is an n × n symmetric positive definite matrix, b is an n-vector, and c
is a scalar.
(a) (7%)Show that Newton’s method for minimizing this function converges in one iteration from any starting point x0 . (Hint: Newton’s method for solving a nonlinear function g(x) = 0 is given by
−1
xk+1 = xk − J−1
g (xk )g(xk ), where Jg (x) is the Jacobian matrix
of g(xk )
Solution:
Because A is symmetric,
g(x) = 5f (x) = Ax − b,
7
Jg (x) = A
Substitute g and J into Newton’s iterative equation,
xk+1 = xk − J−1
g (xk )g(xk )
= xk − A−1 (Axk − b)
Therefore,
x1 = x0 − A−1 (Ax0 − b)
= A−1 b
= x∗
(b) (8%) If the steepest descent method is used on this problem, what
happens if the starting value x0 is such that x0 − x∗ is an eigenvector of A, where x∗ is the solution (minimum of f (x)). Please
explain or prove your answer. Steepest decent method in this case
is,
ri = b − Axi
rTi ri
rTi Ari
= xi + αri
αi =
xi+1
Solution:
Explanation:
If we start the steepest descent method from any point on one
of its eigenvector, the negative gradient vector’s direction coincidentally toward the center, i.e. the minimum. Therefore, it will
converge within one iteration.
Proof:
Let eigenvector ei = xi − x∗ with eigenvalue λ and rearrange the
8
residual
ri = b − Axi
= b − A(ei + x∗ )
= b − A(ei + A−1 b)
= −Aei
= −λei
Then, rearrange the iterative equation
xi+1 = xi + αri
xi+1 − x∗ = xi − x∗ + αri
ei+1 = ei + αri
Finally, plug residual and α into the equation above
ei+1 = ei + αri
rTi ri
ri
rTi Ari
(−λei )T (−λei )
(−λei )
= ei +
(−λei )T A(−λei )
(−λei )T (−λei )
(−λei )
= ei +
(−λei )T A(−λei )
eT ei
= ei + T i
(−λei )
ei Aei
eT ei
= ei + T i
(−λei )
ei (λei )
= ei − ei
=0
= ei +
Thus, we can get instant convergence if the e0 is the eigenvector
of A.
10. (10%) Please describe the basic idea of conjugate gradient method and
compare it with the steepest descent method.
Solution:
9
(a) CG generates sequence of conjugate search directions, implicitly
accumulating information about Hessian matrix.
(b) Searching direction and convergence speed.
CG avoids repeated searches by modifying the gradient at each
step to remove components in previous directions.
Theoretically, CG is exact after at most n iterations for a quadratic
objective function in n dimensions
11. (10%) True or False
(a) The error amplification factor (growth factor) of a numerical ODE
solver depends on the step size and the Jacobian of the function
f in x = f (x, t).
(b) Because Simpsons 1/3 rule approximates the function to be integrated by a 2nd-order polynomial function, it has lower order of
accuracy than Simpsons 3/8 rule, which approximates the function by a 3rd-order polynomial function.
(c) Runge-Kutta method is a multi-step method.
(d) The shooting method solves a boundary value problem by converting it as an initial value problem.
(e) For minimizing a real valued function of several variables, the
steepest descent method is usually more rapidly convergent than
Newton’s method.
Solution:
(a) True.
(b) False.
(c) False.
(d) True.
(e) False.
10