::3030
-4
::pm
30
December
14
6129!!12
-9
pm
AM-12
: 00 PM
14
April
2018
TakeExam
15
2024
December
08:
:30-8
:30
pmPM
27
April
2017,
00
-pm
3PM
30pm
12
01
////Home
2023
18
///12
04
2023
December
18
2018
13
/04October,
02
/19,
2024
13
December
2017
17
2022
1519am
:5:30
:30
29
68:84:00
pm
14
9
14
129
11
12
10
10
Signal Processing
1
Numerical
Methods
xxx
ECE3780
ECE
2240
December
December
1618/ 6-December
PM
33hrs
3 Hours
December
20 /202119 / 6 PM
2hrs
PART
S
FINAL
EXAM
Midterm
Test
Deferred
Final Exam
, G.Thomas
J.Peters
A.Gole & M.Pawlak
M.Pawlak
I
II
III
IV
V
VI
10
25
15
28
20
12
Mark
Out of
TOTAL
110 POINTS = 55%
Material Allowed: Two (double sided) handwritten 8.5 by 11 inch aid sheets, calculators.
Material Not Allowed: Notebooks, tablets, cell phones.
Answer All Questions in the Provided Space. You may want to first work things through on scratch paper
and then transfer to this booklet. Only this booklet will be considered in the grading.
Academic Integrity Contract
I understand that cheating is a serious offense. As members of the University Community, Students have an
obligation to act with academic integrity. Any Student who engages in Academic Misconduct in relation to a
University Matter will be subject to discipline.
1/14
PART I: True or False
1. By Taylor’s formula, we obtain
f ( 2 ) ( x ) f ( 3) (! )
f ( x + 1) = f ( x ) + f ( x ) +
+
2
6
(1)
f ( 2 ) ( x ) f ( 3) (! )
f ( x ! 1) = f ( x ) ! f ( x ) +
!
.
2
6
(1)
So by adding, we obtain an exact expression for f ( ) ( x ) : f ( x + 1) + f ( x ! 1) ! 2 f ( x ) = f ( ) ( x ) . Is it true ?
2
2
T
F
2. The Lagrange polynomials interpolation method reveals the improved accuracy if the interpolation nodes
are not equally spaced.
T
F
3. Determining cubic splines on 4 intervals require determining 12 unknown coefficients.
T
F
4. Euler’s method can be only used for numerical solution of single nonlinear differential equations.
T
F
5. The truncation error of the difference formula f (1) ( xi ) !
T
f ( xi + h ) ! f ( xi ! h )
is of order O ( 2h ) .
2h
F
6. Assuming we are computing an integral to a given accuracy, it is usually more efficient to use the
trapezoidal rule than Simpson’s rule.
T
F
2
(
)
7. There is a non-zero error in estimating the following integral " 3x 3 ! 2x 2 + x + 1 dx by Simpson’s rule.
0
T
F
8. If a matrix A is nonsingular then the number of solutions to the linear system Ax = b depends on the
particular choice of right hand vector b .
T
F
9. A large value of the condition number ! ( A ) of a matrix A is a reliable indicator that a matrix is nearly
singular.
T
F
10. If a matrix A is nonsingular, then pivoting is unnecessary in Gaussian elimination.
T
F
2/14
PART II: Root Finding and Optimization
Problem 2.1. Consider the problem of finding a root of the following function f ( x ) = x 3 ! 2x + 2 .
A. If Newton’s method is used finding a root and x0 = 0 , what is x2 ?
B. If the secant method is used finding a root and x0 = 0 and x1 = 1 , what is x2 ?
1
A. Solution: f ( x ) = x 3 ! 2x + 2 ! f ( ) ( x ) = 3x 2 ! 2 ! xn+1 = xn !
! x1 = x0 !
f ( xn )
f (1) ( xn )
f ( x0 )
f ( x1 )
!
x
=
x
!
=
1
= 0. Hence, Newton’s method
2
1
1
f ( ) ( x0 )
f (1) ( x1 )
does not converge as it generates the cyclic sequence: 0 ! 1 ! 0 ! 1 ! ....
B. Solution: f ( 0 ) = 2 , f (1) = 1 ! xn+1 = xn ! f ( xn )
! x2 = x1 ! f ( x1 )
xn ! xn!1
f ( xn ) ! f ( xn!1 )
x1 ! x0
= 2.
f ( x1 ) ! f ( x0 )
Note: The root of f ( x ) = x 3 ! 2x + 2 is x = "1.769 . Here is the plot of f ( x ) .
!
6
4
2
0
�2
�2
�1
0
1
3/14
2
Problem 2.2. Using Newton’s method determine an approximation for the reciprocal of the square root of a
1
1
positive number, i.e.,
. Hint: Consider the function f ( x ) = x 2 ! . Perform 3 iterations of the proposed
R
R
1
method to approximate
starting with x0 = 1 .
5
Solution:
1
xn2 !
1
(1)
2
R ! x = 1 !x + 1 1 $ .
! f ( x ) = 2x ! xn+1 = xn !
1. f ( x ) = x !
" n
%
n+1
2xn
R
2#
R xn &
2. With R = 5 and x0 = 1 the above algorithm gives
3
7
47
x1 = = 0.6 , x2 =
= 0.466 , x3 =
= 0.4476 .
5
15
105
1
= 0.447214...
Note:
5
4/14
Problem 2.3. Consider the following function f ( x ) = e! x ! cos ( x ) that plot is given below.
A. Write a formula for the gradient decent method (use ! = 1 ) to find the minimum of f ( x ) . Use x0 = 1 and
find x1 .
B. Write a formula for Newton’s iterative method to find the minimum of f ( x ) . Use x0 = 1 and find x1 .
1
A. Solution: f ( x ) = e! x ! cos ( x ) ! f ( ) ( x ) = !e! x + sin ( x )
! xn+1 = xn ! f (1) ( xn ) ! with x0 = 1 we have x1 = x0 ! f (1) ( x0 ) = 0.5264 .
1
2
B. Solution: f ( x ) = e! x ! cos ( x ) ! f ( ) ( x ) = !e! x + sin ( x ) and f ( ) ( x ) = e! x + cos ( x )
f (1) ( xn )
f (1) ( x0 )
! xn+1 = xn ! ( 2 )
! with x0 = 1 we have x1 = x0 ! ( 2 )
= 0.4785 .
f ( xn )
f ( x0 )
!
( )
Note: True minimum is at x = 0.5885 , where f x ! = "0.2766 . Note that
x ! " 0.5264 = 0.062
x ! " 0.4785 = 0.11 .
Hence, the gradient decent gives the better approximation of the minimum value.
5/14
PART III: Curve Fitting
Problem 3.1 Suppose that the following table of n data points is known to conform a model y ! c log ( x ) .
Derive a formula for an estimate of c obtained by the least squares method ?
x
x1
x2
……...
xn
y
y1
y2
……...
yn
Table 1
n
Solution:
n
! ( y ! c log ( x )) " min . The derivative of ! ( y ! c log ( x )) with respect to c is
2
k
k
c
2
k
k=1
k
k=1
n
2" ( yk ! c log ( xk )) ( ! log ( xk )) .
k=1
n
n
k=1
k=1
(
)
Then, 2" ( yk ! c log ( xk )) ( ! log ( xk )) = 0 if ! yk log ( xk ) ! c ( log ( xk )) = 0 . This gives
2
n
! y log ( x )
cLS = k=1
n
k
k
! ( log ( x ))
2
.
k
k=1
Problem 3.2 Suppose that the data set in Table 1 is know to conform a nonlinear model y ! ! e" x . We wish
to apply the linear least squares approach to fit the data to the assumed model. To do so, we should apply the
least squares criterion to the following data sets (choose one)
(A) The data as in Table 1 (above)
(B)
log ( x )
log ( x1 )
log ( x2 )
……...
log ( xn )
log ( y )
log ( y1 )
log ( y2 )
……...
log ( yn )
log ( x )
log ( x1 )
log ( x2 )
……...
log ( xn )
y
y1
y2
……...
yn
x
x1
x2
……...
xn
log ( y )
log ( y1 )
log ( y2 )
……...
log ( yn )
(C)
(D)
We should use the data from table (D) since
y ! ! e" x ! log y ! ! x + log (! ) .
Hence, we have the linear regression problem
Y ! a + bX
with X = x , Y = log ( y ) , a = log (! ) , b = ! .
6/14
Problem 3.3 An overdetermined system of 3 linear equations with 2 unknowns is pictured in the plot.
y + 2x = !1
y ! 3x = !2
y! x =1
y! x =1
y + 2x = !1
y ! 3x = !2
A. How many solutions does this system have according to the classical linear algebra theory ?
B. Can you propose a generalized “solution” of this problem using the least squares approach ?
Solution: A: None
B: Form the LS criterion
LS ( x, y ) = ( 2x + y + 1) + ( !3x + y + 2 ) + ( !x + y ! 1) ! min x,y .
2
2
2
!LS ( x, y )
= 0 ! 4 ( 2x + y + 1) + 2 ( !3) ( !3x + y + 2 ) + 2 ( !1) ( !x + y ! 1) = 0
!x
!LS ( x, y )
= 0 ! 2 ( 2x + y + 1) + 2 ( !3x + y + 2 ) + 2 ( !x + y ! 1) = 0
!y
! x=
5
11
= 0.1315 , y = ! = !0.578
38
19
7/14
PART IV: Interpolation
Problem 4.1 Suppose you have the following distinct data points
x
x0
x1
……...
xn
y
y0
y1
……...
yn
A unique polynomial of following degree can be found that passes through these points (choose one)
(A) n (B) n or less (C) n + 1 (D) n + 1 or more
Problem 4.2 This table lists the population of Canada every 5 years from 2000 to 2015:
Year
Population (in mln)
2000
30.69
2005
32.24
2010
34.01
2015
35.83
Using the Lagrange interpolation polynomial to express the population as
P3 ( x ) = a0l0 (x) + a1l1 (x) + a2l2 (x) + a3l3 (x)
one can approximate the population in any year between 2000 and 2015.
A. What are the values for a0 , a1 , a2 , a3 ?
B. What is the formula for l0 (x) ?
Solution:
A. p3 ( x ) = 30.69l0 ( x ) + 32.24l1 ( x ) + 34.01l2 ( x ) + 35.83l3 ( x )
B. l0 (x) =
( x ! 2005 )( x ! 2010 )( x ! 2015 ) .
( x ! 2005 )( x ! 2010 )( x ! 2015 )
= !
750
( 2000 ! 2005 )( 2000 ! 2010 )( 2000 ! 2015 )
1.0
0.8
0.6
0.4
0.2
0.0
2000
2002
2004
2006
2008
8/14
2010
2012
2014
Problem 4.3 Consider the following function f ( x ) = sin ( 2! x ) + cos ( 2! x ) , for 0 ! x ! 0.5 depicted in the
plot below.
A. Give a Taylor’s series representation of f ( x ) = sin ( 2! x ) + cos ( 2! x ) at a point x = 0 of order 2 (up to the
2
term x ).
B. We sample the function f ( x ) at 3 points and this is represented by the following table
x
f ( x)
0
1
0.25
1
0.5
-1
Derive the Lagrange interpolation polynomial corresponding to the above data.
C. Comment on the accuracy of the solutions obtained in A and B.
3
2
2! x )
2! x )
(
(
5
+ O ( x ) + 1!
+ O ( x 4 ) = 1+ 2! x ! 2! 2 x 2 + O ( x 3 ) as x ! 0 .
Solution: A. f ( x ) = 2! x !
6
B. p2 ( x ) =
2
( x ! 0.25 )( x ! 0.5 ) ! 1+ ( x " 0 )( x ! 0.5 ) ! 1+ ( x " 0 )( x ! 0.25 ) ! "1 = !16x 2 + 4x + 1 .
( )
( 0 ! 0.25 )( 0 ! 0.5 )
( 0.25 ! 0 )( 0.25 ! 0.5 )
( 0.5 ! 0 )( 0.5 ! 0.25 )
Plot of f ( x ) (black), approximation in A (blue), p2 ( x ) (red).
1.5
1.0
0.5
0.0
�0.5
�1.0
0.0
0.1
0.2
0.3
0.4
0.5
C: Taylor’s expansion is more accurate close to x = 0 . The interpolation polynomial is more accurate on the
whole interval [ 0,0.5 ] .
9/14
Problem 4.4 A robot has to follow a path that passes through 7 planning points in order to avoid two
obstacles, as shown in the plot. To find the shortest path that is also smooth, which of the following solution
strategies would you recommend (choose one and explain why) ?
(A) Pass a 6th degree polynomial through the data
(B) Pass linear splines through the data
(C) Pass cubic splines through the data
(D) Use least squares to fit a 3rd degree polynomial
Problem 4.5 Determine a,b and c such that S ( x ) is a cubic spline function:
" x 3,
0 ! x !1
$
S ( x) = # 1
.
3
2
$ 2 ( x " 1) + a ( x ! 1) + b ( x ! 1) + c, 1 ! x ! 3
%
Verify whether S ( x ) is a natural spline on [ 0, 3] .
" S ( x ) = x 3,
0 ! x !1
$ 0
Solution: S ( x ) = #
.
1
3
2
S
x
=
x
!
1
+
a
x
!
1
+
b
x
!
1
+
c,
1
!
x
!
3
(
)
(
)
(
)
(
)
$ 1
2
%
" 3x 2 ,
0 ! x !1
"$ 6x,
$
(2)
S
x
=
S (1) ( x ) = # 3
;
( ) #
2
3( x " 1) + 2a,
$ 2 ( x " 1) + 2a ( x ! 1) + b, 1 ! x ! 3
%$
%
0 ! x !1
1! x ! 3
1. S0 (1) = S1 (1) ! c = 1
1
1
2. S0( ) (1) = S1( ) (1) ! b = 3
" x 3,
0 ! x !1
$
! S ( x) = # 1
.
3
2
$ 2 ( x " 1) + 3( x ! 1) + 3( x ! 1) + 1, 1 ! x ! 3
%
2
2
3. S0( ) (1) = S1( ) (1) ! a = 3
4. Natural spline condition: S0( ) ( 0 ) = 0 , S1( ) ( 3) = 12 ! This is not a natural spline.
2
2
10/14
.
PART V: Numerical Integration and Differentiation
Problem 5.1. You want to estimate the first derivative of f ( x ) , given values of the function at discrete
points
x
0
0.1
0.2
……..
1
Which of these formulas is appropriate (choose one) for estimating f ( ) (1) ?
1
"3 f ( xi ) + 4 f ( xi + h ) ! f ( xi + 2h )
3 f ( xi ) ! 4 f ( xi ! h ) + f ( xi ! 2h )
(B) f (1) ( xi ) !
2h
2h
f ( xi + h ) ! f ( xi ! h )
(C) f (1) ( xi ) !
(D) All of the above.
2h
(A) f (1) ( xi ) !
Problem 5.2. Consider the following system of two nonlinear differential equations
dx ( t )
= !y ( t ) ! x 2 ( t )
dt
dy ( t )
= 2x ( t ) ! y 3 ( t )
dt
with the initial conditions x ( 0 ) = 1 , y ( 0 ) = 1 . We are interesting in a solution of these equations for
0 ! t ! 10 . In order to get accurate results, we choose a step size h = 0.1 . Derive Euler’s numerical
procedure for solving the above equations. Evaluate the values of x ( t ) and y ( t ) at t = 0.2 .
Solution:
xk+1 = xk + hf1 ( xk , yk )
yk+1 = yk + hf2 ( xk , yk )
,
where f1 ( x, y ) = !x 2 ! y , f2 ( x, y ) = 2x ! y 3 , x ( 0 ) = x0 = 1 , y ( 0 ) = y0 = 1 , and h = 0.1 .
Then,
x1 = x0 + hf1 ( x0 , y0 ) ! x1 = 0.8
y1 = y0 + hf2 ( x0 , y0 ) ! y1 = 1.1
and next
x2 = x1 + hf1 ( x1 , y1 ) ! x2 = 0.626
y2 = y1 + hf2 ( x1 , y1 ) ! y2 = 1.1269 .
Problem 5.3. We wish to approximate ! f ( x ) dx given the table of values
2
1
x
1
f ( x)
10
5/4
8
6/4
7
7/4
6
Compute an estimate of ! f ( x ) dx by the trapezoid rule.
2
1
Solution:
!
2
1
f ( x ) dx "
1
57
h n
= 7.125 .
fi#1 + fi ) = {(10 + 8 ) + ( 8 + 7 ) + ( 7 + 6 ) + ( 6 + 5 )} =
(
!
8
8
2 i=1
11/14
2
5
Problem 5.4. Consider the following integral I ( f ) = ! e x dx .
2
0
A. Find a bound for the error in the trapezoidal integration rule Tn ( f ) for evaluation of I ( f ) .
B. Estimate the number of subdivisions to estimate I ( f ) by Tn ( f ) to the accuracy ! = 0.5 ! 10 .
"4
C. Find a bound for the error in Simpson’s integration rule Sn ( f ) for evaluation of I ( f ) .
D. Estimate the number of subdivisions to estimate I ( f ) by Sn ( f ) to the accuracy ! = 0.5 ! 10 .
"4
2
2 1
1 " 2%
Solution: A. En ( f ) ! 2 $ ' max 0!x!2 f ( 2 ) ( x ) = e2 2 .
3 n
12 # n & !##"##$
=e2
B.
2 2 1
e 2 ! 0.5 " 10 #4 ! n ! 314 .
3 n
4
8 2 1
1 " 2%
e
C. En ( f ) !
.
2 $ ' max 0!x!2 f ( 4 ) ( x ) =
45 n 4
180 # n & !##"##$
=e2
D.
8 2 1
e
! 0.5 " 10 #4 ! n ! 14 ! even
45 n 4
12/14
PART VI: Numerical Linear Algebra
Problem 6.1. You have a system of 3 linear equations with 3 unknowns. If you perform Gaussian elimination
and obtain the following upper triangular representation
" 1 !2 4 | 6 %
$
'
$ 0 1 0 | !3 '
$# 0 0 3 | 0 '&
then the system has (choose one)
(A) a unique solution (B) no solution (C) infinitely many solutions (D) more than one solution
Solution: The solution is
x1 ! 2x2 + 4x3 = 6
x2 = !3
3x3 = 0
! x1 = 0, x2 = !3, x3 = 0
Problem 6. 2. Consider the following system of linear equations
2x1 + 6x2 + 10x3 = 0
x1 + 3x2 + 3x3 = 2
.
3x1 + 14x2 + 28x3 = !8
A. Represent the linear system in the augmented matrix form [ A | b ] .
B. Perform the Gauss elimination algorithm (with pivoting if necessary) in order to obtain the upper
triangular representation [ U | g ] .
C. Find the final solution by the back substitution procedure.
D. Find the value of det [ A ] .
Solution:
" 2 6 10 | 0 %
A. $ 1 3 3 | 2 '
$
'
$# 3 14 28 | !8 '&
B.
Step 1:
" 2 6 10 | 0 %
" 2 6 10 | 0 %
$
' ! $
'
$ 1 3 3 | 2 '
$ 0 0 !2 | 2 '
$# 3 14 28 | !8 '&
$# 0 5 13 | !8 '&
Need pivoting: row 2 ! row 3 !
Step 2:
" 2 6 10 | 0 %
$
'
$ 0 5 13 | !8 '
$# 0 0 !2 | 2 '&
2x1 + 6x2 + 10x3 = 0
C. 5x2 + 13x3 = !8
!2x3 = 2
= [U | b] .
! x3 = !1 , x2 = 1 , x1 = 2 .
D. det [ A ] = ( !1) det [ U ] = ( !1) " 2 " 5 " ( !2 ) = 20 .
13/14
Appendix: Useful Formulas
• f ( x + h ) = f ( x ) + hf (1) ( x ) +
h 2 (2)
h 3 ( 3)
f ( x) +
f (! ) , where x < ! < x + h .
2
6
x x2 x3
xn
1
x n+1e!
• e = 1+ + + + ....+ + ...+
1! 2! 3!
n!
( n + 1)!
x
x3 x5 x7
( !1) x 2n+1 + ( !1) x 2n+3 cos !
• sin ( x ) = x ! + ! + ....+
( )
3! 5! 7!
( 2n + 1)!
( 2n + 3)!
n
n+1
x2 x4 x6
( !1) x 2n + ( !1) x 2n+2 cos !
• cos ( x ) = 1! + ! + ....+
( )
2! 4! 6!
( 2n )!
( 2n + 2 )!
n
n+1
1
(b ! a ) h 2 f (2) (! )
12
b
1
• Simpson’s Rule Error: " f ( x ) dx ! Sn ( f ) = !
(b ! a ) h 4 f ( 4 ) (! )
a
180
• Trapezoid Rule Error: " f ( x ) dx ! Tn ( f ) = !
b
a
14/14