MATH380 Numerical Methods for Engineers 2020 - 2021 Week 1 Prof. Dr. Ayhan Aydın, Prof. Dr. Sofiya Ostrovska Atılım University Mathematics Department, Ankara Textbook: J.H. Mathews, K.D. Fink, Numerical Methods using Matlab, 4th ed.(2004) Introduction Estimating the accuracy is an important part of numerical analysis. In real problems, there are three main sources of errors: (i) errors in the input data; (ii) roundoff errors; (iii) truncation errors. Errors in the input data occur by incorrect measurements, imperfect devices, etc. Roundoff errors arise when we perform calculations with numbers, whose representation is restricted to a certain finite number of digits, as usually is the case (calculators). Truncation errors are related to the methods of calculation. Mostly, numerical methods do not provide the exact solution to a problem, even if the calculations are carried out without rounding. 1 For example, the problem of summing a convergent infinite ∞ X series S = an can be replaced by the simpler problem of n=1 finding a partial sum of the first N terms: SN = N X an and then n=1 using the approximation S ≈ Sn . 2 In this case, E = S − SN is a truncation error. 3 Truncation errors occur as a result of simplifying mathematical problem with the help of an approximate formula. Here are the examples of some commonly used approximate formulae: sin x ≈ x cos x ≈ 1 − for small x; x2 2 for small x; Representation of numbers There are different ways to present real numbers. For example, 3.99999999 . . . and 4 are two different representations of the same number 4, that is, 3.99999999 · · · = 4. Also, we may write either 2000 or 2 × 103 to represent the same number. For computer calculations, it is common to use a floating-point representation of the form: a × 10m where |a| < 1 and m ∈ Z (positive or non-negative integer). For example, we have 315 = 0.315 × 103 . here, a = 0.315 is called the mantissa and m = 3 is said to be the exponent, which indicates the position of the decimal point with respect to mantissa. Any computer has a limited number of digits both in a representation of mantissa and exponent. Obviously, a floating-point representation of a number is not unique. The most common forms of number representation are: a normalized floating point representation, where 0.1 ⩽ |a| < 1, that is a = ±0.d1 d2 . . . dk . . . × 10m , where d1 6= 0. For example, 315 = 0.315 × 103 = 0.0315 × 104 = 0.00315 × 104 and only the first one is normalized. a scientific notation representation: a = ±t.d1 d2 . . . dk . . . × 10m , where t 6= 0. For example, 315 = 0.315 × 103 = 3.15 × 102 . Here, the second one is a scientific notation form. C HOPPING OFF AND ROUNDING OFF Let x be a number written in the normalized form as x = ±0.d1 d2 . . . dk dk+1 · · · × 10m . Suppose that the maximum number of decimal digits allowed by the computer’s floating-point representation is k. The approximation x = ±0.d1 d2 . . . dk × 10m is the chopped floating-point representation. Another way to approximate x using k-digit decimals mantissa is the rounded floating-point representation constructed as follows: 0.d1 d2 . . . dk 0.d1 d2 . . . (dk + 1) 0.d1 d2 . . . dk dk+1 · · · −→ 0.d1 d2 . . . (dk−1 + 1)0 and so on if dk+1 ⩽ 4 if dk+1 ⩾ 5 and dk 6= 9 if dk+1 ⩾ 5, dk = 9, dk−1 6= 9 Example. For each number, find 3-digit chopped and rounded floating point representations. x = 0.2483 × 104 . Then chopped representation x = 0.248 × 10 , and rounded representation is x∗∗ = 0.248 × 104 , that is x∗ = x∗∗ ; ∗ 4 y = 0.2487 × 104 . Then chopped representation y = 0.248 × 10 , and rounded representation is y∗∗ = 0.249 × 104 , that is x∗ 6= x∗∗ ; ∗ 4 z = 0.2497 × 104 . Then chopped representation z = 0.249 × 10 , and rounded representation is z∗∗ = 0.250 × 104 , that is, z∗ 6= z∗∗ . ∗ 4 O RDER OF APPROXIMATION Let f (h) and g(h) be functions defined in a neighbourhood of 0. We say that f (h) = O(g(h)) as h → 0 if there exist M > 0 and h0 > 0 such that |f (h)| ⩽ M|g(h)| for all |h| ≤ h0 . f (h) In other words, the ratio g(h) is bounded when |h| < h0 and g(h) 6= 0. E XAMPLES 1 sin h = O(h) as h → 0. 2 ln(1 + h) = O(h) as h → 0. 3 3h2 + 2h3 = O(h2 ) as h → 0. Indeed, 3h2 + 2h3 = h2 (3 + 2h), hence |3h2 + 2h3 | ⩽ 5h2 when |h| ⩽ 1. TAYLOR ’ S FORMULA Let us recall the Taylor’s formula: If a function f has (n + 1) derivatives in some neighbourhood of a point x0 , then f (x0 + h) = f (x0 ) + f 0 (x0 ) · h + f (n) (x0 ) f 00 x0 )2!h2 + · · · + + Rn (x0 ; h), ( n! where Rn (x0 ; h), is a reminder satisfying Rn (x0 ; h) = O(hn+1 ) as h → 0. E XAMPLES 1 x0 = 0, n = 3 McLaurin’s formula eh = 1 + h + 2 h2 h3 h2 h3 + + O(h4 ) = 1 + h + + + O(h4 ), h → 0. 2! 3! 2 6 x0 = 0, n = 3 McLaurin’s formula sin h = h − 3 h3 + O(h4 ), h → 0. 6 x0 = 0, n = 4 McLaurin’s formula sin h = h − h3 h3 + 0 · h4 + O(h5 ) = sin h = h − + O(h5 ), h → 0. 6 6 E RROR ANALYSIS The results of numerical computations are usually not exact solutions to practical problems. Therefore, it is important to estimate the errors occuring in such computations. Let p̂ serve as an approximation to exact value p. Then Ep = |p̂ − p| is E |p̂−p| an absolute error of approximation, and, Rp = pp = p , p 6= 0 is a relative error. The relative error expresses an error as a percentage of the true value. For example, we think that 400cm ± 1cm is a good accuracy of measurements, while 4cm ± 1cm is definitely not. Notice that the absolute errors in both cases are equal: E1 = E2 = 1cm. However, the 1 relative errors are not close as R1 = 400 = 0.0025 = 0.25%, while 1 R2 = 4 = 0.25 = 25%, that is R1 << R2 . E RROR ANALYSIS A number p̂ approximates a value p to d significant digits if d is the greatest non-negative integer satisfying Rp < 101−d = 5 × 10−d . 2 Example. For p = 2.71828183 and b p = 2.71 we have |p − b p| |2.71828183 − 2.71| = = 0.003046 7 < 0.005 = 5 × 10−3 , |p| |2.71828183| but 0.003046 7 ≮ 0.0005 = 5 × 10−4 . So, b p approximates p to d = 3 significant digits. Example. If p = 4713.82 and b p = 4712, then |p − b p| = 0.00038609 < 0.0005 = 5 × 10−4 , |p| but 0.00038609 ≮ 0.00005 = 5 × 10−5 . So, b p approximates p to d = 4 significant digits. E XAMPLE Example. If p = 0.00004567 and b p = 0.00003567, then |p − b p| = 0.21896212 < 0.5 = 5 × 10−1 , |p| but 0.21896212 ≮ 0.05 = 5 × 10−2 , so, b p approximates p to d = 1 significant digits. L OSS OF SIGNIFICANCE Apart from roundoff errors which are inevitable and often difficult to control, there are errors that can be avoided by using more appropriate programming algorithms. Let x = 2.71828182 and y = 2.71826143. They have 9 decimal digits of the precision. If we perform the operation x − y we have x − y = 0.00002039 = 0.2039 × 10−4 which has 4 decimal digits of precision. This phenomenon is called loss of significance. One of the typical situations: subtraction of nearly equal quantities. Let x and y be real numbers, whose exact values are: x = 0.3721478693 and y = 0.3720230572. If we use 5-digit rounded mantissa representation, we have x∗ 0.37215 and y∗ = 0.37202. Here, the relative errors are Rx = 0.21307 × 10−5 x∗ − x = 0.57254 × 10−5 (small), = x 0.3721478693 Ry = y∗ − y 0.30572 × 10−5 = 0.82178 × 10−5 (small). = y 0.3720230572 However, if we take x − y ≈ x∗ − y∗ , we obtain Rx−y = (x∗ − y∗ ) − (x − y 0.00013 − 0.000124812 = = 0.4157×10−1 . x−y 0.000124812 The relative error is much greater in magnitude, may be not sufficient for practical calculations. E XAMPLE . √ Consider the calculation of the expression 1 + x2 − 1 when x is close to 0. Here, we have the subtraction of 1 from the quantity very close to 1 for small x (x ≈ 0.) To overcome the problem, we may write the given expression in a mathematically equivalent form, which does not have this type of subtraction. √ √ p 2 − 1)( 1 + x2 + 1) 1 + x ( x2 √ 1 + x2 − 1 = =√ . 1 + x2 + 1) 1 + x2 + 1 E XAMPLE . Consider the calculation of the expression x − sin x when x is close to 0. As we know, sin x ≈ x when x is small, so we have the same situation with subtraction of nearly equal quantities. To cope with the problem, we may use Taylor’s formula: sin x = x − Hence, x3 x5 + + O(x6 ), 3! 5! x → 0. x5 x3 − + O(x6 ), x → 0. 3! 5! The truncation error may be estimated from the range of values for x and desired accuracy. x − sin x = E XERCISE . Suggest ways to avoid the loss of significance in these calculations. 1 2 3 ln x − ln y when x is close to y. Re-write: ln x − ln y = ln xy . √ √ x + 2 − 2 when x is close to 0. We can transform the expression as follows: √ √ √ √ √ √ x + 2 − 2)( x + 2 + 2) x √ √ . x+2− 2= =√ √ x+2+ 2 x+2+ 2 ex − 1 when x is close to 0. We use Taylor’s formula ex = 1 + x + x2 x3 x4 + + + O(x5 ), 2! 3! 4! x→0 and obtain: x x2 x3 + O(x5 ), ex − 1 = x 1 + + + 2 6 24 x → 0. S OLVING NON - LINEAR EQUATIONS Let f (x) be a continuous function. Consider equation f (x) = 0. A number x0 for which f (x0 ) = 0 is a solution to a given equation, also called a root or zero of function f . In general, an equation may have 1, several, infinitely many solutions or no solutions at all. To solve an equation means to find all of its solutions. We know that a linear equation of the form ax + b = 0, a 6= 0 has a unique solution and can be easily solved as x = − ba . If a functions f is not linear, then we face a non-linear equation. Some of the can be solved easily (quadratic equations, standard logarithmc, exponential or trigonometric equations). However, in many applied problems, we have to solve equations which either do not have explicit formulae providing solutions or these formulae are too complicated to be used practically. In this case, practitioners usually find solutions with the help of modern software or numerical algorithms.These ways, as a rule, lead to approximate solutions, which may be sufficient for applications. S OLVING NON - LINEAR EQUATIONS A common procedure of solving equations numerically can be outlined as follows. 1 Localization of roots. Here, we try to find intervals so that every one contains exactly one root. This is usually done with the help of Calculus methods. 2 Choosing an initial approximation to the root in this interval. 3 Constructing a sequence converging to this root, that is, choosing a suitable algorithm. 4 Estimating error after n steps. E XAMPLES . In order to locate and separate roots, we have to examine the behaviour of a function f . The following intermediate value theorem from the calculus course is often helpful. Theorem If f (x) ∈ C[a, b] and f (a) · f (b) < 0, then there is at least on point c ∈ [a, b] such that f (c) = 0. If, in addition, f (x) is strictly increasing or decreasing on [a, b], then such point c is unique. Graphical representations of functions may be convenient to locate their roots. Example Consider an equation ex + x = 0. Show that the equation has exactly one real solution. Find the an interval of length at most 1 containing the solution. Solution: We can write this equation as ex = −x. A solution of the equation is an intersection point of the graphs y = ex and y = −x. y −1 x0 0 x From the graphs, we see that there is exactly one intersection point, and, hence, exactly one solution to the given equation. Take f (x) = ex + x. Then, f (0) = e0 = 1 > 0, while f (−1) = e−1 − 1 < 0. Since f (x) is continuous, we conclude that it has a root on [−1, 0]. How to find solutions of non-linear equations numerically? Next, we are going to discuss different methods used for solving such equations.