NUMERICAL METHODS Systems Of Linear Algebraic Equations Systems of equations arise in all branches of engineering and science. These equations may be algebraic, transcendental (i.e., involving trigonometric, logarithmic, exponential, etc., functions), ordinary differential equations, or partial differential equations. The equations may be linear or nonlinear. NUMERICAL METHODS Systems of non-linear Equations Nonlinear equations arise in many physical problems. Finding their roots, or zeros is a common problem. The problem can be stated as follows: Given the continuous nonlinear function f(x), find the value of x = α such that f(α) = 0, where α is the root, or zero, of the nonlinear equation. Nonlinear equations are solved by iterative methods. A trial solution is assumed, the trial solution is substituted into the nonlinear equation to determine the error or mismatch, and the mismatch is used in some systematic manner to generate an improved estimate of the solution. NUMERICAL METHODS The workhorse methods of choice for solving nonlinear equations are Newton’s method, the secant method, the Bisection method, and fixed point iteration. Nonlinear equations occur throughout engineering and science. Nonlinear equations also arise in other areas of numerical analysis. Implicit methods for solving nonlinear differential equations yield nonlinear difference equations Note Bisection and Newton- Raphson are based on the intermediate value theorem (IVT), If 𝑓(𝑥) is continuous in the interval (a,b ) where 𝑓 𝑎 𝑓 𝑏 < 0, (𝑓 𝑎 𝑎𝑛𝑑 𝑓 𝑏 ) have different sign, there exist at least a solution C 𝜖( a, b) where 𝑓 𝑐 = 0 NUMERICAL METHODS • EXP 1: Find the possible sub-interval that contains the solution of 𝑓 𝑥 = 𝑥3 3 − 𝑥 2 + 1 in the interval (-2, 3) with step size (h) = 1 using the intermediate value theorem. Use 4 decimal points in your calculation Bisection method This method is a root-finding method that applies to any continuous functions with two known values of opposite signs. The interval defined by these two values is bisected and a sub-interval in which the function changes sign is selected. This sub-interval must contain the root. These two steps are repeatedly executed until the root is in the form of the required precision level. This method is also called as interval halving method, the binary method, or the dichotomy method. The Method: Explained Let f be a continuous function defined on an interval [a,b] where f(a) and f(b) have opposite signs. The bisection method is applicable for solving the equation 𝑓(𝑥)=0 for a real variable x. At each step, the interval is divided into two parts/halves by computing the midpoint, 𝑐𝑖 = 𝑎𝑖 +𝑏𝑖 2 and the value of f(c) at that point. Unless the root is c, there are two possibilities: 𝑓(𝑎) and: 𝑓(𝑐) have opposite signs and bracket a root, OR: 𝑓(𝑐) and: 𝑓(𝑏) have opposite signs and bracket a root. One of the sub-intervals is chosen as the new interval to be used in the next step. This process is carried out again and again until the interval is sufficiently small. Bisection METHODS If 𝑓(𝑎) and 𝑓(𝑐) have opposite signs, then the value of b is replaced by c. If 𝑓(𝑏) and 𝑓(𝑐) have opposite signs, then the value of a is replaced by c. In the case, that: 𝑓(𝑐) = 0, c will be taken as the solution and the process stops Steps 1. Using the two initial guesses, find the root in some interval 𝑐𝑖 = 𝑎𝑖 +𝑏𝑖 2 The size of the interval will decrease until the value of 𝑓 𝑐𝑖 = 0 0r 𝑏𝑖 − 𝑐𝑖 ≤ 𝜀 indicating to stop the iteration where 𝜀 is the error of tolerance This method always coverage NUMERICAL METHODS Figure NUMERICAL METHODS EXP II Find the solution of 𝑒 −𝑥 − 𝑥 = 0 in the interval (0,1) using the bisection method. Use 3 decimal places in all calculations Solution The root of 𝑓 𝑥 is approximately 0.567 NUMERICAL METHODS EXP III : Find the root of x4-x-10=0 accurate of ε=0.015 in the interval (1.5, 2) by using the bisection method, use 3 decimal places in all calculation 𝒊 𝒂𝒊 𝒃𝒊 𝒄𝒊 = 𝒂𝒊 + 𝒃𝒊 𝟐 𝒇(𝒂𝒊 ) 𝒇(𝒃𝒊 ) 𝒇(𝒄𝒊 ) 𝒂𝒊 − 𝒄𝒊 < 𝜺 𝒃𝒊 − 𝒄𝒊 < 𝜺 0 1.5 2 1.75 -6.438 4 -2.370 2-1.75 = 0.25 1 1.75 2 1.875 -2.371 4 0.485 1.75-1.875 = -0.125 2 3 NUMERICAL METHODS EXERCISE a) Find the approximate root of equation x3-x-4 = 0 b) Determine the root of the equation, f(x)=x3–x–2 for x ∈ [1,2]. c) You have a spherical storage tank containing oil. The tank has a diameter of 6 m. You are asked to calculate the height, h, to which a dipstick 8m long would be wet with oil when immersed in the tank when it contains 4 m3 of oil. Dipstick Spherical Storage Tank r h The equation that gives the height, h, of liquid in the spherical tank for the given volume and radius is given by f h h3 9h 2 3.8197 0 NUMERICAL METHODS Use the bisection method of finding roots of equations to find the height, h, to which the dipstick is wet with oil. Find the absolute relative approximate error at the end of each iteration and the number of significant digits at least correct at the end of each iteration. Newton-Raphson Method gg AB tan( AC f(x) f(xi) f ( xi ) f ' ( xi ) xi xi 1 B C xi+1 A xi f ( xi ) xi 1 xi f ( xi ) X Newton-Raphson Method Only one initial guess 𝑥𝑖 is needed to start the iteration Formula 𝑥𝑖+1 = 𝑥𝑖 − 𝑓(𝑥) 𝑓′(𝑥) Stop the iteration when the value of 𝑓(𝑥𝑖) = 0 or 𝑥𝑖 − 𝑥𝑖−1 ≤ 𝜀 where 𝜀 is the error of tolerance If the method converges, it is much faster than the bisection method Newton-Raphson Method EXP IV: Use Newton Raphson method to estimate the root of 𝑓 𝑥 = 𝑥 2 + 2𝑥 − 5 = 0 accurate to 𝜀 < 0.001 and given an initial point x0 = 1. Use 4 decimal places in all calculation 𝒙𝒊 i 𝒇(𝒙𝒊) 𝒇′(𝒙𝒊) 0 1 -2 4 1 1.5 0.25 5 2 3 𝒙𝒊 − 𝒙𝒊 − 𝟏 𝟏. 𝟓 − 𝟏 = 𝟎. 𝟓 Newton-Raphson Method • EXP V: Find the approximate root of x3+3x-7 up to 4 iterations Secant Method – Derivation The Geometric Similar Triangles f(x) f(xi) AB DC AE DE B can be written as f ( xi ) f ( xi 1 ) xi xi 1 xi 1 xi 1 C f(xi-1) xi+1 E D xi-1 A xi X xi 1 xi f ( xi )( xi xi 1 ) f ( xi ) f ( xi 1 ) Secant Method – Derivation Secant Method The open method which may converge or may not converge If it converges, it is faster than the bisection and Newton-Raphson method. Use two initial guesses 𝑥𝑖 𝑎𝑛𝑑 𝑥𝑖−1 The formula for the next 𝑥𝑖+1 = 𝑥𝑖 − 𝑓(𝑥𝑖 ) 𝑥𝑖 −𝑥𝑖−1 𝑓 𝑥𝑖 −𝑓(𝑥𝑖−1 ) Stop the iteration when 𝑥𝑖+1 = 𝑥𝑖 (the point is repeated) 𝑓 𝑥𝑖 ≈ 0 A secant line is the straight points line joining two points on a function Secant Method – Derivation Exp 8: Solve x2- 3 = 0 using secant method with initial value x0 = 1 and x1 = 2 use 4 decimal place in all calculations 𝑖 0 𝑥𝑖 1 𝑓(𝑥𝑖 ) 𝒙𝟐 = 𝟐 − 𝟏 𝟐−𝟏 𝟏− −𝟐 = 𝟏. 𝟔𝟔𝟔𝟕 𝒙𝟑 = 𝟏. 𝟔𝟔𝟔𝟕 − −𝟎. 𝟐𝟐𝟐𝟏 𝟏.𝟔𝟔𝟔𝟕−𝟐 −𝟎.𝟐𝟐𝟐𝟏−𝟏 -2 The root of 𝒇(𝒙) is approximate 1.7321 1 2 1 2 1.6667 -0.2221 3 1.7273 -0.0164 4 1.7321 0.0002 5 1.7321 0.0002 = 𝟏. 𝟕𝟐𝟕𝟑 The fixed-point interaction method Fixed point of a function = element of the function’s domain that is mapped to itself by the function The number p is a fixed point for a given function 𝑔 𝑥 𝑖𝑓 𝑔 𝑝 = 𝑝 Reformulate a nonlinear equation 𝑓 𝑥 = 𝑜 𝑡𝑜 𝑥 = 𝑔(𝑥) Start with one initial guess Stop the interaction when 𝑥𝑖+1 − 𝑥1 = 0 (the point is repeated) The fixed-point interaction method Convergence condition If 𝑔 𝑥 𝑎𝑛𝑑 𝑔′(𝑥) are continuous for all in the interval (a, b), then the iteration process defined by 𝑥𝑖+1 = 𝑔(𝑥) will converge to the solution s if 𝑔′(𝑥0 ) < 1 with 𝑥0 is an initial value where s and 𝑥0 belong to the interval (a, b) The fixed-point interaction method Example IX: Solve 2x2-6x+3 = 0 using fixed point iteration method with initial value x0 =1 and 𝑥=𝑔 𝑥 = 2𝑥 2 +3 6 Use 4 decimal places in all calculations Solution 𝑔 𝑥 = 2𝑥 2 +3 6 , 𝑔′ 𝑥 = When X0 = 1 𝑔 1 = Hence 𝑥𝑖+1 = 2𝑥 2 +3 6 2 3 2𝑥 3 1 = 0.6667 < 1 , it converges The fixed-point interaction method Example IX: 𝑖 𝑥𝑖 𝑥𝑖+1 − 𝑥𝑖 − 0 1.0000 1 0.8333 0.1667 2 0.7315 0.1018 3 0.6784 0.0531 4 0.6534 0.0250 5 0.6423 0.0111 6 0.6375 0.0048 7 0.6355 0.0020 8 0.6346 0.0009 9 0.6342 0.0004 10 0.6341 0.0001 11 0.6340 0.0001 12 0.6340 0.0000 The root of 𝑓 𝑥 = 2𝑥 2 − 6𝑥 + 3 is approximately 0.6340 Systems of nonlinear equations So far, the roots of a single equation have been handled , now it is time to focus on solving the roots of a set of simultaneous equations . Newton-Raphson method Consider a system of two equation 𝑓 𝑥, 𝑦 = 0 𝑎𝑛𝑑 𝑔 𝑥, 𝑦 = 0 Let (𝑥0 , 𝑦0 ) are initial approximation to the root of equation If (𝑥0 + ℎ, 𝑦0 + 𝑘) is the root of the system then 𝑓 𝑥0 + ℎ, 𝑦0 + 𝑘 = 0 g 𝑥0 + ℎ, 𝑦0 + 𝑘 = 0 Systems of nonlinear equations Assuming 𝑓 𝑎𝑛𝑑 𝑔 are sufficiently differential then by Taylor's series we obtain 𝑓0 + ℎ 𝑑𝑓 𝑑𝑥0 +𝑘 𝑑𝑓 𝑑𝑦0 + ⋯……… = 0 𝑔0 + ℎ 𝑑𝑔 𝑑𝑥0 +𝑘 𝑑𝑔 𝑑𝑦0 + ⋯……… = 0 Where 𝑑𝑓 𝑑𝑥0 = 𝑑𝑓 , 𝑑𝑥 𝑥=𝑥0 𝑓0 = 𝑓(𝑥0 , 𝑦0 ) etc Neglecting the second and higher order derivative terms, we obtain the following system of linear equations 𝑑𝑓 𝑑𝑓 ℎ +𝑘 = −𝑓0 𝑑𝑥0 𝑑𝑦0 𝑑𝑔 𝑑𝑔 ℎ +𝑘 = −𝑔0 𝑑𝑥0 𝑑𝑦0 Systems of nonlinear equations The above equations solution of the determinant of the Jacobian matrix 𝑑𝑓 𝑑𝑥0 𝑑𝑔 𝑑𝑥0 D= 𝑑𝑓 𝑑𝑦0 𝑑𝑔 𝑑𝑦0 ≠0 By Cramer's rule , the solution of equation is given by h= 1 𝐷 −𝑓0 −𝑔0 𝑑𝑓 𝑑𝑦0 𝑑𝑔 𝑑𝑦0 and k 𝑑𝑓 1 𝑑𝑥 = 𝑑𝑔0 𝐷 𝑑𝑥0 −𝑓0 −𝑔0 The new approximations are therefore 𝑥1 = 𝑥0 + ℎ 𝑦1 = 𝑦0 + 𝑘 The process is to be repeated until we obtain the roots to desired accuracy . Systems of nonlinear equations Example: Solve the following system using Newton-Raphson method 3𝑦𝑥2 − 10𝑥 + 7 = 0 𝑦2 − 5𝑦 + 4 = 0 𝑓 𝑥, 𝑦 = 3𝑦𝑥2 − 10𝑥 + 7 = 0 𝑔 𝑥, 𝑦 = 𝑦2 − 5𝑦 + 6 = 0 𝑑𝑓 𝑑𝑥 = 6xy − 10 , 𝑑𝑓 𝑑𝑦 = 3x2, 𝑑𝑔 𝑑𝑥 =0, Taking 𝑥0 = 𝑦0 = 0.5 as initial approximate 𝑓0 = 3 0.5 0.5 𝑔0 = 0.5 2 2 − 10 0.5 + 7 = 2.375 − 5 0.5 + 4 = 1.75 𝑑𝑔 𝑑𝑦 = 2y − 5 , Systems of nonlinear equations 𝑑𝑓 𝑑𝑥 0 = 6 0.5 0.5 − 10 = −8.5 𝑑𝑔 𝑑𝑥 0 =0 D= 𝑑𝑓 𝑑𝑥0 𝑑𝑔 𝑑𝑥0 𝑑𝑔 𝑑𝑦 0 , 𝑑𝑓 𝑑𝑦0 = 3 0.5 2 = 0.75 = 2 0.5 − 5 = 4 𝑑𝑓 𝑑𝑦0 𝑑𝑔 𝑑𝑦0 −8.5 0.75 = 34 0 −4 = h = 𝑑𝑓 𝑑𝑦0 1 𝑑𝑔 𝐷 −𝑔0 𝑑𝑦 0 K = 𝑑𝑓 1 𝑑𝑥0 𝐷 𝑑𝑔 𝑑𝑥0 −𝑓0 1 = 34 −𝑓0 −𝑔0 −2.375 0.75 = 0.3180 −1.75 − 4 1 = 34 −8.5 − 2.375 = 0.4375 −0 − 1.75 𝑥1 = 𝑥0 + ℎ = 0.5 + 0.3180 = 0.818 𝑦1 = 𝑦0 + 𝑘 = 0.5 + 0.4375 = 0.9375 Systems of nonlinear equations For 2nd approximation 𝑓1 = 3 0.9375 0.818 2 − 10 0.818 + 7 = 0.7019 𝑔1 = 0.9375 2 − 5 0.9375 + 4 = 0.1914 𝑑𝑓 𝑑𝑥1 = 6 0.9375 0.818 − 10 = −5.3988 𝑑𝑔 𝑑𝑥1 =0 D= 𝑑𝑓 𝑑𝑥1 𝑑𝑔 𝑑𝑥1 𝑑𝑔 𝑑𝑦 2 𝑑𝑓 , 𝑑𝑦1 = 3 0.818 = 2 0.9375 − 5 = -3.125 𝑑𝑓 𝑑𝑦1 𝑑𝑔 𝑑𝑦1 −5.3988 2.0074 = 16.8712 0 − 3.125 = h = 𝑑𝑓 𝑑𝑦1 1 𝑑𝑔 𝐷 −𝑔1 𝑑𝑦 1 K = 𝑑𝑓 1 𝑑𝑥1 𝐷 𝑑𝑔 𝑑𝑥1 −𝑓1 1 = 16.8712 −𝑓1 −𝑔1 1 −0.7019 −0.1914 = 16.8712 2.0074 = 0.1528 − 3.125 −5.3988 − 0.7109 = 0.0612 −0 − 0.1914 𝑥2 = 𝑥1 + ℎ = 0.818 + 0.1528 = 0.9708 𝑦2 = 𝑦1 + 𝑘 = 0.9375 + 0.0612 = 0.9987 2 = 2.0074 Systems of nonlinear equations For 3nd approximation 𝑓2 = 3 0.9987 0.9708 𝑔2 = 0.9987 2 2 − 10 0.9708 + 7 = 0.1156 − 5 0.9987 + 4 = 0.0039 𝑑𝑓 𝑑𝑥 2 = 6 0.9987 0.9708 − 10 = −4.1827 𝑑𝑔 𝑑𝑥 2 =0 D= 𝑑𝑓 𝑑𝑥2 𝑑𝑔 𝑑𝑥2 𝑑𝑔 𝑑𝑦 2 𝑑𝑓 , 𝑑𝑦2 = 3 0.9708 2 = 2.8273 = 2 0.9987 − 5 = -3.0026 𝑑𝑓 𝑑𝑦2 𝑑𝑔 𝑑𝑦2 −4.1827 2.8273 = 12.55897 0 − 3.0026 = h = 𝑑𝑓 𝑑𝑦2 1 𝑑𝑔 𝐷 −𝑔2 𝑑𝑦 2 K = 𝑑𝑓 1 𝑑𝑥2 𝐷 𝑑𝑔 𝑑𝑥2 −𝑓2 1 = 12.55897 −𝑓2 −𝑔2 1 −0.1156 −0.0039 = 12.55897 2.8273 = 0.0285 − 3.0026 −4.1827 − 0.1156 = 0.00129 −0 − 0.0039 𝑥3 = 𝑥2 + ℎ = 0.9708 + 0.0285 = 0.9993 𝑦3 = 𝑦2 + 𝑘 = 0.9987 + 0.00129 = 0.9999 Therefore x = 1 and y = 1 Systems of nonlinear equations EXERCISE 1. Solve the system of nonlinear by Newton Raphson method x2 + y2 -1 y = x2 Initial approximation x0 = 0.5 and y0 = 0.5 2. Solve the following equation using Newton Raphson’s method sin 𝑥 − 𝑦 = −0.9793 cos 𝑦 − 𝑥 = −0.6703 with xo = 0.5 , y0 = 1.5 as initial approximation 3. Solve the following system of nonlinear equations -2x3 +3y2+42 = 0 5x2 +3y3-69 = 0 use Newton’s method , start x = 1 ,y = 1 and carryout the first five iterations Systems of nonlinear equations EXERCISE 1. Solve the system of nonlinear by Newton Raphson method x2 + y2 -1 y = x2 Initial approximation x0 = 0.5 and y0 = 0.5 2. Solve the following equation using Newton Raphson’s method sin 𝑥 − 𝑦 = −0.9793 cos 𝑦 − 𝑥 = −0.6703 with xo = 0.5 , y0 = 1.5 as initial approximation 3. Solve the following system of nonlinear equations -2x3 +3y2+42 = 0 5x2 +3y3-69 = 0 use Newton’s method , start x = 1 ,y = 1 and carryout the first five iterations Interpolation polynomials Interpolation Process of constructing a polynomial /mathematical equation based on set of data points (𝑥0, 𝑦0), (𝑥𝑛 − Number of point n Order /degree of polynomial 2 P1(x) first order 3 P2(x) Second order 4 P3(x) third order n+1 Pn(x) n order Interpolation polynomials Interpolation polynomial methods Lagrange interpolation Newton divided interpolation Neville’s method Lagrange Interpolation Definition Given data points (x0, yo), (x1,y1)……………………….(xn-1, yn-1) (xn,yn), the interpolation polynomial represented by Lagrange interpolation is Pn(x) = L0(x)yo+ L1(x)y1+ L2(x)y2+ …………………………………………..Ln(x)yn Where 𝑥 − 𝑥0 𝑥 − 𝑥1 … … (𝑥 − 𝑥𝑖−1 )(𝑥 − 𝑥𝑖+1 )(𝑥 − 𝑥𝑛) 𝐿𝑖 = (𝑥𝑖 − 𝑥0) 𝑥𝑖 − 𝑥1 … . . 𝑥𝑖 − 𝑥𝑖−1 … 𝑥𝑖 − 𝑥𝑖+1 . . (𝑥𝑖 − 𝑥𝑛) Interpolation polynomials Example One: Form an interpolation polynomial using Lagrange methods for the following data sets . Then find y for x= 2.5 x 1 2 3 y 2 5 7 Example TWO :Apply Lagrange interpolation polynomial to evaluate P3( 3.5) based on the following data set x 1 2 3 5 y 2 5 7 3 Interpolation polynomials Example Three: Given the following data x y 2 7.2 4.25 7.1 5.25 6.0 7.81 5.0 9.20 3.5 10.60 5.0 What is the value of y at x = 4 Using the first order Lagrange interpolation polynomial. Interpolation polynomials Newton divided difference interpolation Process of constructing a polynomial /mathematical equation based on set of data points (𝑥0, 𝑦0), (𝑥𝑛 − Interpolation polynomials x y/f(x) X0 f0 X1 f1 First d.d 𝑓 𝑥0 , 𝑥1 𝑓1 − 𝑓𝑜 𝑥1 − 𝑥0 𝑓 𝑥1, 𝑥2 X2 x3 f2 f3 𝑓2−𝑓1 𝑥2−𝑥1 𝑓 𝑥2, 𝑥3 𝑓3 − 𝑓2 𝑥3 − 𝑥2 Second d.d 𝑓 𝑥0, 𝑥1, 𝑥2 𝑓 𝑥1, 𝑥2 − 𝑓(𝑥0, 𝑥1) 𝑥2 − 𝑥0 𝑓 𝑥1, 𝑥2, 𝑥3 𝑓 𝑥2, 𝑥3 − 𝑓(𝑥1, 𝑥2) 𝑥3 − 𝑥1 Third d.d 𝑓 𝑥0, 𝑥1, 𝑥2, 𝑥3 𝑓 𝑥1, 𝑥2, 𝑥3 − 𝑓(𝑥0, 𝑥1, 𝑥2) 𝑥3 − 𝑥0 Interpolation polynomials Example: Form an interpolation polynomial using Newton divided difference methods for the following data sets . Then find y for 2.5 x 1 2 3 y 2 5 7 P2 x = f x0 + x − x0 f x0, x1 + x − x0 x − x1 f x0, x1, x2 = 2 + 𝑥 − 1 𝑓 𝑥0, 𝑥1 + 𝑥 − 1 𝑥 − 2 𝑓(𝑥0, 𝑥1, 𝑥2) x Y=f(x) 1 2 First d.d Second d.d 5−2 =3 2−1 2 3 5 7 7−5 3−2 =2 2−3 = −0.5 3−1 = 2 + 𝑥 − 1 3 + 𝑥 − 1 𝑥 − 2 (−0.5) = -0.5x2 +4.5x-2 P2(2.5) = -0.5(2.5)2+4.5(2.5) = 6.125 Interpolation polynomials Neville’s Method Newton's method of interpolation involves two steps: computation of the coefficients, followed by evaluation the polynomial. This works well if the interpolation is carried out repeatedly at different values of x using the same polynomial. If only one point is to be interpolated, a method that computes the interpolant in a single step, such as Neville’s algorithm, is a better choice. Interpolation polynomials Let f be defined at (n+1)distinct points x0, xi……xn Then for each 0≤ 𝑖 < 𝑗 ≤ 𝑛 Then 𝑃𝑖,𝑖+1…..𝑗(𝑥) = 𝑥−𝑥𝑖 𝑃𝑖+1,𝑖+2….𝑗 𝑥 −(𝑥−𝑥 )𝑃 𝑗 𝑖,𝑖+1,…𝑗−1(𝑥) 𝑥𝑗 −𝑥𝑖 is the 𝑛𝑡ℎ Lagrange polynomial NB : The above theorem implies that the interpolating polynomial can be generated recursively. for Example Interpolation polynomials Interpolation polynomials Neville’s Method Example Consider the following table of x and corresponding y values. x 8.1 8.3 8.6 y 16.9446 17.56492 18.50515 8.7 18.82091 Suppose we are interested in interpolating a polynomial that passes through these points to approximate the resulting y value from an x value of 8.4. Interpolation polynomials x 8.1 8.3 8.6 8.7 y 16.9446 17.56492 18.50515 18.82091