CHAPTER 2 Solution of Non-linear Equations 2.1 INTRODUCTION We have seen that expression of the form f(x) = a0xn + a1xn – 1 + .... + an –1x + an where a’s are constant (a0 ≠ 0) and n is a positive integer, is called a polynomial in x of degree n, and the equation f (x) = 0 is called an algebraic equation of degree n. If f (x) contains some other functions like exponential, trigonometric, logarithmic etc., then f (x) = 0 is called a transcendental equation. For example, x3 – 3x + 6 = 0, x5 – 7x4 + 3x2 + 36x – 7 = 0 are algebraic equations of third and fifth degree, whereas x2 – 3 cos x + 1 = 0, xex – 2 = 0, x log10 x = 1.2 etc., are transcendental equations. In both the cases, if the coefficients are pure numbers, they are called numerical equations. In this chapter, we shall describe some numerical methods for the solution of f(x) = 0 where f(x) is algebraic or transcendental or both. Method for finding the root of an equation can be classified into following two parts: (a) Direct methods (b) Iterative methods Direct Methods:- In some cases, roots can be found by using direct analytical methods. For example, for a quadratic equation ax2 + bx + c = 0, the roots of the equation, obtained by √ These are called closed form solution. Similar formulae are also available for cubic and biquadratic polynomial equations but we rarely remember them. For higher order polynomial equations and non-polynomial equations, it is difficult and in many cases impossible, to get Iterative Methods:-These methods, also known as trial and error methods, are based on the idea of successive approximations, i.e., starting with one or more initial approximations to the value of the root, we obtain the sequence of approximations by repeating a fixed sequence of steps over and over again till we get the solution with reasonable accuracy. These methods generally give only one root at a time. For the human problem solver, these methods are very cumbersome and time consuming, but on other hand, more natural for use on computers, due to the following reasons: (1) These methods can be concisely expressed as computational algorithms. (2) It is possible to formulate algorithms which can handle class of similar problems. For example, algorithms to solve polynomial equations of degree n may be written. 1 (3) Rounding errors are negligible as compared to methods based on closed form solutions. Since, we cannot perform infinite number of iterations; we need a criterion to stop the iterations. We use one or both of the following criterion: (i) The equation ( ) | ( is satisfied to a given accuracy or ( ) is bounded by an error tolerance ε. )| (ii) The magnitude of the difference between two successive iterates is smaller than a given accuracy or an error | bound ε. | For example, if we require two decimal places accuracy, then we iterate until | | require three decimal places accuracy, then we iterate until | | . If we . Common Iterative numerical methods can be classified as: Bracketing methods: These methods start with guesses that bracket (contain) the root and then systematically reduce the width of the bracket. i) Bisection ii) False position Open methods: These methods also involve systematic trial-and-error iterations but do not require that the initial guesses contain the root. i) Secant method ii) Fixed point iteration iii) Newton-Raphson method Theorem: - (Intermediate value theorem) Let f(x) be continuous in [a, b] and let p be any number between f(a) and f(b), then their exist a number in (a, b) such that f (c ) p. Corollary: - If f(x) is continuous in [a, b] and f (a ) f (b) 0 , then f (c) 0 for at least one number c such that a c b. i) Bisection Method: The bisection method is a root-finding algorithm which repeatedly bisects an interval then selects a subinterval in which a root must lie for further processing. Suppose ( ) is continuous on an interval [a, b] such that Then the first estimate of the root, is given by ( ) ( ) , , We make the following evaluations to determine the subinterval in which the root lies: If ( ) then is the root of . Otherwise i) if ( ) ( ) then the root lies in the subinterval , - ii) If ( ) ( ) then the root lies in the subinterval , - Continue the process until either we get the exact root or we may have an approximate root with the required degree of accuracy. 2 The Bisection method generates a sequence * i.e ( ) + of mid points of the reduced interval. where Condition of convergence Theorem: Assume that , exists a number is, - such that ( ) is continuous on , . The sequence * - and ( ) ( ) + converges to the zero , then there . That . Example 1: Approximate the root of ( ) ( ) & also ( ) in [2, 3]. is cont on [2, 3]. Thus has at least one root b/n 2 & 3. ( ) 1 2 3 2.5 5.6250 > 0 2 2 2.5 2.25 1.8906 > 0 3 2 2.25 2.125 0.3457 > 0 4 2 2.125 2.0625 -0.3513 < 0 5 2.0625 2.125 2.09375 -0.0089 < 0 The desired approximation is if | | | Theorem: Termination criteria for a given tolerance (acceptance) or error | : The number of iteration ( ) process will be terminated when the length of the interval ( small. i.e 3 ) becomes very Remark: - In order that the absolute error mn ba 2n ba ln required to achieve this accuracy is given by n ln 2 (a given tolerance), the number of iterations = 0 ( ) 1 Example 2: Determine approximately how much iteration is necessary to solve ( ) , with an accuracy of Solution: The maximum number of iteration ( It requires for and is given by ( ) ) iterations to obtain an approximation to an accurate of 3 Example3:- Find the number of iterations needed to approximate the positive root of x 2 x 5 0 with an absolute error < 0.001 and also find that approximate root. ( ) Solution: Tolerance, ( ) . ., / The number of iterations needed to achieve the tolerance 0.001 is an integer greater than or equals 9.965784, i.e. at least for n = 10. ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) See that │ Example4: - Use Bisection Method to approximate the root of x3 –x2 -1 = 0 correct to three decimal places. (Proceed until the first three decimal places of two consecutive iterates agree.) Solution:- The following Fig. Matlab graph of x3 –x2 -1 on [-3, 3] 4 20 10 0 -10 -20 -30 -40 -3 ( ) -2 -1 0 1 2 3 ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Therefore, the root correct to three decimal places is 1.465 Example 5: Using the bisection method, find the real root of the equation ( ) Sol. The given equation at , ( ) √ . Therefore, a root lies between 0 and 1 and at 5 √ , ( ) √ then ( First approximation: root lie b/n ) ( ) ( √ ) . This shows the . 2nd appx. then ( 3rd appx. ) then ( lies b/n ( ) ) ( √ ( ) ) . Thus the root lies b/n ( √ ) . Thus the root . 4th appx. then ( root lies b/n ) ( ) ( √ ) . Thus the . 5th appx. the root lies b/n 7th approximation: ) ( ) √ ( ) . Thus then ( ) ( ) √ ( ) . Thus . 6th appx. the root lies b/n then ( . . From the last two observations, that is, x6 = 0.3907 and x7 = 0.3985, the approximate value of the root up to two places of decimal is given by 0.39. Hence the root is 0.39 approximately. Example6: - Use the Bisection Method to approximate a) √ b) √ accurate to within i) 10-2 ii) 10-4 Solution: - i) 3 is one root of the equation f ( x) 0, where f ( x) x 2 3 and it is located between 1 and 2 Since 01 after 6 iterations the root is: ii) 6 Since is: 1.73204041 after 16 iterations the root The fastness of convergence towards a root in any method is represented by its rate of convergence. Remarks: - The Bisection Method always converge Since the error decrease at each step by a factor of ½ convergence in this method has order one (or linear). Very reliable and has good error bounds It converges slowly. 2. Method of False position /Regula Falsi Method /linear interpolation method: One of the shortcoming of bisection method is that, in dividing the interval , - no account is taken of the magnitude of ( ) ( ) For example if ( ) is much closer to zero than ( ) then it is likely that the root is closer to than . An alternative method, false position that exploits this graphical insight is to join ( ) (chord). ( ) by a straight line The intersection of this line with x-axis represents an improved estimate of the root. Procedure: In this method we choose two points of ( ) must lie in between these points, given that ( ) ( ) are of opposite signs. Hence, a root is continuous on , Now the equation of the chord joining the two points ( ( )) & ( ( ) ( ) The intersection of this line with the x-axis is the -intercept say ( ( ) ) ( ( ) Check the functional value of If ( ) ( ) ) ( ( ) ) ( ( ) then x3 is the exact root of ( ) 7 ) ( )) is given by ( ) which is the first estimate of the root; If ( ( ) the root is in [ , ] (considering ( ) ), In this case join the points ( ( )) using a chord to get the next better approximate root . ( )) & Continue the process by bracketing the root until the root is estimated adequately by; ( ) ( ( ) ) ( ) , for the interval , - that contains the root Condition of convergence Theorem: , Assume that is continuous on [a, b] and there exists a number opposite signs, and ( ( ) ( - such that ( ) . If ( ) ( ) have ) ) ( ) represents the sequence of points generated by the false position process, then the sequence zero . That is, . converges to the Termination criteria for a given tolerance of error : Here, we use the size of | ( )| as the stopping criterion for | ( )| it. Example: Find a root of the equation in [2, 3], using regula falsi method up to 4th iteration. ( ) Solution: The formula for the method is - ( ) Since ( ) ) ( Now find ( ) ) ) ( ) ( ) ( ( ( ) the root lies b/n 2 & 3 = ( = ) ( ) Now use ( ) ( ( ( ) ) ( the root is in , )( - ) ) etc. Remark: This method Is faster than bisection method Has a limitation when there is a discontinuity over an interval, when there are distinct roots and over a "large" interval. Open Methods:-these methods do not contain the root by an interval. 8 a)The Secant method: The secant method is an open method of root-finding algorithm that uses a succession of roots of secant lines to better approximate a root of a function f. It resembles totally the method of false position except that no attempt is made to ensure the root is enclosed. This method needs two initial values (initial approximate roots). Procedure: of Let ( ) i.e be the root of ( , and assume ( )) . Draw the secant line passing through( -axis, say ( ) ( be two initial approximate roots ( )). Find the point at which the line crosses the and from the slope of the line, we have ) ( ) ( ) - ( ) Further approximations of – ( ) ( ( ( ( ) ) ( ) are computed using the iteration; ) ) ( ) Condition of convergence Theorem: The iterates of the secant method converge to a root of f, if the initial values and are sufficiently close to the root. If the initial values are not close to the root, then there is no guarantee that the secant method converges. Termination criteria within a given tolerance : The stopping criterion for this method is the size of | ( Example1: Use the secant method to approximate the root of & & | ( )| < = ( ) Solution: ( ( ), ) ( ), ( ) ( ) ( ) , ( ( ) ( ) ) . and , ( ) - 9 )| with initial approximations And ( ) – ( ), ( ) and ( ) ( ) and ( ) . . . so on. accurate to within 10-4. Example2: Use Secant Method to find a root of the equation Solution: ( ) Taking , the root lies between . and x2 x1 x1 x0 * f ( x1 ) f ( x1 ) f ( x0 ) = 0.61108428 x3 x 2 x2 x1 * f ( x2 ) f ( x2 ) f ( x1 ) = 0.72328609 x 4 x3 x3 x 2 * f ( x3 ) f ( x3 ) f ( x 2 ) = 0.73956633 x5 x 4 x 4 x3 * f ( x4 ) f ( x 4 ) f ( x3 ) = 0.73908347 x 6 x5 x5 x 4 * f ( x5 ) f ( x5 ) f ( x 4 ) = 0.73908514 x6 x5 .0001 and hence the root to the given accuracy is : 0.73908347 Remark: In this method We are not sure about the convergence of the approximated roots to the exact root. But, the process is simpler; because the sign of ( ) is not tested, and often converges faster. Exercise: Use the method of regula falsi method to find the root | ( )| in the interval , - ( ) until (Hint: Use radian measure). c) Fixed point Iteration This method is also known as the direct substitution method. To find the root of the equation ( ) successive approximations we rewrite the given equation in the form of ( ). A number is a fixed point for a given function equivalent classes in the following sense: if ( ) 10 by . Root-finding problems and fixed point problems are Given a root finding problem ( ) , we can define functions with a fixed point at in a number of ways, ( ) or as ( ) for example, as ( ) ( ). Conversely, if the function has a fixed point at , then the function defined by ( ) ( ) has a zero at . To approximate the fixed point of a function , we choose an initial approximation and generate the sequence * + by letting ( ), for each . If the sequence converges to is continuous, then ( ) ( ) ( ) and a solution ( ) is obtained. This technique is called fixed point iteration or functional iteration. The procedure is illustrated in the figure below. Example 1 The function ( ) and ( ) for has fixed points at There are many ways to change the equation ( ) manipulation. For example the function ( ) iv) With ( ) . ) ( ) using simple algebraic to the fixed point form can be changed to the fixed point form as ( ) i) since ( and ( ) ii) / v) . / iii) ( ) ( ) ( ) , the table below lists the results of the fixed point iterations for all five choices of . () ( ) ( ( ) ( ) 1.5 ( ) ) 1.365230013 7 8 9 11 10 15 20 25 30 The actual root is 1.365230013. It can be seen that excellent results have been obtained for choices iii), iv) and v), It is interesting to note that choice i) was divergent and that ii) became undefined because it involved the square root of a negative number. Even though the various functions in the above example are fixed point problems for the same root-finding problems, they differ vastly as techniques for approximating the solution to the root finding problem. How can we find a fixed point problem that produces a sequence that reliably and rapidly converges to a solution to a given root-finding problem? The following theorem gives us some clues concerning the paths we should pursue and, perhaps more importantly, some we should reject. Theorem 2.1. (Fixed point theorem) - for all , -. Suppose in addition that Let is continuous on , ( ) , ( ). Then for any number that a constant exists with | ( )| , for all , ( ) defined by converges to the unique fixed point Example 2: Discuss the convergence of the method for different choices of ) and exists on ( , - the sequence on example 1 above. a) For ( ) , we have ( ) and ( ) , so does no map , - in to itself. , -. Although theorem 2.1 does not guarantee that ( ) Moreover, , so | ( )| for all the method must fail for this choice of , there is no reason to expect convergence. b) With ( ) . defined when | ( )| does not map , / , we can see that - in to , . Moreover there is no interval containing , since | c) For the function ( )| ( ) - and the sequence * + is not such that . There is no reason to expect this method will convergence. ( ) , ( ) ( , ) -, So is strictly decreasing on , -. However,| ( )| , so the condition | ( )| fails on , -. A closer examination of the sequence * + starting with shows that it suffice to consider the interval , - instead of , -. On this interval it is still true that ( ) is strictly decreasing, but , additionally, , - this shows that maps the interval , - on to ( ) ( ) ( ) for all itself. Since it is true that | ( )| | ( )| on this interval, theorem 2.1 confirms the convergence of which we are already aware. d) For ( ) . / , we have | ( )| | √ ( ) | for all √ 12 , - ( ) is much smaller than the bound (found in (c)) on the magnitude of The bound of the magnitude of which explains the more rapid convergence using . e) The sequence defined by ( ) ( ) converges much more rapidly than the other choices. Example3. Find all zeros of ( ) by using the fixed point iteration method for an appropriate iteration function . Find the zeros accurate to within Solution: Using graphical method or IVT it can be shown that ( ) has four roots and changes signs in each of the - , - , - , -. Therefore each of these intervals contains only one intervals , root. ( ) can be changed in to the fixed point function in many ways. Among them ( ). With the initial approximation i) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) | | | ( ) ( ) precede the iteration as follows:- we ( ) . | The first zero of ( ) accurate to within which is in , is -. For other zeros their approximation function, initial approximations are summarized in the table below. Interval , initial approximation accurate to within - approximating function ( ) ( ) ( ) , - ( ) , - ( ) 13 d) Newton - Raphson Method Newton’s method is one of the most powerful and well known numerical methods for solving a rootfinding problem. There are many ways of introducing Newton’s method. One means of introducing Newton’s method is based on Taylor Polynomial. In Newton’s method, instead of constructing the chord between two points on the curve of ( ) against (as done in linear interpolation method), the tangent to the curve is notionally constructed at each successive value of which the tangent cuts the axis ( ) If the ( ) ( ) , and the next value, value is , the tangent to the curve of ( ) at that point has slope ( )). Its equations is thus ( ( The value of scheme ) ( ) ( at which ) is taken as ( ( ) ) ( , is taken as the point at ) and passes through the point ------------------------------------- (i) thus the condition yields from (i) above , the iteration -------------------------------------- (ii) This is Newton’s – Raphson iteration formula. Remark: Note that if any of comes close to the stationary point of , so that ( ) is close to zero, the scheme is not going to work well. The iteration procedure terminates when the relative error for two successive approximations becomes less than or equal to the prescribed tolerance. 14 Eample1. Find real root of the equation b/n Solution: let ( ) and , now ( ) by Newton’s –Raphson method and . Therefore the root lies in , ( ) - Newton’s- Raphson method becomes ( ( ) ) Let 1st appx; 3rd appx: Sinec ( ) ( ( 2nd appx; ( ) ( 4th appx: ) ) ( ) ) ( ) , The root of the equation is 4.5616. Example 2: Use the Newton’s – Raphson method to estimate the root of ( ) of employing an initial guess ( ) Solution. The first derivative of the function can be evaluated as This can be substituted along with the original function in to (ii). Starting with an initial guess of , this iterative equation can be applied to compute | | 0.0000220 Thus, the approach rapidly converges on the true root. Exercises 1. Use the Bisection method to find a solution accurate to with in i) for the following problems ii) 2. Let ( ) . With a) Use the secant method and , find b) Use the method of false position 3. Using the a) secant method b) Method of false position c) Newton’s Method to find the solution accurate to with in , for the following problems i) 15 - ii) 0 1 4. Using the a) secant method b) Method of false position c) Newton’s Method to find the solution accurate to with in for the following problems i) ii) 5. Employ fixed point iteration to locate the root of ( ) until (√ ) 6. Find √ using Newton – Raphson method taking use an initial guess of and iterate . 7. Use a fixed point iteration method to determine a solution accurate to with in for on , -. Use 8. Use a fixed point iteration method to determine a solution accurate to with in ( ) 9. Determine the lowest positive root of ( ) a) Using the secant method (three iterations, b)Using the Newton – Raphson method (three iterations 16 ) for , - CHAPTER THREE Solutions of System of equations A system of algebraic equations has the form (3.1) where the coefficients and the constants notation the equations are written as are known, and represent the unknowns. In matrix (3.2) or, simply (3.3) There are two classes of methods for solving systems of linear, algebraic equations: direct and iterative methods. The common characteristic of direct methods is that they transform the original equations into equivalent equations (equations that have the same solution) that can be solved more easily Iterative, or indirect methods, start with a guess of the solution x, and then repeatedly refine the solution until a certain convergence criterion is reached. Iterative methods are generally less efficient than their direct counterparts due to the large number of iterations required. But they do have significant computational advantages if the coefficient matrix is very large and sparsely populated (most coefficients are zero). 3.1 Direct methods for solving system of linear equations From the direct methods we have studied Gauss elimination, Gauss-Jordan elimination and matrix inversion methods in our previous courses. Here, form the direct methods we concentrate on matrix decomposition methods LU Decomposition Methods It is possible to show that any square matrix A can be expressed as a product of a lower triangular matrix L and an upper triangular matrix U: (3.4) 1|Page The process of computing and for a given A is known as decomposition or factorization. decomposition is not unique (the combinations of and for a prescribed are endless), unless certain constraints are placed on or . These constraints distinguish one type of decomposition from another. Three commonly used decompositions are listed in Table 3.2. Name Constraints Doolittle’s decomposition = 1, i =1,2,...,n Crout’s decomposition =1, i =1,2,...,n Choleski’s decomposition Table 3.1 After decomposing A, it is easy to solve the equations . We first rewrite the equations as . Upon using the notation , the equations become which can be solved for y by forward substitution. Then will yield x by the back substitution process. The advantage of decomposition over the Gauss elimination method is that once is decomposed, we can solve for as many constant vectors b as we please. The cost of each additional solution is relatively small, since the forward and back substitution operations are much less time consuming than the decomposition process. 3.1.1 Doolittle’s Decomposition Method Decomposition Phase Doolittle’s decomposition is closely related to Gauss elimination. In order to illustrate the relationship, consider a 3×3 matrix A and assume that there exist triangular matrices such that A = LU. After completing the multiplication on the right hand side, we get Let us now apply Gauss elimination to Eq. (above). The first pass of the elimination procedure consists of choosing the first row as the pivot row and applying the elementary operations 2|Page , The result is In the next pass we take the second row as the pivot row, and utilize the operation ending up with The foregoing illustration reveals two important features of Doolittle’s decomposition: o o The matrix U is identical to the upper triangular matrix that results from Gauss elimination. The off-diagonal elements of L are the pivot equation multipliers used during Gauss elimination; that is, is the multiplier that eliminated . It is usual practice to store the multipliers in the lower triangular portion of the coefficient matrix, replacing the coefficients as they are eliminated ( replacing ). The diagonal elements of L do not have to be stored, since it is understood that each of them is unity. The final form of the coefficient matrix would thus be the following mixture of L and U: Solution Phase: Consider now the procedure for solving of the equations is (recall that ) Solving the equation for by forward substitution. The scalar form yields (3.5) 3|Page The back substitution phase for solving is identical to that used in the Gauss elimination method. EXAMPLE 3.1 Use Doolittle’s decomposition method to solve the equations ,where Solution: We first decompose A by Gauss elimination. The first pass consists of the elementary operations Storing the multipliers and in place of the eliminated terms, we obtain The second pass of Gauss elimination uses the operation Storing the multiplier in place of , we get The decomposition is now complete, with Solution of equations is 4|Page by forward substitution comes next.The augmented coefficient form of the The solution is Finally, the equations , or are solved by back substitution. This yields 3.1.2 Choleski’s Decomposition Choleski’s decomposition o o has two limitations: Since is always a symmetric matrix, Choleski’s decomposition requires A to be symmetric. The decomposition process involves taking square roots of certain combinations of the elements of A. It can be shown that in order to avoid square roots of negative numbers A must be positive definite. Although the number of multiplications in all the decomposition methods is about the same, Choleski’s decomposition is not a particularly popular means of solving simultaneous equations due to the restrictions listed above. We study it here because it is invaluable in certain applications. Let us start by looking at Choleski’s decomposition (3.6) of a 3×3 matrix: After completing the matrix multiplication on the right hand side, we get 5|Page (3.7) Note that the right-hand-side matrix is symmetric, as pointed out before. Equating the matrices and element-by-element, we obtain six equations (due to symmetry only lower or upper triangular elements have to be considered) in the six unknown components of L. By solving these equations in a certain order, it is possible to have only one unknown in each equation. Consider the lower triangular portion of each matrix in Eq. (3.7) (the upper triangular portion would do as well). By equating the elements in the first column, starting with the first row and proceeding downward, we can compute , and in that order: The second column, starting with second row, yields Finally the third column, third row gives us We can now extrapolate the results for an triangular portion of is of the form and : : matrix. We observe that a typical element in the lower- Equating this term to the corresponding element of yields (3.8) The range of indices shown limits the elements to the lower triangular part. For the first column (j =1), we obtain from Eq. (3.8) (3.9) 6|Page Proceeding to other columns, we observe that the unknown in Eq. (3.8) is (the other elements of L appearing in the equation have already been computed). Taking the term containing outside the summation in Eq. (3.8), we obtain If (a diagonal term), the solution is (3.10) For a non-diagonal term we get (3.11) EXAMPLE 3.2 Compute Choleski’s decomposition of the matrix Solution: First we note that A is symmetric. Therefore, Choleski’s decomposition is applicable, provided that the matrix is also positive definite. An a priori test for positive definiteness is not needed, since the decomposition algorithm contains its own test: if the square root of a negative number is encountered, the matrix is not positive definite and the decomposition fails. Substituting the given matrix for A in Eq. (3.7), we obtain Equating the elements in the lower (or upper) triangular portions yields 7|Page Therefore, The result can easily be verified by performing the multiplication . 3.2 Iterative Methods So far, we have discussed only direct methods of solution. The common characteristic of these methods is that they compute the solution with a finite number of operations. Moreover, if the computer were capable of infinite precision i.e. no round off errors, the Solution would be exact. Iterative, or indirect methods, start with an initial guess of the solution x and then repeatedly improve the solution until the change in x becomes negligible. Since the required number of iterations can be very large, the indirect methods are, in general, slower than their direct counterparts. However, iterative methods do have the following advantages that make them attractive for certain problems: o o For large systems with a high percentage of 0 entries, these techniques are efficient in terms of both computer storage and computation. Iterative procedures are self-correcting, meaning that round off errors(or even arithmetic mistakes)in one iterative cycle are corrected in subsequent cycles. A serious drawback of iterative methods is that they do not always converge to the solution. It can be shown that convergence is guaranteed only if the coefficient matrix is diagonally dominant. The initial guess for x plays no role in determining whether convergence takes place—if the procedure converges for one starting vector, it would do so for any starting vector. The initial guess affects only the number of iterations that are required for convergence. Diagonal Dominance An matrix is said to be diagonally dominant if each diagonal element is larger than the sum of the other elements in the same row(we are talking here about absolute values). Thus diagonal dominance requires that 8|Page For example, the matrix Is not diagonally dominant, but if we rearrange the rows in the following manner Then we have diagonal dominance. 3.2.1 Gauss Jacobi Method The Gauss Jacobi iterative method is obtained by solving the obtain (provided For each equation in for to ) , generate the components of from the components of by (3.12) Example 3.3: The linear system given by has the unique solution to starting with 9|Page . Use Jacobi’s iterative technique to find approximations until Solution: We first solve equation for , for each From the initial approximation Additional iterates, below. to obtain we have ( given by ) are generated in a similar manner and are presented in Table We stopped after ten iterations because In fact . 3.2.2 The Gauss-Seidel Method A possible improvement of Gauss Jacobi can be seen by reconsidering Eq. (3.12). The components of are used to compute all the components 10 | P a g e of . But, for , the components ,..., of have already been computed and are expected to be better approximations to the actual solutions than are . It seems reasonable, then, to compute recently calculated values. That is, to use using these most (3.13) for each instead of Eq. (3.12). This modification is called the Gauss-Seidel iterative techniqueand is illustrated in the following examples. Example 3.4 Solve the equations by the Gauss–Seidel method. Solution: With the given data, the iteration formulas in Eq. (3.13) become Choosing the starting values The second iteration yields 11 | P a g e , we have for the first iteration and the third iteration results in After five more iterations the results would agree with the exact solution five decimal places. with in Example 3.5 Use the Gauss-Seidel iterative technique to find approximate solutions to starting with and iterating until Solution The solution was approximated by Gauss Jacobi method in Example 3.3. For the Gauss-Seidel method we write the system, for each as 12 | P a g e When ,wehave give the values in table below . Subsequent iterations is accepted as a reasonable approximation to the solution. Note that Gauss Jacobi method in Example3.3 required twice as many iterations for the same accuracy. Exercises 13 | P a g e 4.2. Interpolation ( ) The statement means corresponding to every value of in the range there exists one or more values of . Assuming that ( ) is single valued and continuous and that is known explicitly, then the values of ( ) corresponding to certain given values of , say can easily be computed and tabulated. The central problem of numerical analysis is the )( ) ( ) ( ) satisfying the converse one; given the set of tabular values ( ( ) where the explicit nature of ( ) is not known, it is required to find a simpler function relation say ( ), such that ( ) ( ) agree at the set of tabulated points. Such a process is called interpolation. If ( ) is a polynomial, then the process is called polynomial interpolation and ( ) is called the interpolating polynomial. Polynomial interpolation consists of determining the unique order polynomial that fits n+1 data points. This polynomial then provides a formula to compute intermediate values. Although there is one and only one nth order polynomial that fits n+1 points, there are a variety of mathematical formats in which this polynomial can be expressed. 4.2.1 Newton’s formula for interpolation 4.2.1.1 Newton’s forward difference interpolation )( ) ( ) ( Given the set of values, i.e. ( find ( ) a polynomial of the degree such that the functions points. Let the values of be equidistant, i.e. This means step size. Since ( ) ( ) ; where is the interval of differencing called degree, it may be written as ( ) is a polynomial of the ( ) ( )( ) and substituting ( ) ( )( ) ( ) (4.1) ( ) should agree at the set of tabulated points, we obtain Imposing now the condition that Setting ) of it is required to ( ) agree at the tabulated equation (4.1) gives ( )( ) ( )( ) ( ( )) (4.2) (4.2) is called Newton’s forward interpolation formula and useful for interpolation near the beginning of a set of tabulated values. 1|Page Example1. Find the cubic polynomial which takes the following values. Hence, or otherwise obtain ( ) x 0 1 2 3 y 1 0 1 10 Solution Form the difference table x 0 y 1 1 0 2 1 -1 2 1 3 Here gives 6 8 9 10 . Hence using the formula . Substituting this value of ( ) ( and choosing in (4.2) we get ( ) ) ( )( we obtain this ) This is a polynomial from which we obtain the above tabular value. To compute y(4) we observe that , hence formula (4.2)gives ( ) ( ) ( ) ( )( ) That is the same value as that obtained by substituting we call extrapolation in the cubic polynomial above. This is what 4.2.1.2 Newton’s backward difference interpolation Instead of assuming ( ) ( ( ) as in (4.1) if we choose it in the form ) Imposing the condition that (4.3) we obtain ( )( ) ( )( ) ( ( ) should agree at the tabulated points and setting and substituting 2|Page in equation (4.3) gives ) (4.3) in ( ( ) ) ( )( ) ( )( ) ( ( )) (4.4) This is Newton’s backward difference formula and it uses tabular values to the left of therefore useful for interpolation near the end of the tabular values. . This formula is Example 2 The population of a town in the decennial census was given below. Estimate the population for the year 1925 year x 1891 1901 1911 1921 1931 population y (in thousands) 46 66 81 93 101 Solution Here interpolation is desired at the end of the table and so we use formula (4.4) with with we obtain ( ) The backward difference table x 1891 y 46 1901 66 20 2 15 1911 81 1921 93 12 8 101 1931 Hence using formula (4.4) ( ( ) ( ( ) ( ) )( ( )( )( )( )( ) )( ) ( ( )( )( Therefore the population in the year 1925 is )( )( )( )( ) ) ) thousands. Additional examples 3. Find ( ) given that ( ) 3|Page ( ) ( ) ( ) The third difference being constant. Solution Since ( ) ( ) ( ( ) ) ( ) ( ) ( ) ( ) ( ) ( ) Difference table x y ( ) 0 ( ) ( ) 9 1 6 9 2 2 8 4 3 12 Higher differencing being zero We have ( ) ( ( ) ) ( ) ( ) ( ( ) ) ( ) ( ) ( ) ( ) ( ) ( ) 4. Construct the backward difference table from the data , assuming the third difference to be constant, find the value of Solution First construct the backward difference table x 25 y 30 0.07336 35 0.0692 40 45 4|Page 0.0643 ( Therefore, ( ) ( ) ( ( ( ( ) ( ) ) ( ) then and ) ) ) ( ) this yields ( this yields ( ) and ) Exercises 1. Evaluate ( a) 2. Prove that ( ) ( )) ( ) b) ( ) 3. The table below gives the values of x Find i) 0.10 0.1003 ii) 0.15 0.20 0.1511 0.2027 iii) ( ) ( ) ( ) for 0.25 0.30 0.2553 0.3093 iv) 4.2.2 Lagrange’s Interpolation formula The disadvantage of Newton’s interpolation formula is it requires the values of the independent variable to be equally spaced .The Lagrange’s interpolation formula is used to interpolate with unequally spaced values of the argument. ), given the Let ( ) be continuous and differenciable times in the interval ( points ( )( ) ( ) ( ) where the values of need not necessarily be equally spaced, we wish to find a polynomial of degree , ( ) such that ( ) ( ) (4.5) a) First consider the Lagrange’s polynomial of degree one passing through two points ( ) ( ) Let the polynomial be ( ) such that ( ) Where From (4.7) we see that 5|Page ( ) ( ) ( ) ( ) { ( ) ∑ ( ) (4.6) (4.7) (4.8) Therefore ∑ ( ) ( ) ( ) (4.9) b) Lagrange’s polynomial of degree two passing through ( )( ) ( ) Similarly the quadratic Lagrange’s polynomial is written as ( ) ∑ ( ( ) )( ( )( ) ( ) )( ( ) )( ( ) ( )( ) )( (4.10) ) Where ( ) satisfy the condition given in (4.8) and (4.9) To derive the general formula, let ( ) (4.11) be the desired polynomial of the degree such that the condition (4.5)[called the interpolatory conditions] are satisfied. Substituting these conditions in (4.11) (4.12) The set of equation(4.12)has a solution if | | Equation (4.11) is a linear combination of ( ) ∑ Where ( ) are polynomials in Equation (4.9) gives ( ) ( ( ( ) (4.13) ( ) of degree . Since ( ) for . Hence ( ) may be written as ( ) )( )( . Hence we write ) ) ( ( )( )( ) ( ) ) ( ) )( )( ( which satisfies condition (4.8) If we now set 6|Page ( ) ( )( ) ( ) ( ) ) ( ) ( ) Then ( ) ( ( ) ) ( ) ( ) ( | )( ) ( )( ) ( so that , therefore (4.13) becomes ( ) ∑ ( ( ) ) ( ) ( ) This is called Lagrange’s interpolation formula. The coefficients ( ) defined in (4.14) are called Lagrange’s interpolation coefficient. Interchanging in (4.15) we obtain the formula ( ) ( ) ∑ ( ( ) ) (4.16) is useful for inverse interpolation. The major advantage of this formula is that the coefficients in (4.18) are easily determined. Further it is applicable to either equal or unequal intervals and the abscissae need not be in order. Example 5. Certain corresponding values of are given in the table below. Find 300 304 305 307 2.4771 2.4829 2.4843 2.4871 Solution From formula (4.15) we obtain ( ( ( ( )( )( )( )( )( )( )( )( ) ( ) ) ( ) ( ( ( ( ) ) )( )( )( )( )( )( )( )( ) ( ) ) ( ) ) ) Example 6 Find the Lagrange’s interpolating polynomial of degree2 approximating the function defined by the following table of values. Hence determine the value of Solution ( ) ∑ 2 0.69315 ( ) ( )( )( ( ( ( )( )( ( 7|Page ) ( ) 2.5 0.91629 ( ) ) )( ( ( ) ) 3.0 1.09861 ( ) ( ) )( )( )( )( ( ( ( ) ) ) ) ( ( ( ) )( ) ) )( )( ) ) ( ( ( ( ) ) )( ) )( ( ) ) . This is the required polynomial ( ) . The actual value of So that | | Example 7 Using Lagrange’s interpolation formula, find the form of the function ( ) form the table 0 -12 1 0 3 12 Solution Since ( ) 4 24 , it follows that is a factor. Let ( ) ( ) ( ). Then and ( ). . We now tabulate the values of 0 3 4 ( ) 12 6 8 Applying Lagrange’s formula to the above table, we find ( ( ) ( ( )( )( )( ) ) ( ( ( ) ) )( )( ( ) ( ) ) ) ( ( ( )( )( ) ( ) ) ) Hence the required polynomial approximation to ( ) is given by ( ) ( )( ) 4.2.3 Errors in polynomial interpolation ) Let the function ( ) defined by the ( ) points ( differentiable times and let ( ) be approximated by a polynomial such that be continuous and ( ) of degree not exceeding ( ) (4.17) Now use ( ) to be approximate value of ( ) at some points other than those defined by (4.17). Since the expression ( ) ( ) vanishes for we put ( ) ( ) ( ) (4.18) Where ( ) And ( )( ) ( ) (4.19) is to be obtained such that (4.18) hold for any intermediate value of , say clearly ( ) ( ( ) 8|Page ) (4.20) We construct a function ( ) such that ( ) ( ) ( ) ( ) (4.21) ( ) ( ) ( ) Where is given in (4.20).It is clear that ( ) , that is ( ) vanished (n+2) times in the interval ; Consequently, by the repeated application of Rolle’s theorem ( ) must vanish (n+1)times , ( ) must vanish n times , etc, in the interval ; In particular, ( ) ( )must vanish once in the interval. Let this point be given by differentiating (4.21) n+1times with respect to and putting , we obtain ( ) ( ) ( ) So that ( . On ) ( (4.22) ) On comparison of (4.20) and (4.22), we get ( ) Where ( ) ( ( ) ( ) )( ) (4.23) . This is the required expression for error. 4.2.4 Error in Lagrange’s interpolation formula To determine the error of Lagrange’s interpolation formula for the class of functions which have continuous derivatives of order up to ( ) on , we use equation (4.23) ( ) ( ) ( ) ( ( ) ( ) )( ) , and the quantity may be taken as estimate of error. Further if we assume that | ( | ) ( ) | ( )| where ( )| , then ( )| ( ) from the data given using Lagrange’s interpolation formula. Hence Example 8 Find the value of estimate the error in the solution 0 0 ( ) Solution Now, ( ) | ( )| , | 9|Page ( )( ( )( ) ( )( ) ( ) )( ( ) ( ) ) | ( )( ) ( )( ) ( ) ( ) ( ) , hence ( ) which agrees with the actual error in problem , when Example 9 Values of elliptic integral ( ) ( ) for which Find √ ∫ are given below √ ( ) Solution By Lagrange’s inverse interpolation formula ( ( )( )( ) ) ( ( ( ( )( )( ) ) ( ( )( )( ( ( )( )( ) ) ) ) ( ( )( )( )( )( ) ) ) ) 4.2.4 Divided difference Lagrange’s interpolation formula has the disadvantages that if any other interpolation point were added, the interpolation coefficient will have to be recomputed. So an interpolation polynomial which has the property that a polynomial of higher degree may be obtained from it by simply adding new terms has to be derived. Newton’s general interpolation formula is one such formula and it employs what are called divided difference. Let [ be the values of the function corresponding to values of the argument which are equally spaced. Then the divided difference of order 1, 2, . . .,n are given as ] [ ] [ ] and so on [ The second order divided difference is given as ] [ ] [ ] [ and so on. The third order divided difference for is given by and the order divided difference is Now let the arguments be equally spaced so that [ ] 10 | P a g e [ ] [ , then ] ( ) and in general ] . If the tabulated function is a polynomial of nth degree, would be a th constant and the n divided difference would also be constant. Newton’s divided difference formula From the definition of divided difference we’ve ; So that ( Again ) (4.24) , which gives ( ) (4.25) From (4.24) and (4.25) we have ( )([ ] Also ( ) ) (4.26) , which gives ( ) )( ) ( )( (4.27) From (4.26) and (4.27) ( ) ( ( )( )( ) Proceeding in this manner, we get ( ) ( ( ( (( ) )( )( )( )( ) )( ) ) ) ( ( ) )) (4.28) This is called Newton’s general interpolation formula with divided difference the last term being the remainder y term after (n+1) terms. Newton’s divided difference formula can also written as 11 | P a g e ( ( ) ( )( )( )( ) ) ( ( )( )( ) ) (4.29) are the 1st, the 2nd, the 3rd, . . . divided difference operators respectively Where Example 9 Construct a divided difference table for the following data 1 2 ( ) 22 30 4 82 7 12 106 216 Solution ( ) 1 22 2 30 4 82 ( ) ( ) ( ) ( 7 106 12 216 ( ) ) Example 10 Using Newton’s divided difference formula calculate the value of ( ) from the following data 1 2 7 8 ( ) 1 5 5 4 The divided difference table is 1 ( ) 1 2 5 ( ) ( ) ( ) 4 0 7 5 8 4 Applying Newton’s divided difference formula, ( ) ( ( ) 12 | P a g e ( ) )( ( ) )( ( ) ) ( ( )( )( )( ( )( )( ) ) ) Exercises 1. Applying Lagrange’s formula find cubic polynomial which approximate the following data ( ) 2. From the following table find 2 3 3 5 for which 2.6 2.8 6.695 8.198 3. The following table is given 0 ( ) 2 1 3 3.0 10.018 2 12 5 147 What is the form of the function? 4. Using Newton’s divided difference method compute ( )from the following table 0 1 2 4 5 ( ) 1 14 15 5 6 5. The following table gives the value of and 1.2 2.1 2.8 4.2 6.8 9.8 Find the value of corresponding to 13 | P a g e 4.1 13.4 6 19 4.9 6.2 15.5 19.6 using Lagrange’s technique of inverse interpolation