What is Fixed-Point Iteration?
In numerical analysis, fixed-point iteration is a method for finding the fixed points of a function.
A fixed point of a function f is a value x where f(x) = x. In simpler terms, it's a point that doesn't
change when you apply the function to it.
How Does it Work?
1. Rearrange the Equation: Start with an equation you want to solve (often in the form f(x)
= 0) and rearrange it into the form x = g(x). There might be multiple ways to do this.
2. Initial Guess: Choose a starting value x₀ (your initial guess for the fixed point).
3. Iterate: Repeatedly apply the function g to your previous guess:
x₁ = g(x₀)
o x₂ = g(x₁)
o x₃ = g(x₂)
o ...and so on.
4. Convergence: If the process converges, the sequence of values x₀, x₁, x₂, ... will approach
the fixed point.
o
Why Does it Work?
The idea is that if g is continuous and your initial guess is close enough to the fixed point,
repeatedly applying g will "pull" the values closer and closer to that fixed point.
Important Considerations
•
Convergence: Fixed-point iteration doesn't always converge. The choice of how to
rearrange the equation (x = g(x)) and your initial guess can greatly affect whether it
converges and how quickly. A key condition for convergence is that the absolute value of
the derivative of g near the fixed point must be less than 1.
•
Multiple Fixed Points: A function might have multiple fixed points. The iteration will
converge to one of them, and which one depends on your initial guess.
•
Speed: Even when it converges, fixed-point iteration can be slow compared to other
methods.
Example
Let's say we want to solve f(x) = x² - x - 2 = 0.
1. Rearrange: We can rearrange this as x = √(x + 2). So, g(x) = √(x + 2).
2. Initial Guess: Let's start with x₀ = 3.
3. Iterate:
x₁ = g(x₀) = √(3 + 2) = √5 ≈ 2.236
o x₂ = g(x₁) = √(2.236 + 2) ≈ 2.058
o x₃ = g(x₂) ≈ 2.014
o ...and so on.
If we continue this process, the values will get closer and closer to 2, which is a fixed point of
g(x) and a solution to the original equation f(x) = 0.
o
Applications
Fixed-point iteration is used in various areas, including:
•
•
•
Root finding: As shown in the example, it can be used to find solutions to equations.
Numerical solutions of differential equations: Some methods for solving differential
equations rely on fixed-point iteration.
Optimization: Certain optimization algorithms use fixed-point iteration as a component.
What is the Newton-Raphson Method?
The Newton-Raphson method is an iterative technique that uses the derivative of a function to
approximate its roots. It's based on the idea of using the tangent line to a function at a given
point to estimate where the function crosses the x-axis.
How Does it Work?
1. Initial Guess: Start with an initial guess x₀ for the root. The closer your guess is to the
actual root, the better the method's performance will generally be.
2. Iteration: Repeat the following steps until a desired level of accuracy is reached:
Calculate the function value f(xₙ) and its derivative f'(xₙ) at the current guess xₙ.
o Calculate the next guess xₙ₊₁ using the formula:
xₙ₊₁ = xₙ - f(xₙ) / f'(xₙ)
o
•
This formula is derived from finding where the tangent line to the function at xₙ
intersects the x-axis.
3. Convergence: The sequence of guesses x₀, x₁, x₂, ... will ideally converge to a root of the
function.
Why Does it Work?
The Newton-Raphson method works because it uses the tangent line to approximate the
function near the current guess. The tangent line is a good local approximation of the function,
and its x-intercept is usually closer to the root than the current guess. By repeatedly finding the
x-intercept of the tangent line, the method gets closer and closer to the actual root.
Geometric Interpretation
Imagine the graph of the function f(x). At a point (xₙ, f(xₙ)), draw the tangent line to the curve.
The point where this tangent line intersects the x-axis is the next guess, xₙ₊₁. The method then
repeats this process, drawing a new tangent line at (xₙ₊₁, f(xₙ₊₁)) and finding its x-intercept.
Example
Let's say we want to find a root of the function f(x) = x² - 5.
1. Initial Guess: Let's start with x₀ = 3.
2. Iteration:
o
o
f(x₀) = 3² - 5 = 4
f'(x₀) = 2 * 3 = 6
x₁ = x₀ - f(x₀) / f'(x₀) = 3 - 4/6 = 2.333...
o f(x₁) = (2.333...)² - 5 ≈ 0.444...
o f'(x₁) = 2 * (2.333...) ≈ 4.666...
o x₂ = x₁ - f(x₁) / f'(x₁) ≈ 2.238...
If we continue this process, the iterates will converge quickly to √5 ≈ 2.236.
o
Advantages of the Newton-Raphson Method
•
Fast Convergence: When it converges, it usually converges very quickly (quadratically,
meaning the number of correct digits roughly doubles with each iteration).
• Widely Applicable: It can be used to find roots of a wide variety of functions.
Disadvantages of the Newton-Raphson Method
•
•
•
•
•
Requires Derivative: You need to be able to calculate the derivative of the function. This
can be difficult or impossible for some functions.
May Not Converge: The method is not guaranteed to converge. The choice of initial
guess is crucial. Bad initial guesses can lead to divergence, oscillation, or convergence to
a different root.
Sensitive to Initial Guess: As mentioned, the convergence depends heavily on the initial
guess.
Division by Zero: If f'(xₙ) is zero, the method will fail due to division by zero. This can
happen if the initial guess is near a local maximum or minimum of the function.
Convergence to Unintended Root: If the function has multiple roots, the method may
converge to a root other than the one you were looking for.
The secant method is a root-finding algorithm that's closely related to the Newton-Raphson
method. It's particularly useful when you can't or don't want to calculate the derivative of the
function you're working with.
What is the Secant Method?
Like the Newton-Raphson method, the secant method is an iterative process used to
approximate the roots of a function f(x). However, instead of using the derivative f'(x), it
approximates the derivative using a difference quotient.
How Does it Work?
1. Two Initial Guesses: The secant method requires two initial guesses, x₀ and x₁, rather
than just one. These guesses should ideally be close to the root you're trying to find.
2. Iteration: The core of the secant method is the following iterative formula:
xₙ₊₁ = xₙ - f(xₙ) * (xₙ - xₙ₋₁) / (f(xₙ) - f(xₙ₋₁))
Notice the term (f(xₙ) - f(xₙ₋₁)) / (xₙ - xₙ₋₁). This is a difference quotient that
approximates the derivative f'(xₙ). It represents the slope of the secant line passing
through the points (xₙ₋₁, f(xₙ₋₁)) and (xₙ, f(xₙ)).
3. Convergence: The sequence x₀, x₁, x₂, ... will ideally converge to a root of f(x).
Why Does it Work?
The secant method works by approximating the tangent line used in the Newton-Raphson
method with a secant line. A secant line is a line that intersects the curve of the function at two
points. The slope of this secant line provides an approximation of the derivative, allowing us to
estimate where the function crosses the x-axis.
Geometric Interpretation
Imagine the graph of f(x). Choose two points on the curve, (xₙ₋₁, f(xₙ₋₁)) and (xₙ, f(xₙ)). Draw the
line (the secant line) that passes through these two points. The point where this secant line
intersects the x-axis is the next guess, xₙ₊₁. The method then repeats this process, using the new
point and the previous point to define a new secant line.
Example
Let's use the same example as before: f(x) = x² - 5.
1. Initial Guesses: Let's start with x₀ = 3 and x₁ = 2.5.
2. Iteration:
x₂ = x₁ - f(x₁) * (x₁ - x₀) / (f(x₁) - f(x₀)) = 2.5 - (-1.25) * (-0.5) / (-1.25 - 4) ≈ 2.286
o x₃ = x₂ - f(x₂) * (x₂ - x₁) / (f(x₂) - f(x₁)) ≈ 2.236
If we continue this process, the iterates will converge to √5 ≈ 2.236.
o
Advantages of the Secant Method
•
No Derivative Required: The main advantage is that you don't need to calculate the
derivative of the function. This is very helpful when the derivative is difficult or
impossible to compute.
•
Simpler Formula: The formula for the secant method is often easier to implement than
the Newton-Raphson formula, especially if the derivative is complicated.
Disadvantages of the Secant Method
•
Slower Convergence: The secant method typically converges slower than the NewtonRaphson method (its convergence is superlinear, while Newton-Raphson is quadratic). It
still converges faster than linear convergence.
•
Two Initial Guesses: It requires two initial guesses, which might be less convenient than
just one.
•
Potential for Instability: Like the Newton-Raphson method, the secant method can be
sensitive to the initial guesses. Poor choices can lead to divergence or convergence to
the wrong root. If the difference between f(xₙ) and f(xₙ₋₁) is very small, the method can
also become unstable due to division by a very small number.