Introduction to Simulation - Lecture 8
1-D Nonlinear Solution Methods
Jacob White
Thanks to Deepak Ramaswamy Jaime Peraire, Michal
Rewienski, and Karen Veroy
Outline
• Nonlinear Problems
– Struts and Circuit Example
• Richardson and Linear Convergence
– Simple Linear Example
• Newton’s Method
–
–
–
–
–
Derivation of Newton
Quadratic Convergence
Examples
Global Convergence
Convergence Checks
Nonlinear
problems
( x0 , y0 )
( x2 , y2 )
Strut Example
( x1 , y1 )
Given: x0, y0, x1, y1, W
Find: x2, y2
Load force
W
Need to Solve ¦ f x
0
¦f
y
W
0
Struts Example
Nonlinear Problems
Y
Reminder: Strut Forces
L0 L
EAc
L0
f
fx
f
(0,0)
L
x1 , y1
fy
fx
X
fy
L
SMA-HPC ©2003 MIT
H L0 L x1
f
L
y1
f
L
2
1
2
1
x y
Nonlinear
problems
Strut Example
( x1 , y1 )
( x0 , y0 )
2
x2 x0 y2 y0 L1
2
x2 x1 y2 y1 L2
f1
( x2 , y2 )
x2 x0
H ( Lo L1 )
L1
x2 x1
H ( Lo L2 )
L2
f1x
f2
f2 x
Load force
W
¦f
1x
f2 x
0
2
¦f
1y
f2 y W
0
2
Nonlinear
problems
Strut Example
Why Nonlinear?
y2 y1
H ( Lo L2 ) L2
y2 y0
H ( Lo L1 ) W
L1
Pull Hard on the
Struts
The strut forces change
in both magnitude and
direction
0
Nonlinear
problems
v1
10v
Circuit Example
v2
10
1
I r Vr
10
+
+
- Vd
-
Vd
I d I s (e
Need to Solve
Id Ir
I vsrc I r
0
0
Vt
0
1)
0
Nonlinear
problems
Solve Iteratively
Hard to find analytical solution for
f ( x)
0
Solve iteratively
0
guess at a solution x
x0
repeat for k = 0, 1, 2, ….
x
k 1
W x
k
k 1
f
x
|0
until
Ask
• Does the iteration converge to correct solution ?
• How fast does the iteration converge?
Richardson
Iteration
Definition
Richardson Iteration Definition
x
k 1
k
k
x f (x )
An iteration stationary point is a solution
x
k 1
x
k
Ÿ f ( xk )
Ÿ xk
0
x* ( Solution)
Richardson
Iteration
Example 1
f ( x)
Start with
x
0.7 x 10
0
0
x1
x 0 f ( x 0 ) 10
x
x2
x1 f ( x1 ) 13
x 6 14.27
x3
x 2 f ( x 2 ) 13.9
x7
x4
x3 f ( x3 ) 14.17
x8 14.28
5
14.25
14.28
Converged
Richardson
Iteration
Example 1
f ( x)
k
x x
*
0.7 x 10
Richardson
Iteration
Example 2
f ( x)
Start with
2 x 10
x0
0
x1
x0 f ( x0 ) 10
x2
x1 f ( x1 )
x3
x2 f ( x2 ) 130
x4
x3 f ( x3 )
40
400
No convergence !
Richardson
Iteration
Convergence
Setup
Iteration Equation
Exact Solution
x
k 1
x
*
k
k
x f (x )
*
*
x N
f (x )
0
Computing Differences
x
k 1
x
*
k
*
k
*
x x f (x ) f (x )
Need to Estimate
Richardson
Iteration
f (v ) f y Convergence
Mean Value Theorem
wf v v y
wx
v  > v, y @
f
v
v
y
x
Richardson
Iteration
Convergence
Use MVT
Iteration Equation
Exact Solution
x
k 1
x
*
k
k
x f (x )
*
*
x N
f (x )
0
Computing Differences
x
k 1
x
*
k
*
k
*
x x f (x ) f (x )
§ wf x · k
*
¨1 ¸ x x
wx ¹
©
Richardson
Iteration
If
1
And
Then
Or
Convergence
Richardson Theorem
wf x wx
*
d J 1 for all x s.t. x x G
0
*
x x G
x
k 1
*
k
x dJ x x
lim k of x
k 1
x
*
*
k
0
lim k of J x x
*
Linear Convergence
0
Richardson
Iteration
Example 1
f ( x)
k
x x
*
0.7 x 10
Richardson
Iteration
Problems
• Convergence is only linear
• x, f(x) not in the same units:
– x is a voltage, f(x) a current in circuits
– x is a displacement, f(x) a force in struts
– Adding 2 different physical quantities
• But a Simple Algorithm
– Just calculate f(x) and update
Newton’s method
Another approach
From the Taylor series about solution
0
df k *
f ( x ) f ( x ) ( x ) ( x xk )
dx
*
k
Define iteration
Do k = 0 to ….
1
ª df k º
k 1
k
x
x « ( x )» f ( xk )
¬ dx
¼
ª df k º
if « ( x ) »
¬ dx
¼
until convergence
1
exists
Newton’s Method
Graphically
Newton’s Method
Example
Newton’s Method
k
x x
*
Example
Newton’s Method
0
f ( x* )
Convergence
2
df
d
f
k
k
k
*
f ( x ) ( x )( x x ) 2 ( x )( x* x k ) 2
dx
dx
k
*
some x  [ x , x ]
Mean Value theorem
truncates Taylor series
But
0
df k k 1 k
f ( x ) ( x )( x x )
dx
k
by Newton
definition
Convergence
Newton’s Method
Contd.
2
df k k 1 *
( x )( x x )
Subtracting
dx
Dividing through
d f
k
* 2
x
x
x
(
)(
)
2
d x
2
df
d
f
k 1
k 1
k
*
* 2
( x x ) [ ( x )]
(
)(
)
x
x
x
2
dx
d x
1
2
df
d
f
ª
º
Suppose « ( x) »
( x) d L for all x
2
¬ dx ¼ d x
then x
k 1
*
k
x dL x x
* 2
Convergence is quadratic if L is bounded
Convergence
Newton’s Method
Example 1
f ( x) x 2 1 0,
df k
( x ) 2 xk
dx
k
2x (x
k 1
k
k 1
2x (x
or ( x
k 1
find x ( x* 1)
k
x
*
k
x )
k
*
2
1
k
x ) 2x (x x )
*
x )
1
k
* 2
(x x )
k
2x
x
k
x 2
*
2
Convergence is quadratic
Convergence
Newton’s Method
Example 2
2
*
f ( x) x 0, x 0
§ df ·
df k
Note : ¨
¸ not bounded
k
© dx ¹
(x ) 2 x
dx
away from zero
2
k
k 1
k
Ÿ 2 x ( x 0) ( x 0)
1 k
*
k 1
k
for x z x 0
x 0
x 0
2
1
*
*
( xk x )
or ( xk 1 x )
2
1
Convergence is linear
Newton’s Method
Convergence
Examples 1 , 2
Newton’s Method
Convergence
1
2
ª df
º d f
Suppose « ( x) »
( x) d L for all x
2
¬ dx ¼ d x
if L x0 x* d J 1
then xk converges to x*
Proof
x1 x * d L ( x0 x * ) x0 x *
Ÿ x1 x * d J x0 x *
Ÿ x2 x * d LJ x0 x * x1 x *
or x2 x * d J 2 x1 x * d J 3 x0 x *
Ÿ x3 x * d J 4 x 2 x * d J 7 x 0 x *
Newton’s Method
Convergence
Theorem
df
d2 f
If L is bounded (
bounded away from zero ;
bounded)
2
dx
dx
then Newton's method is guaranteed to converge given a "close
enough" guess
Always converges ?
Newton’s Method
Convergence
Example
Convergence Depends on a Good Initial Guess
f(x)
x1
x1
x
SMA-HPC ©2003 MIT
2
x
0
x
0
X
Convergence
Newton’s Method
Convergence Checks
Need a "delta-x" check to avoid false convergence
f(x)
x
k 1
x
x
k 1
k
x ! H xa H xr x
k
x
f x
SMA-HPC ©2003 MIT
k 1
H fa
k 1
*
X
Convergence
Newton’s Method
Convergence Checks
Also need an "f x " check to avoid false convergence
f(x)
f x
x*
x
SMA-HPC ©2003 MIT
k
x H xa H xr x
! H fa
X
x k 1 x k
k 1
k 1
k 1
Summary
• Nonlinear Problems
– Struts and Circuit Example
• Richardson and Linear Convergence
– Simple Linear Example
• 1-D Newton’s Method
–
–
–
–
–
Derivation of Newton
Quadratic Convergence
Examples
Global Convergence
Convergence Checks