I ntnat. J. Mh. Math. Si.
Vol. 4 No. 2 (1981) 383-391
383
CONVERGENCE ANALYSIS OF A PERTURBED ITERATIVE
SCHEME (PIS) FOR SOLUTION OF NONLINEAR SYSTEMS
S.K. DEY
Visiting Professor
von Karman Institute for Flud Dynamics
B-1640 Rhode-ST-Genese
Belgium
Present Address
Department of Mathematics
Eastern I111nols University
Charleston, Illlnols 61920
U.S.A.
(Received October 15, 1979 and in revised form February 13, 1980)
ABSTRACT.
In the references [i, 2, 3] a perturbed iterative scheme PIS) has
been studied both theoretlcally and computatonally to solve nonlinear equations.
In this article a more general analysis of its convergence properties has been
done
i.
INTRODUCTION.
A perturbed iterative scheme (PIS) has been developed in [1, 3] to solve
nonlinear equations.
It is a functional iterative scheme obtained by adding a
predetermined perturbation parameter to nonlinear Gauss-Seidel iterations.
has a simple algorithm.
PIS
Other similar iterations [6, 7, 8] had more restricted
applications and complicated algorithms.
Theorems on convergence properties of
S.K. DEY
384
PIS, derived in [I] for one nonlinear system, were modified in [3] for coupled
nonlinear systems.
In this article more generalized convergence properties of
PIS have been analyzed by applying decaying matrices [5].
tion of this concept is also given.
A practical demonstra-
Theorems on convergence of PIS derived in
[1, 2, 3] may be interpreted as particular cases of those derived in this paper.
2.
ALGORITHM OF PIS [I].
Let us consider a nonlinear system:
Fi(Xl, x2,...xn}
0, i
This equation may be expressed as F6xl
F
D C Rn
/
(2.1)
i, 2,...n
0 where x
Rn (Rn is the real n-dimensional space).
x*
admits a root x
E
D.
x
)T
We assume that (2.1)
Our objective is to develop a perturbed iteratlve
scheme LPIS) to solve (2.1) and cmpute
xi
6x I x 2
Gi(x 1, x2,...xn),
i
x*.
If (2.1) is written as:
(2.2)
1, 2,...n
nonlinear Gauss-Seidel iteration at sme kth step is:
(2.3)
x
where
x at the kth not exponent)
i 1 x’-I xk-l)"
n
value of
G Gi’ 2
Let G
(2.4)
1
D x D C Rn x Rn
and,
iteration and
/
xk
G6xk, xk-l)
x*
G
*
DEFINITION I:
D.
Then 2.3) may be expressed as
(2.5)
x*
In such a case,
(2.6)
x*
is called a fixed image of G on D x D.
Now (2.3) may be perturbed as follows:
To compute the perturbation parameter
small, such that terms of the order
and (iii)
(2Gi
/
x) k is bounded.
,
we assume that V i,k:
(i)2 may be neglected,
i’s are
(i)
(ll} (G
i
/
xi)k
i,
CONVERGENCE ANALYSIS FOR NONLINEAR SYSTEMS
If (2.7) converges after (k-l) iterations,
-i
x.
i
385
*
x..
i
Then,
Taylor’s series and using the above assumptions we get
(2.8)
i Gi(xkl xi_k’1 i’ xi+k-iI -i) and
iG {Gi / xi)l i-l’ G, i n-l.
where
w.’s
1
Thus in (2.7),
are computed in terms of quantities known apriorl.
The
convergence criterion is:
(2.9)
where E is positive and arbitrarily small.
3.
ANALYSIS OF CONVERGENCE.
By convergence we mean lim xk
k
wk + G(xk, xk-l)
xk
where wk
n )T
{i
THEOREM 1:
lim
k
n.
R
Now {2.7} may be expressed as:
Let G be continuous on D x D.
A necessary condition for convergence is that, for some norm
I w ll
PROOF:
E
x*
o
Assuming that (2.7) converges to
Gk, Xk-l)
G(X* x*)
x*
x*, limxk=x*
k
and
,
Then, (3.1) gives lira wk
k
which implies
(3.2).
To prove that (3.2) may also be a sufficient condition for convergence, the
following concept is used:
DEFINITION 2:
A sequence of commuting square matrices
AI, A2,...An
is
called a sequence of decaying matrices or simply D-matrices if
(3.3)
S.K. DEY
386
Each
Obviously, the eements of
is called a D-matrix.
are variable and
Several properties of D-matrlces are given in [5].
they change as k changes.
The one we need is:
A sufficient condition such that Ak is a D-matrix is that for
LEMMA 1:
k > K,
some par.t.i.cu.l.ar norm and
llAkl
0 iff A--
where
,
u, S); b i,j
y
and hence
is now rather trivial.
/
xk, yk),
D
[u, 8)
l(I
b ,j
an element of
a,j, b,j
DEFINITION 2:
Also,
A
are square matrices (n x n) and, a i,j
i,j
xk,
@. Proof
D x D C Rn x Rn
Let H
kll < I AIII II A211 ---II II
IIA1 A2
For any norm,
PROOF:
If (xk,
change.
),
Let
xl
(a, B}
D x D.
Let us express
an element of
y
u, 8).
Xnl )T"
(IXll Ix21
D x D, Ak,
As k changes,
are continuous,
Ak)-ll
(3.6}
is bounded, and
is continuous and form a sequence of D-matrlces, the mapping H
D x D c Rn x Rn
D is called a D-mapping on D x D.
/
THEOREM 2:
converge to
radius of
,
+
Then,
x*,
)
If in (3.1)G is a D-mapping and (3.2) is true, PIS [3.1)will
a fixed image of G on D x D.
< 1 Vk > K,
PROOF:
Subtracting
kk
x*
Ixk
+
x*
Furthermore, if P
(the spectral
is the unique fixed image of G on D x D.
(2.6|
from
(3.1|
and using (3.5) we get xk -x
xk- x*.
x*[ <_ Cklwkl
+
EklXk-I x*
where Ck and Ek are defined in (3.6) and (3.7} respectlvely.
recurrently,
k
(3.8)
Now, using (3.8}
CONVERGENCE ANALYSIS FOR NONLINEAR SYSTEMS
_
k
j-1
ko
Ek0+l)j=l (Ek0 Ek0-1
(Ek Ek-i
k
+
Z
jffik0+l
_.... 9+.} c Il
c
lies tt for se
(3.2)
i E E
k 1 2
k
I
d
k
from (3.9) we get lira
k
Ixk
+
.
.
@
y*
Ix*
Ek
(I
/
y*
is
Y*I + B, Cx* y*
I( ’.- ,.1 I* y*t "., I* ,*1-s.o
E, as k /-, pCE,) < i.
llX* Y*I
This gives
@, implies x
It may be observed
that
and
lar matrices with variable elements.
Z
CY- E,}. -I
Cj
berg ded j,
a second root.
ThUS
p(>
E3,
"
>
<
,
Then,
,
Hence
y
are respectively lower and upper trlangu-
+
If
is a tridiagonal matrix it is
However, in general (I
rather easy to check when G is a D-mapping [9].
is not quite simple to find.
is a
A, Cx*
G(y*, y*
G(x* x*)
y*l
E,
Hence
(.9}
which establishes convergence.
In order to prove uniqueness, we assume that x
x*
’I.
I
Also, since
0+I 0+2
x*
lwJl
EJ+I) Cj
<
k0+l
387
)-i
In such cases, theorem 2 cannot be used and the
following approach may be taken.
Let Vk,
THEOREM 3
where (I
Ak)-i
II II + I II
is bounded.
<_ u
converge to x * and
I AII
<_ (
<
X*
Let Mk
PROOF:
<
G(x*, x*)
xk-l)
G(xk
a,,( ((
i
x*) +
Akk
k-I
x*}
Let for some particular norm and Vk
K,
Furthermore if (3.2) is true, PIS given by (3.1} will
is the unique fixed image of G on D x D.
(I
(3.lO)
-
Ak)-i Bk. Since
II ",11( II A,II
I Akll _+. II Bkll
<
(-
Ao,
<_ u
< I,
388
S.K. DEY
< a < 1.
x*
*
<
-
-i
-11 II ,.11 + II .’11 II ----’ll
II
-" II <_
II
recurrently we get:
II
=*II s *
.k-ko
II ,,11 +. I!
.
(zko-J
GCy*, y*)
e’, X*)
-
Gk, -i)
Solve:
0.25x2
,,
A,*
II
(3.11)
or
or
It may be easy to prove:
tions
4.
a
I!-,11
wll converge to x- x
,
is that
-
[(I
In (3.9) we set wk
2
AN APPLICATION.
5.00942 x
where G
x I + 0.75x2
0"3.
0,
0.5XlX2
x2 + 0.005
lxI, x2)
o..[-
=_.
0.25x + 0.75x2;
h,,
GC,
Ak}-i Bkl
o.
is a
G
2xI, x2)
-
*
*
.
GO=’,
-
0.5XlX2
o.
=’
In
0.
G 1 (x I, x 2)
We express the system as x I
/
+
II =* :11
0 and the proof follows.
[-0.5, 0.5] x [-0.5, 0.5], ths system has a root given by xI
x
y
x
A sufficient condition such that nonlinear Gauss-Sedel itera-
4:
D-matrix where Ak and Bk are defined in (3.5}.
PROOF:
,
rt.
y*}
y*) + B,*
x" II
.i-t + -kll x
a
y.
THIX)REM
,-ll,
(I
k
"
y*
x*
. II
(
II II +
prove iess, let x
X
Subtracting (2.6) fr (3.1) and using {3.10)
is a D-matrlx.
Hence
3.7606 x 10-3 and
x2
+ 0.005.
G2 (xI, x 2)
Denoting the
-
=’ + kC
=’
389
CONVERGENCE ANALYSIS FOR NONLINEAR SYSTEMS
_
0.75
x*I
-- -0).
u,
is
(v)
0.25()
+
(
,
e
x)
e
2
Also, since Ck
following to be
0o25(x-i) 2 + 0.75-i;
iteration: _)
+
II II.
Ak}-iI
l(I
is
Hence, iterations will converge iff (3.2)
algori, now, rires
(i)
-o.s, o.s,
contuous and a ix.
ded, G is a pping.
e
,
(, @ ,
o,o.y,
0.75
-I
(iv)
(ii) [eI /
(- )
0.5-I + 0.005
(vi)
0. + o.oo,
(
(-
Xl], -I "0.5
/ {I- [G1 /
Xl]G
X2]x
/’I,
(vii) [e 2 /
/(-
If
10
-5
-1,2
D x D, PIS converges in 5 iterations.
oonvergenoe prinoiple as develope in
cuted seentially at y
eor
s sidle
2.
ple
It shs at in
’
lalns
oe
to
prove that Ek is a D-matri some prior knowledge of the roots is required.
5.
DISCUSSIONS.
PIS has some limitations [I].
apriori knowledge of
x*
In order to prove that G is a D-mapping some
is required (as is evident in section 4).
quite effective to solve equations with multiple roots.
x
sin(x) cosCy) + sin(z), y
has three roots:
With (x0,
y0, z 0)
sln(y) cos(z) + sln(x),
PIS is not
For example the system:
z
sin(z) cos(x) +sin(y)
(0, 0, 0), (1.249, 1.249, 1.249) and (-1.249,-1.249, -1.249).
(i, i, I)
(9999, 9999, 9999), (0. 1, -0.1, 0.005), (-9.0,
-9o0, 0.00003), (76, 900, 8000), (-3, -3, 3) it converged to the second root, with
390
S.K. DEY
(x0 y0
, 0}
z
(5555, 3333, 1111}, (56, 92, -9}, C-l, -I, 13), (49, 78, i00) it
converged to the third root but the first root was not found.
this is still under investigation.
If, in some cases,
to zero, PIS becomes less effective [I].
s are identically equal
PIS may be interpreted as a cmbinatlon
of Gauss-Seldel and Lieberstein’s methods [I].
similar methods in [1, 3].
The reason for
It was compared with other
Although Newton’s method has a quadratic rate of
convergence it requires in general initlal estimates to be close to the root and
evaluations of Jacoblans at each iteration.
PIS requires none.
Thus for large
systems whereas Newton’s method is not practical to use, PIS can be easily used
[1, 3].
In [1] it has been proved that PIS has a quadratic rate of convergence.
Also, since PIS-algorlthm is relatlvely simple, computer programming is rather
easy.
Since no matrices are stored, requirement for computer memory storage is
also small.
A
This ork was supported by the Minna-James-Heinemann-
Stiftung Foundation of
annover,
Germany and Eastern Illinols University.
Thanks
are
ue
I.
Dey, S. K.
2.
Dey, S. K. Numerical Solution of Nonlinear Implicit Finite Difference Equations of Flow Problems by Perturbed Functional Iterations. Lecture series
1978-9. Computational. Fluid DYn___aics, Vol. II, Von Kazman Instltte fo
Fluld’ Dynamlcs, Rhode-St-Genese, Belglum, 1978}.
3.
Dey, S. K. Perturbed Iteratlve Solution of Coupled Nonlinear Systems with
Applications to Fluid Dynamics, Journal of Compu.t..t.ona.! .an.d Applled
Ma.thgmatics, Vol. 4, No. 2, (1978) 117-128.
4.
Ortega, J. M. and Rhelnb01dt, W. C. Iterative Solution of Nonlinear Equations
in Several va.r.lables, Academic Press, (1970}, Ne York.
5.
Dey, S. K. Numerical Instabilities of Nonlinear Partial Differentlal Equations, _CFT 7800/SKD/IE, Von Karman Institute for Fluld Dynam/cs, RhodeSt-Genese, Belgium, C1978}.
to the reviewers for their suggestions.
Perturbed Iteratlve Solution of Nonlinear Equations with Applicato ’.ui Dyna.cs,
Cuta.t.on.a!..an pp.,
Vol. 3, NO. i, (1977} 15-30.
ons
ourna. o,.
CONVERGENCE ANALYSIS FOR NONLINEAR SYSTEMS
6.
Delst, F. and Sefor, L.
Parameter Variation,
Solution of Systems of Nonllnear .quatlons by
jo_urnal, Vol. 10, (1976) 78-82.
te
7.
Freudenstein, F. and Roth, B. Numerical Solution of Systems of Nonlinear
Eqqatlons, Journal of ACM, Vol. I0, (1963) 550-556.
8.
Ortega, J. M. and Rockoff, M. Nonlinear Difference Equations and GaussSeldel Type Iterative Methods, SIAM Journal...o.f .Nume.rical Analys.is,
Vol. 3, (1966) 497-513.
9.
Dey, S. K. Applications of General D-mappings to Certain Flow Problems,
Eastern II.lln.ois University, (to be published).
391