Internat. J. Math. & Math. Sci.
VOL. 14 NO. 2 (1991) 349-362
349
ON A GENERAL CONVERGENCE FOR BROYDEN LIKE UPDATE METHOD
RABINORANATH SEN, RINI CHATTOPAOHYAY and TRIPTI SAHA
Department
o[
Applied bathemat[cs
Calcutta Unlvers[ty
q2, A.P.C. Road
Calcutta- 700009
Ind a
(Received December 7, 1987 and in revised form November 20, 1990)
ABSTRACT.
The role of Broyden’s method as a powerful quasi-Newton method for solving
unconstrained optimization problems or a system of nonlinear algebraic equations is
known.
well
We
offer here a general convergence criterion for a method akin
n.
Broyden’s method in R
to
The approach is different from those o[ other convergence
proofs which are available only for the direct prediction methods.
KEY WORDS AND PHRASES.
n,
in R
Unconstrained optimization, Broyden’s method, partial ordering
M-matrix.
1980 AMS SUBJECT CLASSIFICATION CODE. 65K.
INTRODUCTION.
1.
,
x
Let
:D=Rn-----
R
be an F-dlfferentiable functional havlng an optimum
(D).
int
Then the vector x
at which the optimum of
Vb*(x)
P(x)
8
In the above, V
is realized, satisfies the equation
0
3---’
v---n
T
Our concern in this paper is iteratlve
methods for the solution of simultaneous non-llnear equations
F(x)
m
0; F: R
Rm
(I.I)
in the case when the complete computation of F’ is infeasible.
In the case, F
P, solving (I.I) means in effect finding the minimizer
of.
The algorithm under consideration takes the form
Xn+l
where H
n
Xn
nHnF(x n)
(1
is generated by the method in such a way that the quasi-Newton equation
H
n+l
(F(x
n+1
is satisfied at each step.
F(Xn))
Xn+l
x
n
(1
350
R. SEN, R. CHATTOPADHYAY AND T. SAHA
The step length
(n
method (Davidson [I], Fletcher and
[s
considering what
algorithm by which H
[2]) for unconstrained opt[mlzat[ons and by
linear, Broyden [3] in 1965 suggested an
Powel|
when
desirable
is
is obtained from H
n+
By analogy with tlle DFP-
is chosen to promote convergence.
n
by means of a rank-one update.
In the case of minimizing problems, Dixon [4] called a method perfect if
is
through line searches and a direct
obtained
prediction
method
if
Iteratlve procedures (1.2) and (1.3) may be described as follows:
mxm
and also x R
Choose non-singular H R
o
o
let
For n--0,I,2
Sn
Xn+l
"-YnHn F(xn
=x
n
0 then
and choose
Hn+
Vn
m
R
Hn +
Sn nYn
(1.7)
and
VnYn
HnYn)Vn.T
(s n
(1.6)
F(Xn)
T
such that
s
vTH
n n n
O.
(I.8)
method (sometimes called his first or good method)
HTsn/SnYn--,
n
Vn
(1.5)
Hn
}In+l
Broyden’s [3]
chooslng
T
H
as
(1.4)
+s n
Yn F(Xn+l)
Let
The
chosen such that
Yn being
Yn
I.
m.
....
If
Yn
Yn
Yn
in
(I.8)
and
is
defined
for
results from
0 only
Yn
so
long
Yn
0
O.
Broyden’s second or bad method results from choosing v n
T-I
and Is defined so long as
Sn # O.
T
Yn /YnYn
where
YnHn
The convergence results that are available to date are proved for the direct
prediction method [5].
Broyden has shown that his (flrst) method converges locally at
least llnearly on nonlinear problems and at least R-Superllnearly on linear problem
[61.
Later Broyden et al [7] showed that both Broyden’s good and bad methods converge
locally at least Q-superlInearly.
More and Trangsteln [8] subsequently proved that ’locally’ could be replaced by
"globallf’ when a modified form of Broyden’s method is applied to linear systems of
On the other hand, Gay showed in 1979 [5] that Broyden’s good and bad
equations.
methods enjoy a finite termination property when applied to linear systems with a nonHe has also proved that Broyden’s good method enjoys local 2m-step
singular matrix.
Q-quadratlc convergence on non-llnear systems.
[7],
Recently, Dennis and Walker [9] have made generalizations of thelr results
update
[I0] and have put forward convergence theorems for least-change secant
Decker et al, have considered Broyden’s method for a class of problems
methods.
having singular Jacoblan at the root [II].
Our concern is to consider the method (1.2) which is not necessarily a direct
Our method is not perfect because in the case of minimization
prediction method.
reduce
problems exact llne searches have not been performed. However, the scalars do
GENERAL CONVERGENCE FOR BROYDEN LIKE UPDATE METHOD
[I0].
351
Our method can thus be viewed as an Intermediate between d[rect prediction and
perfect methods.
We have called our method Broyden-like because although we have used Broyden’s
good method we have taken B to be an M-matrix [[2] unlike Broyden [3].
We have
o
used
componentwise
partial
m
in R to
ordering
pove monotone convergence of the
We have asserted that under certain conditions,||
can be taken as nonn
[!
matrices.
In our analysis, F is taken as an isotone operator [12].
sequence
negative
Operators which are monotonically decomposible (MDO) [13] can also be brought within
scope of our convergence theorem.
A local linear convergence is achieved.
Experiments with numerical problems are also encouraging.
Even where Newton’s method
the
has diverged, our method converges in a small number of Iterations.
Section 2 contains mathematical preliminaries.
results.
4 gives the algorithm.
,Section
Section 3 contains convergence
Numerical examples are presented in section
5 while we incorporate some ’discussion’ in section 6.
2.
MATHEMATICAL PREL IMINARIES.
The componentwise partial ordering in Rm is defined as follows:
For x,y R,m x
(x l,x 2,...x
m
DEFINITION 2.[.
)r
y
(yl,Y2
xi
Yi’ i
y <==>
x
DEFINITION 2.2.
ym )T,
x )y if and only if
1,2,...m,
y and x
x
#
y.
We define for any x,yR
m,
such that x
y, the order
interval [II]
<x,y>
DEFINITION 2.].
if
Rm/x
{u
u < y}.
A mapping F:D
Rm
m
R
is Isotone (antitone) [12] on D
o
(Fx > Fy) wheneve x
y, x,yD
o
DEFINITION 2.4. We denote by L(Rm) the space of mxm matrices.
Fx
We
introduce
a
partial
ordering
componentwise partial orderlng in R
DEFINITION 2.5.
m)
in L(R
which
m.
A real mxm matrix
(aij)
be an M-matrix [12] if A is non-singular and
with
A-I>
aij
i,s
O, for all i
B s
n n-I
Bn+
and off-diagonal
one i.
is defined to
(2 I)
2
n
--Yn-
Bn+
is given by
(BnS
(2.2)
T
n
s s
n n
In what follows we denote
LEMMA 2.1.
the
O.
According to Broyden’s good method, the update formula for
T
Yn)S u
n
B
m.
J
with
B
so that B is a replacement for the Jacobian
n
n
n
satisfies the quasi-Newton equation
J(x ), and thus B
,2
compatible
-I
In what follows, H
j
D
Fy
Sn
(s
n)) Yn
(y
n) Bn
(b
(n)
ij
i
2 ...m
T
s s
n n
(I-----) is a square trix whose diagonal elements
elementns are non-posltlve provided s i(n) 0 and s(n)> 0
are positive
for at least
352
R. SEN, R. CHATTOPADHYAY AND T. SAHA
The proof is trivial.
LEMMA 2.2.
(i)
B
(ii)
B
Let the following conditions be fulfilled:
(b
n
(lit) {x
(n)
nl
<
’."
ij
tj
for all i
0
(n)
J, b
i
>
0
a strictly diagonally dominant matrix
iq
is a moaotonic increasing sequence in the order sense.
n
(n)
(s(n)
(n)
(n-l)
(iv)
(n-l))
(n)
(sf (n))2
(n)
-Yk(n-)) ls.
Bn+
Then
is an M-matrix.
It ollows from conditions (i) and (ii) that B
PROOF.
Siace B
n
[Bn(S n
th
(n)
>--(-nS.
Xn
the
s
ia a monotonic increasing sequence
*
solution
(n)/E
magnitude, s.
If
n-I )]
T
n
((n)
Yk -Yk(n-l))] s(n)
-Sk(n-l))
i
since
y
is given by
n+
(n)
of
(yn-
n-I
element of B
[bik (Sk
k
s
T
and the (i,j)
(n)
ts an M-matrlx [12].
satisfies the Quasi-Newton equation (2.1)
(Bn n Yn )sT n
b
n
(n)
bij
(s
0,
s
(n)
(n)
will
small
be
and s
xn
Xn+
Sn
(n)
(n-l)
s
O.
will
j, the signs of
bij
of
bij
(n+t)
strictly
Since B
is
a
matrix the strict diagonal dominance property of
Bn+
elements
diagonal
elements
diagonal
smaller
strictly
negative.
positive,
strictly
(n+1)
bij
and
diagonally
the
off-
dominant
will be maintained by virtue of
condition (v).
Let us denote
kZ [bik(n)(sk(n)
by
ci
(n-l)
(n)
l)
s,(.n-
(n)
(Yk -Yk
:(s (n)
(n)
s.___
2
Condition (v) implies that
(n) +
i j
and since
]ij(n)
(n+l)
in
will also have at most a finite magnitude.
(n) will determine the signs of
be
the
be
still
9
)-
will
Therefore
In the neighborhood
(bij
(n)
cij
is large compared to
b(n+l)
(n)
< bii
Icij(n)
+
c(n)
ii
,(2.4) implies that
(2.4)
GENERAL CONVERGENCE FOR BROYDEN LIKE UPDATE METHOD
REMARK 2. l.
If however B
ceases to be an M-matrlx,
n+
353
we can witllot violating
the constraint (2.l), pre-multiply the first term in the e:pression for B
n+l
by a
T
strictly positive diagonl atrix P
n
becomes the predominant term
B+I Pn
Bn
Thus, if
b(n)[i
>
O,
j, and
i
g
o
the expression
’n p_) + YnT
(I
n
of the sae order of B
n
S
S
S
B
such that P g (I- -g)
n n
n
as given below.
n
(2,5)
S
is already a stmictly diagonally dominant matr[ with
Bn+l will also be a strictly diagonally dominant
b (n+l)--. > O.
erefore, if B is an M-atrix we
Bn+l( Bn+l)
-l
n
H
n
and B
-I
T
H
s s
n n
[PnBn (I
n+
B-n Ip-1 Y
+
S S
S
n
T
[I
s s
n n
-----+
S S
T
b
and
+I)
<
O,
can always construct
BnSn Yn"
T
nan)
-I
S
n n
B-1n p-In YnSnT
n n
O, i
we obtain from (2.5)
Hn+[’
n+l
<
matrix with
as an M-matrlx without affecting the condition
Writing B
b
T
-I
B-Ip-I
n
n
S S
n n
which is approximately equal to
s
T
Hn P-n
Hn nP-lyn) --]
s s
[I + (s n
n n
Writing s
which
Bn+lYn Hn+lYn
n
taken as
Hn nP-I
Hn+
Therefore,
H
+
(an
is
approximately
equal
to H P
n n
Yn’ Hn+l
sTHn np-l
Hn yn)P-I
n
p-I
sTH
nun Yn
can
be
(2.6)
as expressed by (2.6) satisfies the Quasi-Newton equation
Hn+
s
n
n+ lYn
LEMMA 2.3.
Let the diagonal
matrix B
satisfy
constant.
Then
.(n)
PROOF.
condition
the
elements,
(n)
l’Dii
bij
.(n) are the eigenvalues of B
n
)
2-’-’ Ai
By the Greschgorin theorem,
(n)
(n)
.(n)
Ai
(n) of the strictly diagonally dominant
M, for all I where M(>O) is a finite
of
b
ii
By strict diagonal dominance of B
354
R. SEN, R. CHATTOPADHYAY AND T. SAHA
erefore,
>
The
lower
the
that
ensures
lemma
above
0"
2_
boun,i
of
n and hPnce H remains nonig,lar at each
n
stability of the iteration proces tq therefore guarauteed,
fo[loa
are
opera,or
and the
denotes an arbitrary norm in
-1
iteration,
strictly positive for al.l
In hak
B
of H
e tenval,,es
tle
induced by the partc,lar vector nor.
3. I.
CONVERGENCE.
Let tle following conditions be fulfilled:
THEOREM 3.1.
ri)
F:D R
([i)
D
m
R
m
ts Frechet differentiable on a convex set D
the
includes
Xo, yo
and
element
null
O
D
O
such
o
D.
if x < y then
that
the
order interval <x,y>D.
O
O, for
(iii) F(x)
all x,y D
(iv) x
D
all x
O
and
F
O
D0
is an inl.ttal approximation to the solution and x
C)
()
The operator G
y H F is such that GD
1
0
(vi) The
with
(o)
bij <
(vii)
operator B
(b
o
n
b
[s
[j
(o)
bii >
s
j
b(n)
b(n) +_k
ij <
0
(Yk
2
(n))
i
(n)
b(n)
ik (Sk
((n)
Yk
Sk(n-l))
.-(n)
11
(s
y
(x)
(xi)
(K
The scalars
>
0), n
yn(>
a
kn- 1)
(n)
s.
_a__
2
0,I,2,...
n
[bii(n)]
dominant
(n)
(n-l)
k
.(s
(viii)
diagonally
ij
(n)
[b ik (s k
0,1,2,
>
O) are to be chosen such that
(a)
F(x)n nnYHF(x
Yn-IH-I
(b)
sup [I
(0
4
yn Hn F’(x)]
(
O
[I
Xn-I
n
)
Hn_
O
<
O.
0
strictly
a
0 and F(x
O
D
O
0
()
O, for all i,j,ij, and
(n)
For B
,k
(ix)
F(y) for
y, F(x)
if x
i.e.
isotone
is
F’ (Xn_
F’(x) is Frechet differenttable for all x..D
0
],
x
<Xn_l, Xn >
matrix
GENERAL CONVERGENCE FOR BROYDEN LIKE UPDATE METHOD
n
oHoF’(Xo)] [l
(xti) lira [[
Then starting from x
O, an
o
0 in the order sense.
Xo
approximation to the solution of F(x)
Inlt[al
ix n of Broyden-llke approximations
is the sequeace
x
x
n+
YnHnF( n ),
n
If
O,l,...
n
to
possible
is
it
particular,
in
n
0
defined by
O.
which converges to a solution of the equation F(x)
i e., A
355
a
find
positive
matrix
convergent
A,
0 as n+=, then the error at the nth stage is given by
II n-)- -o)11
Conditions (iii), (iv) and (xa) yield
PROOF
Xo
I
By (v)
B
0
x
l& Do :D.
YoHoF(xo
GXl Do
Hence
B-l>
Hence H0
[12] by (vi).
is an M-matrix
(3.3)
0
Xo
0
0
0
s
o
x
x
Because of conditions (vl), (vii) and (viii) and because s o
O.
that B Is an M-trix. Therefore, tt
)
O.
0
O, it follows from (22)
B
Isotonicity of F, together with condition (xa) yield,
Hence x2
0.
x
YIHI F(xo
YIHIF(X I)
0
x
Thus x
x
Let us assume by way of induction that 0
(k)
j.
n, with blj < O, i
[,2
matrix, k
n
n
XkD o,
k
1,2,...n, and Bk is an M-
O,
is an M-matrix, H
Since B
YnHnF(Xo
YnHnF(Xn
By conditions (ill) and (xa)
(3.4)
YoHoF(Xo )"
oHoF(Xo Gx I Do.
YoHoF (x o)
Therefore, x
n
.HF(o)
-% H F(o)
’n
Xn+
Gx ED
n
o
(n)
blj <
O, i],
conditions (vii) and (viii), we can conclude from Lemma (2.2) that
(n+l)
with
< O, lCj.
oi.
fact
the
Using
that
Bn
is
an
M-matrlx
with
Xn Xn+ l,
Bn+
and
the
is an M-matrlx
J
Therefore the induction is completed.
is a monotonic increasing sequence in the order sense and
Thus, {x
k}
all k.
Now, 0
x
2
-x
x
’
o- YIHI
-F(Xo )]
[YIHIF(Xo YoHoF(Xo
F(Xo) ]"
Xl Xo- YIHI [F(Xl
-x
[F(x l)
Xk D
O
for
356
D
o
R. SEN, R. CHATTOPADHYAY AND T. SAHA
be[ng a convex set and us[n the mean value theorem in R
x
x
2
f
<
+ t(x
YIHIF’(x o
[I
o
o
o
)dr
we obtain from (3.5)
0
<
t
<
(3.6)
stand for differentiation in the Frechet sense.
Here
<
+ t(
t
x )), 0
o
o
x )) is semi-continuous [12].
o
Egain since F’(x
F’(x
))] (
m
+ t(x
o
+
F’(Xo
Gl(t)
Therefore, the operator
<
is G differenttable,
t(xl -Xo))
is continuous [n
[O,l].
We
thus have the following from (3.6)
0
x
2
1HlF’(Xo
[[
t(Xl
Xo))](l
t(xl- Xo
(Xl- Xo
YIHIF’(Xo
[I
suPt
x
+
+
<
t, 0
assuming the supremum is attained at
t
<
Xo
(3.7)
I.
By (xb), (3.7) further simplifies to
0
x
YoHo F’(xo
x[ < [I
2
(3.8)
x2)"
(l
Argaing analogously as before,
[(I
7nHnF’(Xn_l
+
sup[l
YnHnF’(Xn_l
+ t(x
f
n
Xn+l
o
+
-(nFn F’(xn_
[I
Xn_l))] (n
t(Xn
Xn_l)dt
n- Xn_l))] (n n-I
t(Xn n-I
assuming that the supremum is attained at t
.
)]
(Xn
Xn-I
(Xn
Xn-I
Condition (xb) further reduces (3.9) to
(n-IHn-IF’(Xn-I
YoHo F’(x o)1 (x n
[I
Xn
Xn+
[I
(
YoHoF’(x o )]n
[I
n+pHence
Xn+ p
k
k--n
YoHo F’(xo)]n (x- Xo
Since [I
(x
-(oHo F’(x o)] (Xl
[I
7.
Xn
0 as
Xn_ I)
n* [t
x
(3.10)
o)
(3.11)
Xo
follows from (xii) that
{Xn
is a
uchy sequence and the space is complete,
lira
x
Using convergence of { n
-)--"n
B
n
n
(x
Therefore using (ix)
(3.12)
R-
it follows from
YnttnF(xn
l im
Nou, F(x n)
x
n+l
(3.12) that
(3.13)
O.
Xn).
(x) and the strict diagonal dominance of Bn
2k
0 as n+.
--llXn+l Xnl
llF(Xn)l
Continuity of F yields that
,
Hence x
F(x
O.
is a solution of the equation F(x)
Further if
-oHo F ’(x o
4
O.
A, (3.11) reduces to
(3.14)
GENERAL CONVERGENCE FOR BROYDEN LIKE UPDATE METHOD
x
x
n+ p
n
n+p-1
E
k =n
Ak(xl
x
357
(3.15)
o
II’II"
The error (3.2) follows from (3.15) by making p+ and then taking
In the next theorem we provide a bound for the inverse Jacoblan approximation.
THEOREM 3.2.
Let [n addition to the conditions of theorem (3.1), the following
conditions be satisfied:
(a)
(b)
(c)
Then
-I
Ilt,(o]-ll,
(tllnO +oUoF’ (o>>n(t -xo>ll’
[F’(Xo)]
_
exLsts and
,
Following Rhelnboldt and Ortega [12] we show that iF(x)]
PROOF.
-I
exists at all
the iteration points.
Let
(x o,(8’+’)
-I
denote the closed sphere with
Xo
as the center and
-1
as the radius.
(’w’)-l)f DO
.q (x
Let D
Then, for xD| we have
(3.17)
(x o)
(x
-I
F’(x I) has an Inverse and hence for xED I,
Moreover, for x D uslng the Neumann Lemma we get
Therefore [F’(x
o)]
[F’(x)]
F’(x) has an inverse.
n=o{[F’(Xo)] -I [F’(Xo) -F’(x)]}n[F’(ro)] -I
-I
(3.19)
Therefore,
’ n:EO (s","ll --o11)"
since
"’’11"-" II
0
Therefore for
<
xD
{x
being a monotonic increasing sequence It follows from (3.10) and condition (c)
n
of Theorem (3.2),
"I1-,, -o11’ II J0
Hence
n
<":
"o"o’ CXo) )
(Xo’
Utilizing condition (xb) and (3.21) we get
(’,
"o)11
<
’’
(3.22)
358
R. SEN, R. CHATTOPADHYAY AND T. SAHA
n
(3.23)
Using (3.10) and the monotonic increasing property of {x
we further conclude that
oHo
-I
o
lk=o
THEOREM
3.3.
If
condition
(xb)
of
(x1-
Theorem
3.
o
replaced by
is
the
following
condition:
(I
Yn_lHn_lF’(Xn_l
a’ sup (I
)
xD o
>
a
I,
n
(3.24)
H F’(x ))
n n
n
)
and all other conditions of theorem 3.1 are fulfilled then the superlinear convergence
of the sequence {
POOF.
is ensured.
n
YnHn F’(xn
tim (I
n+m
(I
sup
D
x
o
(I
0 as n
,
If
f
x
x
x
[I
YnHnF’ (x n +
0
1)
.
,
0
>
(3.25)
then
x
lim
YnHn F’(xn
"oHo F’(xo )1’ (1
x
n-I
n
YnHn[F(x
,
F(Xn)]
t(x-x n ))] (x-x n )dt
By arguments analogous to theorem 3.1
+ (
t[O,l]
,
n))ll
II - II-
Therefore,
II *- nll
<
u
,:eto,.l
The continuity of F’(x) for
xD
II ’nHn F’(x
+
t(x*-
x
and relation (3,25) yield,
(3.26)
is proes superIinear convergence
o
{Xn},
REMA 3. I.
It may be noted that conditions (vii) and (viii) of theorem 3.1 are
required only to prove that
is an M-matrlx provided B is so.
The question may
Bn+
be raised as to how one can know these conditions
n
in advance.
experience one can say that such conditions are usually satisfied.
we have indicated in remark 2.1 how
4.
Bn+
Find D
o
x
x
If that is not so,
can be made an M-matrix when B is so.
n
ALGORITHM.
Step 1.
From computational
x in which F(x) is tsotone.
GENERAL CONVERGENCE FOR BROYDEN LIKE UPDATE METHOD
Step 2.
Choose x
O
DO
Ho
Step 3.
Choose
Step 4.
Compute A
O,
)
0 s.t. F(x
O
yo
and
<
O
0
O.
coute xl, k
yo Ho F’(x o ).
YoHoF’(Xo))n (xl- Xn)
I
o
If (I
and x
359
0 for
some n
no go
)
to
step
5,
otherwise
go to step 3.
Step 5.
Compute s
Step 6.
If
Step 7.
Compute
F(Xk+l)
T
O,
Otherwise
Hk+l
Step 8.
Compute
Step 9.
If
0([0
-4)
Hk+l=
H
Yk+t
k
F(Xk+
T
F(Xk)’ SkHkYk
H go to step 8.
k
+ (s k
HkYk)
Hsk
[_.]T
SkHkY k
Yk+iHk+IF(Xo) YkF(Xo)
s.t.
)
YkHkF’ (Xk))
I
x + s and
k
k
Xk+
stop, otherwise go to step 7.
F(Xk+l
Yk
SkHkYk
If
YkHkF(Xk
k
[I
sup
x6
<x k ,Xk+l>
Yk+lHk+l F’(x)]
go to step I0,
otherwise go to step 8.
Step I0. If k
k+l, go to step 5.
REMARK 4.1.
(1) The implementation of the condlttons (xa) and (xb)
determination of the scalar
Yn can
be done by a computer.
(2.1) is arbitrary except that the elements of
lead to some arbitrariness in
Hn+!.
Pn
(li) The matrix P
are greater than
Pn
(>0).
for the
in note
n
This must
In that case the choice of
Yn+l covertly depends
on
in addition to the conditions of (xa) and (xb).
so generated is however a
solution of the Quasi-Newton equation.
(Ill) The total number of multiplications and
divisions in finding Xk+l,
and
+ 3m,
Yk+l respectively is
2
2
i.e. 5m + 5m.
(iv) The number of function evaluations is m + 2m.
5. NUMERICAL EXAMPLE.
Pn
Hn+
(re+m2), 2m2+m, 2m2
Hk+
We take an example [14] in which every equation is linear except for the last
equation which is highly non-linear.
F(x)
where
and
[fl(x)]T,
We choose
fl(x)
1,2,...N-I
n
I
=-(N+I) + 2x + E xj, i- 1,2
fN(x)
-1 + II
N-I
j=l
run for N
The problem was
i
x]
5,
10 and 30.
We take x t
0.5 so that F(x )( O.
Incidentally F(x) is tsotone.
Define D
choose
A
Yo
[I-
o
1.
0.5<xl<l.O, i
hij .01.
hNN-
the rectangular parallelepiped given by
o
o
o
0.5.
where
N,
0.1, i
o
j
matrix,
a
is
)]
convergent
iJ
H
(hij
hl
1,2,...N, and
YoHoF’(xo
We summarize our computational experience in table I.
The computations were
on a Burroughs computer at the R.C.C. Calcutta using a FORTRAN IV
performed
language.
We solved the problem and in each case the exact solution was obtained.
In
360
R. SEN, R. CHATTOPADHYAY AND T. SAHA
table 2
we
provide a
of.
comparison
out"
res,llts
with those
For computational results of Newton’s method
Brown’s method.
[14].
Newton’s raethod and
and Brown’s method see
of
Table
No. of Iterations(n)
Dimenslon(N)
5
5
F(x)
x
(I,I,I)
T
I0
5
(I,I
,I)
30
6
(I,I
,l)
’r
T
CPU Time
0
3.637 sec
0
4.462 sec
0
8.548 sec
Table 2
Br own’ s Me thod
Newton’ s me thod
Dime nslon( N)
Converged in 6 its
Converged in 18 its
3
I0
30
2
Broyden-llke me thod
106
Converged in 5 its
Converged in 9 its
Converged in 5 its
Converged in 9 its
Converged in 6 its
Although Brown’s method has taken a larger nmber of [teratlons, It may take less
CPU time than that of the Broyden-like method because Brown’s method is quadratically
convergent.
6.
DISCUSSIONS.
(1) In the case of Broyden’s method the initial estimate B is found by taking a
o
finite difference analogue of F’(x ). We are considering the cas$ where the complete
o
computation of F’(x) is infeasible.
In our Broyden-like method w start with B an Mo
matrix so that H
0.
Since F is an isotone mapping it could be that all the
o
entries of F’(x
o
are non-negatlve. Nevertheless we can choose B such that
o
o
breover we have used Broyden updates (good method) and as such we have
equation,
called our method the Broyden-llke method. (il) In the case of DFP’s method, H is
o
always taken as a symmetric posltive-deflnite matrix. Mreover, a symmetric M-matrix
is positive
[12].
definite
Hence our H
definite matrix as in DFP’s method.
negative
transform
of
solutions
the given
nonlinear
equation
n
can sometimes
equations.
into
turn out
to be a positve-
(lii) Theorem 3.1 can only provide us with nonWith
a
suitable
another equation having
translation
non-negatlve
we
can
solutions
(iv) Our convergence proof seems to be much more elegant than the cases where
"majorizatlon principle" has been uttllzed or where the Euclidean norm of
-I
Ei(E i --A B i- I, for linear system of equations Ax b)has been utilized. (v) The
convergence theorem 3.1 is applicable where F is Isotone.
This restricts the sphere
only.
of
applicability
of
the
theorem.
However,
in
a
large number of
problems,
the
GENERAL CONVERGENCE FOR BROYDEN LIKE UPDATE METHOD
nonlinear operators are noton[cally Decomposlble (MDO)
361
[13] and convergence theorems
along the llne of theorem 3.1 can be developed for such operators.
The result would
be communicated in a separate paper.
A( NOWL E DGEME NTS.
The authors are ext.emely grateful to Dr. L. Dixon of Hatf[eld Polytechnic U.K.
for some constructive comments.
The second author, who is a N.B.H.M (India) research awardee, is grateful to the
National
Board for Higher Mathematics,
India for [nancial support to carry out the
research.
REFERENCES
I.
2.
3.
4.
DAVIDON, W.C., Variable metric method for minimization. Argonne Nat. Labs.
Re port ANL-5990, Rev. Argonne II (1959).
FLTCHER, R. and POWELL, M.J.D., A rapidly convergent descent method for
mlnimizatlons. Comput J. 6 (1963), 163-168.
BROYDEN, C.G., A class of methods for solving non-llnear simultaneous equations,
Maths. Comp. I9, (1965), 577-593.
DIXON, L.C.W., Numerical methods for non-llnear optimization (1972) ed. F.A.
Looftsma. Academic Press.
5.
6.
7.
GAY, D.M., Some convergence properties of Broyden’ s method. SIAM. J. Num Aual
16(4), (1979), 623-630.
algebraic equations, ed. P.
BROYDEN, C.G., Numerical methods for non, l.in..ear.
Rabinowlt z (1970).
On the local and superllnear
BROYDEN, C.G.
DENNIS, J.E. and MORE, J.J.
convergence of Quasl-newton methods, J. Inst. Maths. Appln.12, (1973), 223245.
MORE, J.J. and TRANGSTEIN, J.A., On the global convergence of Broyden’s method,
Math. Comput. 30, (1976), 523-540.
9. DENNIS, J.R. and WALKER, H.F., Convergence theorems for least change secant
update methods. SlAM J. Numerical Anal. 18, (1981), 949-987.
I0. DENNIS, J.E. and MORE, J.J., A characterization of superllnear convergence and
Its applications to Quasi-Newton methods. Maths. Comp. 28, (197 4), 549-560.
8.
11. DEXER, D.W. and KELLEY, C.T., Broyden’s method for a class of problems having
singular Jacoblan at the root, SIAM J. Numer. Anal. 22(I), (1985), 566-573.
and RHEINBOLD, W.C., Iteratlve solution of
several variables, Academic Press (1970).
12. ORTEGA, J.M.
nonllear equations in
13. COLLATZ, L., Functional Analysis and Numerical Analysis Ed. L.B. Nell, Academic
Press, N.Y. (1966).
14. BYRNE, G.D. and HALL, C.A., Numerical solution of system of non-linear
Academic Press N.Y. (1973).
algebraic eguations