This article was downloaded by:[Chinese University of Hong Kong]
On: 24 November 2007
Access Details: [subscription number 768616791]
Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954
Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Numerical Functional Analysis and
Optimization
Publication details, including instructions for authors and subscription information:
http://www.informaworld.com/smpp/title~content=t713597287
Newton's Method for Underdetermined Systems of
Equations Under the γ-Condition
Jin-Su He a; Jin-Hua Wang b; Chong Li c
a
Department of Mathematics, Zhejiang Normal University, Jinhua, People's
Republic of China
b
College of Sciences, Zhejiang University of Technology, Hangzhou, People's
Republic of China
c
Department of Mathematics, Zhejiang University, Hangzhou, People's Republic of
China
Online Publication Date: 01 May 2007
To cite this Article: He, Jin-Su, Wang, Jin-Hua and Li, Chong (2007) 'Newton's Method for Underdetermined Systems of
Equations Under the γ-Condition', Numerical Functional Analysis and Optimization, 28:5, 663 - 679
To link to this article: DOI: 10.1080/01630560701348509
URL: http://dx.doi.org/10.1080/01630560701348509
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf
This article maybe used for research, teaching and private study purposes. Any substantial or systematic reproduction,
re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly
forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents will be
complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be
independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings,
demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or
arising out of the use of this material.
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
Numerical Functional Analysis and Optimization, 28(5–6):663–679, 2007
Copyright © Taylor & Francis Group, LLC
ISSN: 0163-0563 print/1532-2467 online
DOI: 10.1080/01630560701348509
NEWTON’S METHOD FOR UNDERDETERMINED SYSTEMS
OF EQUATIONS UNDER THE -CONDITION
Jin-Su He Department of Mathematics, Zhejiang Normal University,
Jinhua, People’s Republic of China
Jin-Hua Wang College of Sciences, Zhejiang University of Technology,
Hangzhou, People’s Republic of China
Chong Li Department of Mathematics, Zhejiang University, Hangzhou,
People’s Republic of China
The convergence criterion of Newton’s method for underdetermined system of equations under
the -condition is established and the radius of the convergence ball is obtained. Applications to
analytic operator are provided and some results due to Shub and Smale (SIAM J. Numer. Anal.
1996, 33:128–148) are extended and improved.
Keywords Newton’s method; Smale’s point estimate theory; The -condition;
Underdetermined systems of equations.
AMS Subject Classification 65H10; 49M15.
1. INTRODUCTION
Finding solutions of a nonlinear operator equation
f (x) = 0
(1.1)
in a Banach space is a very general subject that is widely studied in both
theoretical and applied areas of mathematics, where f is a nonlinear
operator from a real or complex Banach space E to another F . When
f is Fréchet differentiable, the most important method to find an
Address correspondence to Chong Li, Department of Mathematics, Zhejiang University,
Hangzhou 310027, People’s Republic of China; E-mail: cli@zju.edu.cn
663
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
664
J.-S. He et al.
approximation solution is Newton’s method. One of the famous results on
Newton’s method is the well-known Kantorovich theorem (cf. [7]), which
guarantees convergence of Newton’s sequence to a solution under very
mild conditions. Another important result concerning Newton’s method
is Smale s point estimate theory (cf. [15–17]). The results on estimates
of the radii of convergence balls of Newton’s method are referred to
[19, 22, 23, 26, 27].
Recent interest is focused on finding zeros of singular nonlinear
systems, see, for example, [4–6, 9, 12, 14, 28]. Let n denote the
n-dimensional Euclidean space and let f : n → m be a nonlinear
Fréchet differentiable mapping with its Fréchet derivative denoted by Df .
In this case when Df (x) is singular for some x, Newton’s method can be
generalized to search for zeros of f , using the Moore–Penrose inverse of
the derivative; that is, the generalized Newton’s method is defined by
xn+1 = Nf (xn ) := xn − Df (xn )† f (xn )
for n = 0, 1 (1.2)
Note that a fixed point x ∗ ∈ n of the Newton operator Nf is a least square
solution of the system f (x) = 0, which, in general, is not necessarily a zero
of f . However, in the case when Df (x ∗ ) is surjective, x ∗ is a zero of f if and
only if it is a fixed point of the Newton operator Nf .
In the case when the initial point x0 is such that Df (x0 ) is surjective or
f has zero as a regular value, the convergence results of Newton’s method
(1.2) are obtained in [14] and have been successfully applied to explore
the complexity theoretic aspects of continuation methods for systems of
polynomial equations (cf. [1, 3, 6, 13, 14]). Here we pay special attention
to the convergence results of Newton’s method (1.2) due to Shub and
Smale in [14]. Suppose that n ≥ m and that x ∈ n is such that Df (x) is
surjective. Define
1
Df (x)† Dk f (x) k−1
,
(f , x) = sup k!
k≥2
(f , x) = Df (x)† f (x)
and
(f , x) = (f , x)(f , x)
Recall from [14] that y is a regular value of f if, for each x ∈ f −1 (y), Df (x)
is surjective. Then the main results about the convergence of Newton’s
method (1.2) in [14] are as follows (cf. [14, Theorems 1.4 and 1.7]).
Theorem 1.1. Let x0 ∈ n be such that Df (x0 ) is surjective. There is a
universal constant 0 approximately equal to 71 with this property: if f , x = x0 , are
as above with
(f , x0 ) < 0 ,
(1.3)
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
Newton’s Method for Underdetermined Systems
665
then Newton’s method (1.2) with initial point x0 is well-defined for all n ≥ 0 and
convergent to a zero x ∗ of f , and
2n −1
1
xn+1 − xn ≤
x1 − x0 2
for each n = 0, 1, (1.4)
Theorem 1.2. Let f : n → m have zero as a regular value, and define
= max (f , x)
x∈f −1 (0)
Then there is a universal constant C such that if the distance
d(x0 , f −1 (0)) <
C
,
(1.5)
then Newton’s method (1.2) with initial point x0 is well-defined for all n ≥ 0 and
convergent to a zero x ∗ of f , and (1.4) holds.
Remark 1.3. Define a real-valued function by
√ 2
for each t ∈ 0, 1 −
2
(t ) = 1 − 4t + 2t
2
Then, from the proofs of Theorems 1.1 and 1.2 in [14], it is easy to see
that 0 is indeed equal
to the unique root of the equation 2t = (t )2 in
√
the interval (0, 1 − 22 ), while C is the smallest positive root of the equation
t
= 0 . Thus, 0 = 0130716944 and C = 0069778332 (t )2
The purpose of the current paper is to use the notion of the -condition
for nonlinear operators in Banach spaces, which was first introduced and
explored by Wang in [25] for the study of Smale’s point estimate theory,
to investigate the convergence of Newton’s method (1.2) with initial point
x such that Df (x) is surjective. The generalized Smale’s -criterion and
-criterion of Newton’s method (1.2) are established under the -condition.
In particular, when the results obtained in the current paper are applied to
the special case when the operator f is analytic, Theorems 1.1 and 1.2 are
improved in such a way that the criterions (1.3) and (1.5) in Theorems 1.1
and 1.2 are respectively replaced by the weaker conditions (1.6) and (1.7)
below:
√
13 − 3 17
(f , x0 ) ≤
≈ 0157671
(1.6)
4
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
666
J.-S. He et al.
and
u0
,
d(x0 , f −1 (0)) <
(1.7)
00776121 is the smallest positive root of the equation
where u0 =
√
t
13−3 17
=
.
4
(t )2
2. NOTIONS AND PRELIMINARIES
Let A : n → m be a linear operator (or an m × n matrix). Recall
from [2, 10, 18, 20] (see also [11] for infinite dimensional Banach spaces)
that an operator (or an n × m matrix) A † : m → n is the Moore–Penrose
inverse of A if it satisfies the following four equations:
AA † A = A,
A † AA † = A † ,
(AA † )∗ = AA † , (A † A)∗ = A † A,
where A ∗ denotes the adjoint of A. Let ker A and im A denote the kernel
and image of A, respectively. For a subspace E of n , we use E to denote
the projection onto E . Then it is clear (cf. [2, 10, 11, 18, 20]) that
A † A = ker A⊥
and
AA † = im A (2.1)
In particular, in the case when A is surjective, A † = A ∗ (AA ∗ )−1 and
AA † = Im (2.2)
For x ∈ n and a positive number r , let B(x, r ) and B(x, r ) denote
respectively the open metric ball and the closed metric ball at x with
radius r . We assume throughout the whole paper that n ≥ m and that
f : n → m is a nonlinear operator that has continuous second Fréchet
derivative that is denoted by D2 f . The following definition is taken
from [25].
Definition 2.1. Let r > 0 and > 0. Let x0 ∈ n . Then f is said to satisfy
the -condition at x0 in B(x0 , r ) if Df (x0 ) is surjective and
Df (x0 )† D2 f (x) ≤
2
(1 − x − x0 )3
for each x ∈ B(x0 , r )
(2.3)
Below, we state a lemma that will be used in the remainder of this
paper.
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
Newton’s Method for Underdetermined Systems
667
√
Lemma 2.2. Let > 0 and 0 < r < 2−2 2 . Suppose that f satisfies the
-condition at x0 in B(x0 , r ). Then, for each point x ∈ B(x0 , r ), Df (x) is surjective
and
Df (x)† Df (x0 ) ≤
(1 − x − x0 )2
1 − 4x − x0 + 22 x − x0 2
(2.4)
Proof. Because
1
Df (x0 )† (Df (x) − Df (x0 )) = Df (x0 )†
D2 f (x0 + (x − x0 ))(x − x0 )d,
0
it follows from (2.3) that
Df (x0 )† (Df (x) − Df (x0 )) ≤
1
Df (x0 )† D2 f (x0 + (x − x0 ))x − x0 d
0
1
2
x − x0 d
3
0 (1 − x − x0 )
1
−1
=
(1 − x − x0 )2
< 1
≤
(2.5)
Hence, from the Banach lemma, it is seen that (In − Df (x0 )† (Df (x0 ) −
Df (x)))−1 exists and
(In − Df (x0 )† (Df (x0 ) − Df (x)))−1 ≤
(1 − x − x0 )2
1 − 4x − x0 + 22 x − x0 2
(2.6)
By (2.1), one has that
In − Df (x0 )† (Df (x0 ) − Df (x)) = ker Df (x0 ) + Df (x0 )† Df (x)
(2.7)
As
Df (x0 )ker Df (x0 ) = 0
and
Df (x0 )Df (x0 )† = Im
(2.8)
thanks to (2.2), we have that Df (x) = Df (x0 )(ker Df (x0 ) + Df (x0 )† Df (x)).
This implies that Df (x) is surjective because Df (x0 ) is surjective and
(ker Df (x0 ) + Df (x0 )† Df (x)) is invertible thanks to (2.7). Note by (2.7) and
(2.8) that
Df (x)† Df (x0 )(In − Df (x0 )† (Df (x0 ) − Df (x)))
= Df (x)† Df (x) = (ker Df (x))⊥ (2.9)
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
668
J.-S. He et al.
It follows from (2.6) that
Df (x)† Df (x0 ) = (ker Df (x))⊥ (In − Df (x0 )† (Df (x0 ) − Df (x)))−1 ≤ (In − Df (x0 )† (Df (x0 ) − Df (x)))−1 ≤
(1 − x − x0 )2
1 − 4x − x0 + 22 x − x0 2
The proof is complete.
3. GENERALIZED -THEORY
The majorizing function h, which is due to Wang [21, 24], will play a
key role in this section. Let > 0 and > 0. Define
h(t ) = − t +
t 2
1 − t
for each 0 ≤ t <
1
(3.1)
Let tn denote the sequence generated by Newton’s method with initial
value t0 = 0 for h, that is,
tn+1 = tn − h (tn )−1 h(tn )
for each n = 0, 1, (3.2)
Then we have the following proposition, which was proved in [21, 24].
√
Proposition 3.1. Suppose that = ≤ 3 − 2 2. Then the following
assertions hold.
(i) The zeros of h are
√
1 + − (1 + )2 − 8
,
r1 =
4
r2 =
1++
√
(1 + )2 − 8
4
(3.3)
and they satisfy
1
1
1 1
≤ r2 ≤ ≤ r1 ≤ 1 + √ ≤ 1 − √
2
2
2 and convergent to r1 .
(ii) tn is monotonic increasing
√
(iii) If = < 3 − 2 2, then
n √
(1 − q 2 ) (1 + )2 − 8
n
tn+1 − tn =
q 2 −1 n −1
n+1 −1
2
2
2(1 − q
)(1 − q
)
(3.4)
for each n = 0, 1, ,
(3.6)
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
Newton’s Method for Underdetermined Systems
669
where
√
1 − − (1 + )2 − 8
q=
√
1 − + (1 + )2 − 8
(3.7)
√
1 + − (1 + )2 − 8
=
√
1 + + (1 + )2 − 8
(3.8)
and
Proposition 3.2 below was known in [21, 24], but a more direct proof
was given in [8].
√
Proposition 3.2. Assume that < 3 − 2 2. Then
n √
(1 − q 2 ) (1 + )2 − 8
≤ 1 for each n = 0, 1, 2(1 − q 2n −1 )(1 − q 2n+1 −1 )
(3.9)
Recall that f has continuous second Fréchet derivative. In the
remainder of this section, let x0 ∈ n be such that Df (x0 ) is surjective.
Define
= Df (x0 )† f (x0 )
and
= Theorem 3.3. Let
√
= ≤ 3 − 2 2
√
1+− (1+)2 −8
. Suppose that f satisfies the -condition at x0 in B(x0 , r1 ).
Let r1 =
4
Then Newton’s method (1.2) with initial point x0 is well-defined and the generated
√
sequence xn converges to a zero of f in B(x0 , r1 ). Moreover, if = < 3 − 2 2,
then
xn+1 − xn ≤ q 2
n −1
x1 − x0 for each n = 0, 1, ,
(3.10)
where q is defined by (3.7).
Proof. We will use mathematical induction to prove that
xn+1 − xn ≤ tn+1 − tn
(3.11)
holds for each n = 0, 1, Granting this, because tn is monotonic
increasing and convergent, xn is a Cauchy sequence and hence converges.
Furthermore, this together with (3.6) and (3.9) implies that (3.10) holds.
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
670
J.-S. He et al.
To verify (3.11), noting that t0 = 0 and t1 = , one has that
x1 − x0 = Df (x0 )† f (x0 ) = ≤ t1 − t0 Therefore, (3.11) holds for n = 0. Assume that (3.11) holds for n =
0, 1, , k − 1. Then, we have
xk − x0 ≤ tk < r1 <
1−
√
2
2
(3.12)
Hence, Df (xk ) is surjective by Lemma 2.2 and xk+1 is well-defined.
Furthermore, by (2.4), we have that
Df (xk )† Df (x0 ) ≤
(1 − xk − x0 )2
1 − 4xk − x0 + 22 xk − x0 2
= −h (xk − x0 )−1 ≤ −h (tk )−1 ,
(3.13)
because −h (t )−1 is monotonic increasing on [0, 1 ). Below, we will prove
that (3.11) holds for n = k. Because xk − xk−1 = −Df (xk−1 )† f (xk−1 ) and
Df (xk−1 )Df (xk−1 )† = Im ,
f (xk ) = f (xk ) − f (xk−1 ) − Df (xk−1 )(xk − xk−1 )
1
=
[Df (xk−1 + (xk − xk−1 )) − Df (xk−1 )](xk − xk−1 )d
0
Hence, by (3.12) and (2.3), one has that
Df (x0 )† f (xk ) = Df (x0 )† (f (xk ) − f (xk−1 ) − Df (xk−1 )(xk − xk−1 ))
1
≤
Df (x0 )† [Df (xk−1 + (xk − xk−1 )) − Df (xk−1 )]xk − xk−1 d
0
≤
1
0
≤
≤
0
Df (x0 )† D2 f (xk−1 + s(xk − xk−1 ))xk − xk−1 2 ds d
1
0
0
1
0
0
2
dsxk − xk−1 2 d
(1 − (xk−1 − x0 + sxk − xk−1 ))3
2
ds(tk − tk−1 )2 d
(1 − (tk−1 + s(tk − tk−1 )))3
= h(tk ) − h(tk−1 ) − h (tk−1 )(tk − tk−1 )
= h(tk ),
(3.14)
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
671
Newton’s Method for Underdetermined Systems
where the last equality holds because −h(tk−1 ) − h (tk−1 )(tk − tk−1 ) = 0 by
(3.2) (with n = k). Because Df (x0 )Df (x0 )† = Im , (3.13) and (3.14) imply
that
xk − xk−1 = Df (xk )† f (xk )
≤ Df (xk )† Df (x0 ) · Df (x0 )† f (xk )
≤ −h (tk )−1 h(tk )
= tk+1 − tk Thus, (3.11) holds for n = k and the proof is complete.
4. GENERALIZED -THEORY
Recall that f is a nonlinear operator that has continuous second
Fréchet derivative. Throughout this section, we assume that x ∗ ∈ n is such
that Df (x ∗ ) is surjective. Recall that
√ 2
2
(t ) = 1 − 4t + 2t for each t ∈ 0, 1 −
2
The following lemma estimates the quantity Df (x0 )† f (x0 ), which will
be used in the proof of the main theorem of this section.
√
Lemma 4.1. Let 0 < r ≤ 2−2 2 and let x0 ∈ B(x ∗ , r ). Suppose that f satisfies
the -condition at x ∗ in B(x ∗ , r ). Then Df (x0 ) is surjective and
Df (x0 )† f (x0 ) ≤
1−u
x0 − x ∗ ,
(u)
(4.1)
where u = x0 − x ∗ .
Proof. By Lemma 2.2, Df (x0 ) is surjective and
(1 − u)2
(u)
(4.2)
x0 − x ∗ (1 − u)
(4.3)
Df (x0 )† Df (x ∗ ) ≤
Below we will show that
Df (x ∗ )† f (x0 ) ≤
Granting this, by (4.2) and Df (x ∗ )Df (x ∗ )† = Im , we have that
Df (x0 )† f (x0 ) ≤ Df (x0 )† Df (x ∗ )Df (x ∗ )† f (x0 ) ≤
1−u
x0 − x ∗ (u)
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
672
J.-S. He et al.
and so (4.1) is seen to hold. Observe that
f (x0 ) = f (x0 ) − f (x ∗ ) − Df (x ∗ )(x0 − x ∗ ) + Df (x ∗ )(x0 − x ∗ )
1
=
[Df (x ∗ + (x0 − x ∗ ))(x0 − x ∗ ) − Df (x ∗ )(x0 − x ∗ )]d
0
+ Df (x ∗ )(x0 − x ∗ )
1 =
D2 f (x ∗ + s(x0 − x ∗ ))(x0 − x ∗ )2 dsd + Df (x ∗ )(x0 − x ∗ )
0
0
(4.4)
Thus, by (2.3),
∗ †
1
Df (x ) f (x0 ) ≤
0
Df (x ∗ )+ D2 f (x ∗ + s(x0 − x ∗ ))(x0 − x ∗ )2 ds d
0
+ Df (x ∗ )† Df (x ∗ ) · x0 − x ∗ 1 2
≤
x0 − x ∗ 2 ds d + x0 − x ∗ ∗ )3
(1
−
sx
−
x
0
0
0
x0 − x ∗ =
1−u
and hence, (4.3) holds.
Let ū0 = 0080851 be the smallest positive root of the equation
√
t
= 3 − 2 2
2
(t )
(4.5)
Recall that u = x0 − x ∗ and = Df (x0 )† f (x0 ). Furthermore, set
=
(u)(1 − u)
and
= √
Theorem 4.2. Let r̄ = 2−2 2 and Suppose that f (x ∗ ) = 0 and f satisfies the
-condition at x ∗ in B(x ∗ , r̄ ). If x0 − x ∗ < ū0 , then the sequence xn generated
by Newton’s method (1.2) with initial point x0 converges to a zero of f . Moreover,
xn+1 − xn ≤ (q )2
where
n −1
x1 − x0 √
1 − − (1 + )2 − 8
q =
√
1 − + (1 + )2 − 8
(4.6)
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
673
Newton’s Method for Underdetermined Systems
Proof. By Lemma 4.1, Df (x0 )† is surjective and
= Df (x0 )† f (x0 ) ≤
1−u
1−uu
x0 − x ∗ =
(u)
(u) (4.7)
Then
= ≤
√
u
ū0
<
=
3
−
2
2
(u)2
(ū0 )2
because the function t →
√
[0, 1 − 22 ). Let
r1
=
t
(t )2
1 + −
(4.8)
is strictly monotonic increasing on
√
(1 + )2 − 8
4
(4.9)
Then by (3.4),
≤
r1
1
1 1
≤ 1+ √ ≤ 1− √
2
2 (4.10)
In order that Theorem 3.3 is applicable, we have to show the assertion:
There exists r ≥ r1 such that f satisfies the -condition at x0 in B(x0 , r ).
For this purpose let
r = r̄ − x0 − x ∗ (4.11)
Then, by (4.10)
√ √ √
2 (u)(1 − u)
2 1
2− 2 u
r =
− ≥ 1−
= 1−
≥ r1 2
2
2 Below we show that f satisfies the -condition at x0 in B(x0 , r ). To do this,
let x ∈ B(x0 , r ). Because f satisfies the -condition at x ∗ in B(x ∗ , r̄ ) and x −
x ∗ ≤ x − x0 + x0 − x ∗ ≤ r̄ thanks to (4.11), we obtain that
Df (x ∗ )† D2 f (x) ≤
2
(1 − x − x ∗ )3
(4.12)
Note that Df (x ∗ )Df (x ∗ )† = Im . Thus, using Lemma 2.2 and (4.12), we
conclude that
Df (x0 )† D2 f (x) ≤ Df (x0 )† Df (x ∗ ) · Df (x ∗ )† D2 f (x)
≤
2
(1 − u)2
(u) (1 − x − x ∗ )3
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
674
J.-S. He et al.
≤
(1 − u)3
2
(u)(1 − u) (1 − (x − x0 + x0 − x ∗ ))3
(1 − u)3
(1 − u − x − x0 )3
2
=
(1 − 1−u x − x0 )3
= 2
≤
2
(1 −
x
(u)(1−u)
− x0 )3
2
,
(1 − x − x0 )3
√
2
because 0 < (t ) < 1 for all t ∈ 0, 1 −
. Therefore, f satisfies the
2
-condition at x0 in B(x0 , r ) and the assertion holds. Thus, we apply
Theorem 3.3 to conclude that the sequence xn generated by Newton’s
method (1.2) with initial point x0 converges to a zero of f , and
=
xn+1 − xn ≤ (q )2
n −1
x1 − x0 ,
n = 0, 1, 2, The proof is complete.
5. APPLICATIONS TO ANALYTIC OPERATORS
Throughout this section, we shall always assume that the nonlinear
operator f : n → m is analytic. Let x ∈ n be such that Df (x) is
surjective. Following [14], we define
1
Df (x)† Dk f (x) k−1
(f , x) = sup
k!
k≥2
(5.1)
Also we adopt the convention that (f , x) = ∞ if Df (x) is not surjective.
Note that this definition is justified and in the case when DX (p) is
surjective, by analyticity, (f , x) is finite.
Let x0 ∈ n be such that Df (x0 ) is surjective. Let
= (f , x0 )
(5.2)
The following lemma shows that an analytic operator satisfies the
-condition at x0 in B(x0 , 1 ), the proof of which is easy and so is omitted
here.
Lemma 5.1. Let 0 < r ≤ 1 . Then f satisfies the -condition at x0 in B(x0 , r ).
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
Newton’s Method for Underdetermined Systems
675
To study the computational complexity of Newton’s method for
nonlinear operators in Banach spaces, Smale introduced in [15] the notion
of an approximation zero for Newton’s method in 1981. But it was found
that it did not describe completely the property of quadratic convergence
of Newton’s method and was inconvenient for the application in the study
of the computational complexity. Hence, Smale proposed in [16] two kinds
of the modifications of the notion again in 1986, see also [17]. Here we are
interested in the first one, which was also stated in [14].
Definition 5.2. A point x0 ∈ n such that Df (x0 ) is surjective is called
an approximation zero of f if Newton’s method (1.2) with initial point x0
converges to a zero x ∗ of f and (3.10) holds for q = 12 (x ∗ is called the
associated zero).
By Theorem 3.3 and Lemma 5.1, we have the following corollary that
improves Theorem 1.1 (i.e., Shub and Smale [14, Theorem 1.4]). Recall
that x0 ∈ n is such that Df (x0 ) is surjective and recall also that =
Df (x0 )† f (x0 ) and = with = (f , x0 ).
Corollary 5.3. If
√
13 − 3 17
≈ 0157671,
= ≤
4
then x0 is an approximate zero of f .
Proof. Let
r1 =
1+−
√
(1 + )2 − 8
4
Then by (3.4), r1 < 1 . Therefore, it follows from Lemma 5.1 that f satisfies
the -condition at x0 in B(x0 , r1 ). On the other hand,
√
√
13 − 3 17
≤
< 3 − 2 2
4
Hence, using Theorem 3.3, one has that Newton’s method (1.2) with initial
point x0 is well defined and the generated sequence xn converges to a
zero of f in B(x0 , r1 ). Moreover,
xn+1 − xn ≤ q 2
where
n −1
x1 − x0 ,
√
1 − − (1 + )2 − 8
q=
√
1 − + (1 + )2 − 8
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
676
J.-S. He et al.
Therefore,
we have that q ≤ 12 because
q increases as does on
√
√
13−3 17
13−3 17
is 12 . Thus x0 is an approximate
[0, 4 ] and the value of q at =
4
zero of f .
The following definition is taken from [3].
Definition 5.4. Let f : n → m be an operator with continuous Fréchet
derivative. A point y ∈ m is called a regular value of f if, for each x ∈
f −1 (y), Df (x) is surjective.
√
Recall that (t ) = 1 − 4t + 2t 2 for each t ∈ [0, 1 − 22 ). Let u0 =
00776121 be the smallest positive root of the equation
√
t
13 − 3 17
(5.3)
=
(t )2
4
By Theorem 4.2 and Lemma 5.1, one has the following corollary, which
improves Theorem 1.2 (i.e., Shub and Smale [14, Theorem 1.7]).
Corollary 5.5.
Let f : n → m have zero as a regular value and define
= max (f , x)
x∈f −1 (0)
Let x0 ∈ n satisfy
d(x0 , f −1 (0)) <
u0
(5.4)
Then x0 is an approximate zero of f .
Proof. By (5.4), there exists x ∗ ∈ f −1 (0) such that x0 − x ∗ <
x0 − x ∗ < (fu,x0 ∗ ) . Let
=
(f , x ∗ )
(u)(1 − u)
and
u0
;
hence
= ,
where = Df (x0 )† f (x0 ) and u = (f , x ∗ )x0 − x ∗ . √By Lemma 5.1,
2− 2
). Because u0
f satisfies the (f , x ∗ )-condition at x ∗ in B(x ∗ , 2(f
,x ∗ )
determined by (5.3) is less than u0 given by (4.5), Theorem 4.2 is
applicable. Thus we have that Newton’s method (1.2) with the initial point
x0 is well defined and
xn+1 − xn ≤ q 2
n −1
x1 − x0 ,
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
Newton’s Method for Underdetermined Systems
where
677
√
1 − − (1 + )2 − 8
q =
√
1 − + (1 + )2 − 8
By Lemma 4.1,
= DX (p0 )† X (p0 ) ≤
It follows that
1−u
x0 − x ∗ (u)
√
u
u0
13 − 3 17
≤
=
,
≤
(u)2
(u0 )2
4
(5.5)
because the function t → (tt )2 is strictly monotonic increasing on
√
[0, 1 − 22 ). Hence, we have that q ≤ 12 and x0 is an approximate zero of f .
We end this paper with a simple example showing that the convergence
is guaranteed by Corollary 5.3 but not by Theorem 1.1.
Example 5.6. Let 2 be endowed with the l∞ -norm and let f : 2 → be given by
f (x) = 1 + t1 +
3 2
1 3
t2 +
t1 + t23
20
300
for each x = (t1 , t2 ) ∈ 2 Then
t2
t2
3t2
Df (x)w = 1 + 1 w1 +
+ 2 w2 for each w ∈ 2 ,
100
10
100
t2
t1
3
D2 f (x)w y =
w1 y1 +
+
w2 y2 for each w, y ∈ 2 ,
50
10 50
1
1
3
w1 y1 z1 +
w2 y2 z2 for each w, y, z ∈ 2 D f (x)w y z =
50
50
and Dk f (x) = 0 for each k ≥ 4. Take x0 = (0, 0). Then we have that
Df (x0 )† = (1, 0)T . Consequently,
= Df (x0 )† f (x0 ) = 1
and
1 Df (x0 )† D2 f (x0 ) Df (x0 )† D3 f (x0 ) 2
, = max = 015
2
3
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
678
J.-S. He et al.
Hence
√
13 − 3 17
= = 015 <
,
4
and Corollary 5.3 is applicable but not Theorem 1.1 because = 015 >
0 = 0130716944 .
ACKNOWLEDGMENTS
This work was supported in part by the National Natural Science
Foundation of China (grant 10671175) and Program for New Century
Excellent Talents in University.
REFERENCES
1. E. Allgower and K. Georg (1993). Continuation and path following. Acta. Numer. 1–64.
2. A. Ben-Israel and T.N.E. Greville (1974). Generalized Inverses: Theory and Applications. John Wiley
& Sons, New York.
3. L. Blum, F. Cucker, M. Shub, and S. Smale (1997). Complexity and Real Computation. SpringerVerlag, New York.
4. J.P. Dedieu and M. Kim (2002). Newton’s method for analytic systems of equations with
constant rank derivatives. J. Complexity 18:187–209.
5. J.P. Dedieu and M. Shub (2000). Newton’s method for overdetermined systems of equations.
Math. Comput. 69:1099–1115.
6. J.P. Dedieu and M. Shub (2000). Multihomogeneous Newton’s method. Math. Comput. 69:
1071–1098.
7. L.V. Kantorovich and G.P. Akilov (1982). Functional Analysis. Pergamon, Oxford.
8. C. Li and J.H. Wang (2006). Newton’s method on Riemannian manifolds: Smale’s point
estimate theory under the -condition. IMA J. Numer. Anal. 26:228–251.
9. C. Li, W.H. Zhang, and X.Q. Jin (2004). Convergence and uniqueness properties of GaussNewton’s method. Comput. Math. Appl. 47:1057–1067.
10. M.Z. Nashed (1976). Generalized Inverse and Applications. Academic Press, New York.
11. M.Z. Nashed (1987). Inner, outer, and generalized inverses in Banach and Hilbert spaces.
Numer. Funct. Anal. Optim. 9:261–325.
12. M.Z. Nashed and X. Chen (1993). Convergence of Newton-like methods for singular operator
equations using outer inverses. Numer. Math. 66:235–257.
13. M. Shub and S. Smale (1993). Complexity of Bezout’s theorem I: Geometric aspects. J. Am.
Math. Soc. 6:459–501.
14. M. Shub and S. Smale (1996). Complexity of Bezout’s theorem IV: Probability of success,
Extensions. SIAM J. Numer. Anal. 33:128–148.
15. S. Smale (1981). The fundamental theorem of algebra and complexity theory. Bull. AMS.
4:1–36.
16. S. Smale (1986). Newton’s method estimates from data at one point. In: The Merging of
Disciplines: New Directions in Pure, Applied and Computational Mathematics (R. Ewing, K. Gross,
and C. Martin, eds.), Springer, New York, pp. 185–196.
17. S. Smale (1997). Complexity theory and numerical analysis. Acta Numer. 6:523–551.
18. G. Stewart and J. Sun (1990). Matrix Perturbation Theory. Academic Press, New York.
19. J.F. Traub and H. Wozniakowski (1979). Convergence and complexity of Newton’s iteration.
J. Assoc. Comput. Math. 29:250–258.
20. G.R. Wang, Y.M. Wei, and S.Z. Qiao (2004). Generalized Inverses: Theory and Computations. Science
Press, Beijing.
Downloaded By: [Chinese University of Hong Kong] At: 08:26 24 November 2007
Newton’s Method for Underdetermined Systems
679
21. X.H. Wang (1999). Convergence of Newton’s method and inverse function theorem in Banach
space. Math. Comput. 225:169–186.
22. X.H. Wang (2000). Convergence of Newton’s method and uniqueness of the solution of
equations in Banach space. IMA J. Numer. Anal. 20:123–134.
23. X.H. Wang (1980). The convergence on Newton’s method. KeXue TongBao (A Special Issue of
Mathematics, Physics and Chemistry). 25:36–37.
24. X.H. Wang and D.F. Han (1990). On the dominating sequence method in the point estimates
and Smale’s theorem. Scientia Sinica Ser. A. 33:135–144.
25. X.H. Wang and D.F. Han (1997). Criterion and Newton’s method. Chinese J. Numer. Appl.
Math. 19:96–105.
26. X.H. Wang and C. Li (2001). Local and global behaviors for algorithms of solving equations.
Chinese Sci. Bull. 46:441–448.
27. X.H. Wang and C. Li (2003). Convergence of Newton’s method and uniqueness of the solution
of equations in Banach spaces. Acta Math. Sinica English Ser. 19:405–412.
28. X.B. Xu and C. Li (2007). Convergence of Newton’s method for systems of equations with
constant rank derivatives. J. Comput. Math. (in press).