ADE (G1156) Spring 2006 Handout 3: Lipschitz condition and Lipschitz continuity
7
3
Lipschitz condition and Lipschitz continuity
3.1 Definition (Lipschitz-continuous function) Lipschitz-continuity comes in three different
flavours. Let f : Rm → Rm .
(a) Given an open set B ⊆ Rm , we say that f is Lipschitz-continuous on the open subset B if
there exists a constant Λ ∈ R+
0 (called the Lipschitz constant of f on B) such that
$f (x) − f (y)$ ≤ Λ $x − y$ ,
∀x, y ∈ B.
(15)
(b) The function f is called locally Lipschitz-continuous, if for each z ∈ Rn there exists an
L > 0 such that f is Lipschitz-continuous on the open ball of center z and radius L
BL (z) := {y ∈ Rm : $y − z$ < L} .
(16)
(c) If f is Lipschitz continuous on all of the space Rm (i.e. B = Rm in (15)), then f is called
globally Lipschitz-continuous.
3.2 Remark (local vs. global) Notice the fundamental difference between the local and global
versions of the Lipschitz-continuity. Whereas in the local version the Lipschitz constant Λ and
the open set B depend on each point z ∈ Rm , in the global version the constant Λ is fixed and
B = Rm . In particular, a globally Lipschitz-continuous function is locally Lipschitz-continuous,
but the viceversa is not true.
3.3 Remark (norms and Lipschitz constants) In (15) the norm $·$ can be any norm. However,
once a norm has been chosen one should stick to that single norm, as the Lipschitz constant Λ
depends on the particular choice of this norm. Unless otherwise stated, we use the Euclidean
norm in all our analysis.
3.4
Interpretation
To see what these definitions mean, let us consider the situation in one dimension. Suppose f
is a Lipschitz function on a neighborhood B of x ∈ R. This implies that, ∀y ∈ B,
!
!
! f (x) − f (y) !
!
!≤Λ
|f (x) − f (y)| ≤ Λ|x − y| ⇒ !
!
x−y
!
!
! f (x + h) − f (x) !
! ≤ Λ by putting y = x + h.
⇒ !!
!
h
If we were to let h → 0 and if the function f were differentiable, then the result would mean
|f ! (x)| ≤ Λ; that is that the derivative is bounded by the Lipschitz constant.
However, there is nothing in the defintion of Lipschitz-continuity that implies that f is differentiable. So in general we can’t proceed to this limit, since we don’t know if f is differentiable
at x. But this tells us all we need to know: being Lipschitz just means f can’t be too steep, the
bound on the difference quotient being Λ.
ADE (G1156) Spring 2006 Handout 3: Lipschitz condition and Lipschitz continuity
8
3.5
Examples and Counterexamples
The functions below are pictured in figure 1; we will examine them in turn, with B = [−1, 1].
3.6 Example Consider f1 (x) = x2 . We will show that f1 is locally Lipschitz-continuous but
not globally so.
This function is continuously differentiable. Pick up any point x ∈ R, we observe that
! ! !
!f1 (y)! =
sup
sup
|2y| ≤ 2 |x| + 1
(17)
y∈(x−1,x+1)
y∈(x−1,x+1)
being |y| ≤ |x| + 1 for y ∈ (x − 1, x + 1) by the triangle inequality. Now picking up two points,
y, z ∈ (x − 1, x + 1) it follows by the mean value theorem that for some ξ between y and z, that
!
!
! ! !
!f1 (θ)! |z − y| .
|f1 (z) − f1 (y)| = !f1! (ξ)(z − y)! ≤
sup
(18)
θ∈(x−1,x+1)
Using (17), we conclude that
|f1 (z) − f1 (y)| ≤ (2 |x| + 1) |z − y| , ∀z, y ∈ (x − 1, x + 1).
(19)
|f1 (y) − f1 (0)|
= |y| → ∞, as y → ∞,
|y − 0|
(20)
Hence the Lipschitz constant of f1 on (x − 1, x + 1) is Λ = 2 |x| + 1 and the function is locally
Lipschitz-continuous on all of R.
Observe that the Lipschitz constant depends on x and its neighbourhood. In particular, if
x → ∞ then Λ → ∞. This is an indication that the function may not be globally Lipschitz
continuous. Indeed, for any y *= 0
which means that there is no Λ that can satisfy the global Lipschitz property for y ∈ R.
" 2
x sin(1/x2 ) if x *= 0,
3.7 Example f2 (x) =
This function is differentiable everywhere:
0
if x = 0.
# $
# $
"
2x sin x12 − x2 cos x12
if x *= 0,
!
f2 (x) =
0
if x = 0.
(One has to use the definition of derivative to obtain f2! (0).)
However, since f2! (x) → 0 as x → 0, the derivative is not continuous. Is f2 Lipschitzcontinuous?
Define xn = (2nπ + π/2)−1/2 and yn = (2nπ)−1/2 for n = 1, 2, . . . , and suppose that f2 is
Lipschitz-continuous. Since xn , yn ∈ B for all n, there must exist Λ such that
!
!
! f2 (xn ) − f2 (yn ) !
! for all n
Λ ≥ !!
!
xn − yn
(2nπ + π2 )−1
(2nπ)−1/2 − (2nπ + π2 )−1/2
%
(
&
π '−1/2
−1/2
= 4n (2nπ)
+ 2nπ +
→ ∞ as n → ∞.
2
=
ADE (G1156) Spring 2006 Handout 3: Lipschitz condition and Lipschitz continuity
9
Plainly no such Λ can exist. Hence f2 is not Lipschitz-continuous.
In the first example we used the theorem “continuously differentiable =⇒ locally Lipschitz”.
Here we saw an example where a differentiable function whose derivative is not continuous turned
out not to be Lipschitz. However, this is not true of all such functions. For example, consider
the function (shown in the handout ‘2.2 Continuity and Differentiability—some facts of life’)
" 2
x sin(1/x) if x *= 0,
f (x) =
0
if x = 0,
which is differentiable, but f ! is not continuous. It is left as an exercise to show that f is globally
Lipschitz continuous.
3.8 Example This example shows that differntiability is a stronger concept than Lipschitz
continuity. f3 (x) = |x| This function is not differentiable at x = 0, but
"
−1 if x < 0,
f3! (x) =
1
if x > 0.
Since
|f3 (x) − f3 (y)| = ||x| − |y|| ≤ |x − y|,
for all x, y ∈ R,
we see that f3 is globally Lipschitz continuous, with Lipschitz constant Λ = 1.
3.9 Example f4 (x) = |x|1/2 This function is not differentiable at x = 0, but
" 1
− 2 (−x)−1/2 if x < 0,
!
f4 (x) =
1 −1/2
if x > 0.
2x
Is f4 Lipschitz-continuous?
Define xn = 1/n2 and yn = 0 for n = 1, 2, . . . , and suppose that f4 is Lipschitz-continuous.
Since xn , yn ∈ B for all n, there must exist Λ such that
!
!
! f4 (xn ) − f4 (yn ) !
!
!
Λ ≥ !
for all n
!
xn − yn
= n → ∞ as n → ∞.
Plainly no such Λ can exist. Thus f4 is not Lipschitz-continuous.
3.10
Relationships between Lipschitz continuity, continuity and differentiability
3.11 Theorem (Lipschitz ⇒ C0 ) Every locally Lipschitz-continuous function is continuous.
The proof of this Theorem is left as an exercise.
3.12 Theorem (C1 ⇒ Lipschitz) Every continuously differentiable function is locally Lipschitz.
ADE (G1156) Spring 2006 Handout 3: Lipschitz condition and Lipschitz continuity
10
Proof Let f : Rm → Rm be continuously differentiable. Fix any two points z, y ∈ Rm and
define the function f : [0, 1] → R as f (θ) := f (z + θ(y − z)). It is clear that
f (0) = f (z) and f (1) = f (y)).
(21)
Furthermore, by the chain rule, we know that f is differentiable and that
d
f (θ) = Df (z + θ(y − z)) · (y − z).
dθ
(22)
Here Df (w) is the Jacobian matrix of f at w:
 ∂f1
∂w1 (w)
···

.
..
..
Df (w) = 
.
∂fm
∂w1 (w) · · ·

∂f1
∂wm (w)

..

.
∂fm
∂wm (w)
(23)
∂fi
where ∂w
(w) is the partial derivative of the i-th component of f with respect to the j-th
j
coordinate.
By the definition of f and then by the fundamental theorem of calculus, we have
/ 1
f (y) − f (z)) = f (1) − f (0) =
f ! (θ) dθ
0
%/ 1
(
#
$
=
Df z + θ(y − z) dθ (y − z);
0
where the integral in the last line is a matrix whose i, j-th component is given by
/ 1
$
∂fi #
z + θ(y − z) dθ.
0 ∂wj
It follows that
$f (y) − f (z)$ ≤ $
/
0
1
#
$
Df z + θ(y − z) dθ$ $y − z$,
where we use the notation $·$ for both the vector norm and its associated matrix norm.
Notice that, by the triangle inequality for integrals, we have
/ 1
/ 1
#
$
#
$
$
Df z + θ(v − z) dθ$ ≤
$ Df z + θ(v − z) $ dθ
0
0
/ 1
≤ sup $ Df (z + θ(v − z))$
dθ
θ∈[0,1]
= sup $ Df (z + θ(v − z))$.
θ∈[0,1]
0
ADE (G1156) Spring 2006 Handout 3: Lipschitz condition and Lipschitz continuity
11
Furthermore by the equivalence of matrix norms we have
∃c0 > 0 : $A$ ≤ c0 $A$∞ , ∀A ∈ Rm×m ,
(24)
where c0 depends only on m and $·$∞ is the maximum row sum norm from last handout.
m
Thus, to
0 1establish Lipschitz continuity, we fix an arbitraty point x ∈ R and we establish a
bound for 0 Df (z + θ(y − z)) dθ in an appropriate neighbourhood B of x.
Let B = BL (x), with L arbitrary. Since f is continuously differentiable on B, which is
closed and bounded, there exists Λ0 such that
1
1
1 ∂fi
1
1
sup 1
(w)1
1 ≤ Λ0 , ∀ij ∈ [1 : m] .
w∈B ∂wj
Here we have applied the Weierstrass theorem which says that each continous function (∂wj fi )
is bounded on a closed and bounded set (B).
Now, given z, y ∈ B, it follows that z + θ(z − y) ∈ B for all θ ∈ [0, 1] because B is a ball, so
1
#
$1
sup 1 Df z + θ(y − z) 1 ≤ c0 mΛ0 =: Λ.
θ∈[0,1]
It follows that
$f (y) − f (z)$ ≤ Λ $y − z$ , ∀y, z ∈ B;
which is to say that f is Lipschitz-continuous on BL (x). Since x is arbitrary this means that f
is locally Lipschitz-continuous on Rm .
3.13 Remark Notice that in the proof above Λ0 , and therefore Λ, might depend on the point
x and the constant L. However, if we can find a bound that is independent of x and L(i.e., a
uniform bound), then it means that the function f is actually globally Lipschitz-continuous.
3.14 Corollary If $ Df (w)$ is uniformly bounded as $w$ → ∞, then f is globally Lipschitz.
3.15
Lipschitz-continuity with respect to some arguments
As you may have noticed in the course ntoes we used a slightly different version of Lipschitz
continuity: the Lipschitz continuity with respect to the first argument of a function
f : Rm × R → R.
(25)
We give now an adaptation of our earlier definition. The main point is that the only variable
that counts is the first one. It is useful though to have the function continuous with respect to
both variables.
3.16 Definition A function f such as in (25) is locally Lipschitz continuous with respect to its
first argument if is continuous and if for each (x, t) ∈ Rm × R there exists a L > 0 and a Λ > 0
such that
$f (y, s) − f (z, s)$ ≤ Λ $z − y$
(26)
ADE (G1156) Spring 2006 Handout 3: Lipschitz condition and Lipschitz continuity
12
for all z, y ∈ BL (x) and s ∈ (t − L, t + L). (Notice how the right-hand side is independent of
the second argument s.)
The function f is called globally Lipschitz-continuous with respect to its first argument if it
is continuous and if there is a Λ > 0 such that (26) is satisfied for all z, y in Rm and s ∈ R.
(Notice again the secondary role played by the time variable.)
A fact that we have used in the proof of uniqueness is the following:
3.17 Lemma (Characterization of local Lipschitz-continuity) A continuous function f , such as
in (25), is locally Lipschitz continuous with respect to its first argument if and only if for each
closed and bounded subset K of Rm × R, there exists an open set A ⊂ Rm such that K ⊂ A
and a constant ΛK > 0 such that
$f (x, t) − f (y, t)$ ≤ ΛK $x − y$ , ∀(x, t), (y, t) ∈ A.
(27)
Proof (For the bold and the knowledgeable.) The “if” part is easy and is left as an exercise. The “only if” part can be proved by contradiction. Suppose that the conclusion is
false. This means is, that there exists a closed and bounded subset K0 ∈ Rm × R such
that any n ∈ Z+ there exist xn , y n ∈ Rm , tn ∈ R, such that (xn , tn ), (y n , tn ) ∈ An , with
An = {(z, s) ∈ Rm × R : ∃(y, t) ∈ K0 : $z − y$ + |s − t| ≤ 1/n} and for which
$f (xn , tn ) − f (y n , tn )$ > n $xn − y n $ .
(28)
Notice that An ⊂ A1 which is closed and bounded. It follows that the sequences ((xn , tn ))n∈Z+
and ((y n , tn )n∈Z+ are bounded. There exists thus a family of integers X ⊆ Z+ and a point
(x∗ , t∗ ) ∈ A (the closure of A) such that
lim (xn , tn ) → (x∗ , t∗ ), n ∈ X.
n→∞
(29)
Likewise, there exists (y ∗ , t∗ ) ∈ A and Y ⊆ X such that
lim (y n , tn ) → (y ∗ , t∗ ), n ∈ Y.
n→∞
(30)
2
Notice that (x∗ , t∗ ), (y ∗ , t∗ ) ∈ n∈Y An = K0 . We show next that x∗ = y ∗ . Indeed, by the
continuity of f , it has to be bounded by a constant M over the closed and bounded set K0 .
So (28) implies that $xn − y n $ ≤ 2M/n, for n ∈ Y , which means, by passing to the limit, as
n → ∞, that
$x∗ − y ∗ $ = 0, i.e., x∗ = y ∗ .
(31)
To conclude the proof it is sufficient to observe that the function f fails to be Lipschitzcontinuous in any neighbourhood of (x∗ , t∗ ). Indeed for any Λ > 0 and for any open ball
B / (x∗ , t∗ ), for n ∈ Y big enough, we have that (xn , tn ) (y n , tn ) ∈ B, and
$f (xn , tn ) − f (y n , tn )$ > n $xn − y n $ ≥ Λ $xn − y n $ ,
in contrast with the local Lipschitz-continuity with respect to the first argument.
(32)
ADE (G1156) Spring 2006 Handout 3: Lipschitz condition and Lipschitz continuity
13
1
f2(x)
1
f (x)
1
0
−1
−1
0
x
1
−1
−1
0
x
1
0
x
1
1
f4(x)
3
f (x)
1
0
−1
−1
0
0
x
1
0
−1
−1
Figure 1: Example functions