MAT2125: Introduction to real analysis
Aaron Tikuisis
Winter 2019
Chapter 1
The real numbers
The field of real numbers, R, is a fundamental starting point for analysis. It is rich with structure:
we will begin with field operations (+, −, ×, ÷) which are applied to finitely many elements at a
time1 , and then order relations (<, >, ≤, ≥); then we will see the operations of least upper bound
and greatest lower bound, which are applied to (typically infinite) sets of real numbers.
We will describe the real numbers by giving axioms; the real numbers R is the unique structure
satisfying these axioms. Proving that the object R exists is a non-trivial feat, which will not be
covered in this course. Rather, we will take R and its axioms as given, and prove many other great
things using them.
1.1
Field axioms
Definition 1.1.1. A field is a set F equipped with two operations, addition + : F × F → F , and
multiplication, · : F × F → F , which satisfy the following properties:
(F1) For all a, b ∈ F , we have a + b = b + a;
(+ is commutative)
(F2) For all a, b, c ∈ F , we have (a + b) + c = a + (b + c);
(F3) There is an element 0 ∈ F such that a + 0 = 0 + a = a for all a ∈ F ;
(+ is associative)
(+ has an identity)
(F4) For any a ∈ F , there is an element −a ∈ F such that a + (−a) = (−a) + a = 0;
inverses)
(F5) For all a, b ∈ F , we have a · b = b · a;
(· is commutative)
(F6) For all a, b, c ∈ F , we have (a · b) · c = a · (b · c);
(· is associative)
(F7) There is an element 1 ∈ F such that 1 6= 0 and a · 1 = 1 · a = a for all a ∈ F ;
identity)
(· has an
(F8) For any a ∈ F \ {0}, there is an element a−1 ∈ F such that a · a−1 = a−1 · a = 1;
inverses)
(F9) For any a, b, c ∈ F , (a + b) · c = a · c + b · c.
(· has
( distributivity)
Example 1.1.2. The real numbers R is a field. Q and C are also fields; N and Z are not.
1
(+ has
Later in the course, we discuss infinite sums, but we need to develop limits before this is possible.
1
All of the valid operations in arithmetic are encoded by the field axioms. In this course, you are
allowed to use intuition about the consequences of these field axioms (e.g., “if ab = ac and a 6= 0
then b = c” – this is true, and can be proven directly from the axioms, but a proof from the axioms
isn’t expected). We will also use familiar notation from arithmetic, such as a + b + c for (a + b) + c,
a − b for a + (−b), ab for a · b, and ab for ab−1 .
Exercise 1.1.1. [TBB08, Exercise 1.3.5] Define Z/5Z
by the following tables.
+ 0 1 2 3 4
0 0 1 2 3 4
1 1 2 3 4 0
2 2 3 4 0 1
3 3 4 0 1 2
4 4 0 1 2 3
to be the set {0, 1, 2, 3, 4} with + and · defined
·
0
1
2
3
4
0
0
0
0
0
0
1
0
1
2
3
4
2
0
2
4
1
3
3
0
3
1
4
2
4
0
4
3
2
1
Prove that Z/5Z is a field.
Exercise 1.1.2. [TBB08, Exercise 1.3.6] Define Z/6Z to be the set {0, 1, 2, 3, 4, 5} with + and · defined
by the following tables.
+
0
1
2
3
4
5
0
0
1
2
3
4
5
1
1
2
3
4
5
0
2
2
3
4
5
0
1
3
3
4
5
0
1
2
4
4
5
0
1
2
3
·
0
1
2
3
4
5
5
5
0
1
2
3
4
0
0
0
0
0
0
0
1
0
1
2
3
4
5
2
0
2
4
0
2
4
3
0
3
0
3
0
3
4
0
4
2
0
4
2
5
0
5
4
3
2
1
Prove that Z/6Z isn’t a field. Determine precisely which axioms (F1)–(F9) don’t hold in Z/6Z.
1.2
Order structure
Definition 1.2.1. An ordered field is a field F equipped with a binary relation < satisfying the
following properties
(O1) If a < b and b < c then a < c;
(< is transitive)
(O2) For all a, b ∈ F , exactly one of the following hold: a = b or a < b or b < a;
order)
(O3) For all a, b, c ∈ F , if a < b then a + c < b + c;
(O4) For all a, b, c ∈ F , if a < b and 0 < c then a · c < b · c;
(< is a total
(< is compatible with +)
(< is compatible with ·)
Example 1.2.2. The real numbers R is an ordered field. So is Q, but C is not (if we try to use the
operation a + bi < c + di if a = c and b < d, then (O2) fails; see Exercise 1.2.3 to see that no other
relation works).
We can define ≤, >, ≥ in terms of <:
a ≤ b means a < b or a = b,
a > b means b < a, and
a ≥ b means a > b or a = b.
2
Exercise 1.2.1. Prove that if F is an ordered field then 1 > 0 in F .
[ Hint. If not, then 0 > 1; in this case, prove that −1 > 0 and use this to prove 1 > 0 after all.]
Exercise 1.2.2. Prove that if F is an ordered field then for all a, b, c ∈ F , if a < b and c < 0 then
a · c > b · c.
Exercise 1.2.3. Prove that for any binary relation < on C, we do not get an ordered field.
[Hint. By contradiction. Consider two cases, depending on whether i > 0 or i < 0.]
Exercise 1.2.4. Prove that for any binary relation on Z/5 (from Exercise 1.1.1), we do not get an
ordered field. More generally, if F is a field in which 1 + · · · + 1 = 0 (some finite, nonempty sum),
then for any binary relation on F , we do not get an ordered field.
Exercise 1.2.5. Let F be an ordered field and let a, b ∈ F . Show that if a < b then there exists c ∈ F
such that
a < c < b.
[Hint. Use the previous exercise to justify that one can average a and b.]
1.3
Bounded sets, infima, and suprema
Definition 1.3.1. Let F be an ordered field, let S ⊆ F be a subset, and let a ∈ F . Then a is an
upper bound for S if for any x ∈ S,
x ≤ a.
Likewise, a is a lower bound for S if for any x ∈ S,
a ≤ x.
S is bounded above if there exists an upper bound for S. S is bounded below if there exists a lower
bound for S. S is bounded set if it is bounded above and bounded below.
Example 1.3.2. Consider S := [0, 1] = {x ∈ R : 0 ≤ x ≤ 1}. An upper bound for S is 1. Another
upper bound for S is 2. A lower bound for S is 0. Another lower bound for S is −100. S is bounded.
Example 1.3.3. Consider N≥1 = {1, 2, 3, . . . }. This set is not bounded above. (However, we don’t
have enough information about R to prove this yet! It will be Theorem 1.3.13.) It is bounded below,
for example by 0.
Example 1.3.4. Consider T := {1/n : n ∈ N≥1 }. Then T is bounded below by 0 and bounded above
by 1.
Sometimes a set S is bounded below by an element a in that set: for example [0, 1] is bounded
below by 0, which is in [0, 1]. In such a case, we say that a is the minimum of S, and write a = min S.
Likewise, if a set S contains an upper bound b, then b is the maximum of S, and write a = max S.
However, some bounded sets don’t contain their bounds. For example, T := {1/n : n ∈
N≥1 } doesn’t contain any element which is a lower bound for T (Exercise 1.3.1).
Remember: when you write a = min S or a = max S, this implies that a ∈ S.
Definition 1.3.5. Let F be an ordered field, let S ⊆ F be a nonempty subset, and let a ∈ F . a is a
least upper bound(or supremum) for S if:
(i) it is an upper bound for S, and
(ii) for any upper bound b of S, we have b ≥ a.
3
When a is a least upper bound for S, we write
a = sup S.
Similarly, a is a greatest lower bound (or infimum) for S, written a = inf S, if
(i) it is a lower bound for S, and
(ii) for any lower bound b of S, we have b ≤ a.
By writing a = sup S or a = inf S, it suggests that the supremum and infimum of a set (if they
exist) are unique; indeed, this is the case (Exercise 1.3.3).
We also have the following conventions:
• If S is not bounded above, we write sup S = ∞.
• If S is not bounded below, we write inf S = −∞.
• We write sup ∅ = −∞ and inf ∅ = ∞.
We emphasize however that ±∞ are not real numbers, and we are not allowed to manipulate
them as if they were.
(Note that every real number is both an upper bound and a lower bound for ∅ – so although ∅
has upper and lower bounds, it does not have least, greatest ones respectively.)
Example 1.3.6. Consider the set S := (0, 1) = {x ∈ R : 0 < x < 1}. Then 1 is the least upper bound
for S. Certainly, by definition, we see that 1 is an upper bound. For (ii), suppose for a contradiction
that b is another upper bound and b ≤ 1 doesn’t hold. Then since < is a total order, it follows that
b < 1. By Exercise 1.2.5, we can find x ∈ R such that b < x < 1. But then x ∈ S and since b < x,
this contradicts that b is an upper bound.
Example 1.3.7. If a = max S then a = sup S. To see this, first a must be an upper bound. For (ii),
suppose that b is another upper bound for S. Then since a ∈ S and b ≥ x for all x ∈ S, we have
b ≥ a.
Definition 1.3.8. An ordered field F is complete if for any nonempty set S ⊆ F which is bounded
above, there is an element a ∈ F which is a least upper bound for S. In short, if S ⊆ F is nonempty
and bounded above, then sup S exists.
Example 1.3.9. The real numbers R is complete. (We will not prove this – but rather take it as a
fundamental fact.)
Interesting fact 1.3.10. In fact (although we also won’t prove it), R is the unique ordered field which
is complete: in other words, if F is some other ordered field which is complete then there is a bijection
φ : F → R which preserves +, ·, ≤, i.e., for all a, b ∈ F ,
φ(a + b) = φ(a) + φ(b),
φ(a · b) = φ(a) · φ(b),
and
a ≤ b if and only if φ(a) ≤ φ(b).
Proposition 1.3.11. For any set S ⊆ R which is nonempty and bounded below, inf S exists.
4
Proof. Exercise 1.3.4
Here is an example of how one might use completeness.
Proposition 1.3.12. There exists a real number a ∈ R such that a2 = 2.
Proof. Set
S := {x ∈ R : x ≥ 0 and x2 ≤ 2}.
Claim. For b > 0, b is an upper bound for S if and only if b2 ≥ 2.
To see this, first suppose b2 ≥ 2. If b isn’t an upper bound then there is x ∈ S such that x > b.
But then x2 > x · b > b2 ≥ 2, which contradicts that x ∈ S.
Conversely, suppose that b is an upper bound for S and, for a contradiction, that b2 < 2. Clearly
b ≥ 1. Then set := 2 − b2 > 0 and x := b + 3b , and observe that
2b
2
+ 2
3b
9b
2
2
+
≤ b2 +
3
9
2
<b +
= 2.
x 2 = b2 +
(since ≤ 2)
This means that x ∈ S, which is a contradiction since x > b. This proves the claim.
We see from the claim that S is bounded above, for example by 2. By the completeness of R, we
may set a := sup S.
Since a is an upper bound for S, we have a2 ≥ 2. If a2 6= 2 then a2 > 2, and we may set
. Then
:= a2 − 2 > 0 and b := a − 2a
b2 = a2 − 2a
2
2
+ 2 = 2 + 2 ≥ 2.
2a 4a
4a
Thus, by the claim, b is an upper bound for S, but since b < a, this contradicts that a is the least
upper bound.
The following may seem obvious, but it is an important consequence of completeness.
Theorem 1.3.13 (The Archimedean Property). The set N≥1 is not bounded above.
Proof. Suppose for a contradiction that N≥1 were bounded above. Then by completeness, it would
have a least upper bound, a := sup N≥1 . Since a is the least upper bound, a − 1 is not an upper
bound, so that there exists some m ∈ N≥1 such that
m > a − 1.
But then m + 1 ∈ N≥1 and m + 1 > a, contradicting that a is an upper bound.
Exercise 1.3.1. Let T := {1/n : n ∈ N≥1 }.
(a) Show that the set T doesn’t contain any element which is a lower bound for T . In other words,
we are not allowed to write min T .
(b) Determine inf T (and prove your answer is correct).
5
Exercise 1.3.2. Show that every finite set is bounded, and contains its bounds.
Exercise 1.3.3. Here we show that suprema are unique (when they exist). Let F be an ordered field,
let S be a subset of F and suppose that a and a0 are both least upper bounds for S. Prove that
a = a0 .
Exercise 1.3.4. Prove Proposition 1.3.11.
Exercise 1.3.5. Find the inf, sup, min, and max of the following sets, or show that they don’t exist.
(a) Z;
(b) N≥1 ;
Exercise 1.3.6. (a) Define S := {x ∈ Q : x2 ≤ 2}. Prove carefully (along the lines of the proof of
Proposition 1.3.12); that a := sup S satisfies a2 = 2. (b) Prove (without using the fact mentioned in
Interesting Fact 1.3.10) that Q is not complete.
Exercise 1.3.7. Define p(x) := x2 − x and S := {x > 0 : p(x) ≤ 1}. Prove, along the lines of the
proof of Proposition 1.3.12 (and definitely without using quadratic formula) that a := sup S exists
and satisfies p(a) = 1.
Exercise 1.3.8. Let S ⊆ Z be a nonempty set which is bounded above, and set p := sup S. Prove
that p ∈ Z (i.e., that p = max S).
1.4
The absolute value and distances between numbers
The absolute value of a real number a ∈ R is defined by
(
a,
if a ≥ 0;
|a| :=
−a,
if a < 0.
In other words, |a| = max{a, −a}.
Here are the basic properties of | · | : R → [0, ∞).
Proposition 1.4.1. Let x, y ∈ R. Then
(i) | − x| = |x|,
(ii) −|x| ≤ x ≤ |x|,
(iii) |xy| = |x| · |y|,
(iv) |x + y| ≤ |x| + |y| ( triangle inequality), and
(v) |x| − |y| ≤ |x − y|.
Proof. Exercise 1.4.1.
Definition 1.4.2. Let x, y ∈ R. The distance between x and y is
d(x, y) := |x − y|.
Here are the basic properties of the distance d : R × R → [0, ∞).
6
Proposition 1.4.3. Let x, y, z ∈ R. Then
(i) d(x, y) = d(y, x),
( d is symmetric)
(ii) d(x, y) = 0 iff x = y, and
( distinct points are separated)
(iii) d(x, z) ≤ d(x, y) + d(y, z).
(triangle inequality)
Proof. Exercise 1.4.2.
Exercise 1.4.1. Prove Proposition 1.4.1.
Exercise 1.4.2. Prove Proposition 1.4.3.
Exercise 1.4.3. [TBB08, Exercise 1.10.3] Let x, L ∈ R and let > 0. Show that the following are
equivalent:
(i) |x − L| < ,
(ii) L − < x < L + .
These conditions will appear prominently in the definitions of limits.
Exercise 1.4.4. [Sav17, Exercise 1.6.7] Let S ⊆ R be a bounded nonempty set. Prove that
sup{|a − b| : a, b ∈ S} = sup S − inf S.
7
Chapter 2
Sequences
2.1
Sequences and boundedness
Definition 2.1.1. A sequence of real numbers is a function f : N≥1 → R.
The above is the formal definition of a sequence. However, in practice, we think of a sequence
f : N≥1 → R as an infinite list of numbers
f (1), f (2), f (3), . . . .
As a convention, we rarely write a sequence as a function, and instead we use the following two ways
of writing sequences:
(an )∞
or (a1 , a2 , a3 , . . . ).
n=1
In both cases, the sequence corresponds to a function f : N≥1 → R by an = f (n).
Definition 2.1.2. A sequence (an )∞
n=1 is bounded if the set {an : n ∈ N≥1 } ⊆ R is bounded. Likewise,
to
be
bounded
above or bounded below, by the corresponding property for
we may define (an )∞
n=1
{an : n ∈ N≥1 }.
Note that in many textbooks (including the ones recommended in the syllabus), sequences are
denoted by {an }∞
n=1 or even {an }. We emphasize that sequences are not sets, for two reasons:
(i) The order in a sequence matters. Sets, by contrast, are not ordered: {1, 2} = {2, 1}.
(ii) Repeats in a sequence matter. Sets, by contrast, do not see repeats: {1, 1} = {1}.
Exercise 2.1.1. Determine which of the following sequences are bounded.
(a) (1, 2, 3, 4, . . . )
(b) (1, 21 , 13 , 14 , . . . )
(c) (an )∞
n=1 defined by
(
k,
an :=
0,
if n = 2k for some k ∈ N≥1 ;
otherwise.
(d) (an )∞
n=1 defined by
a1 := 1,
a2 := 2,
an−1 + nan−2
an :=
,
n+1
8
for n ≥ 3.
Exercise 2.1.2. Prove parts (ii)-(iv) of Proposition 2.3.1
2.2
Convergence
One of the primary things that we are interested in with sequences is their “long-term behaviour”:
that is, what an looks like when n is large. To this end, the first thing we might ask is: does an get
close to some particular number, as n gets large? This is what convergence of sequences means. Here
is the formal definition:
∞
Definition 2.2.1. Let (an )∞
n=1 be a sequence of real numbers and let L ∈ R. We say that (an )n=1
converges to L if for every > 0 there exists n0 ∈ N≥1 such that for all n ≥ n0 ,
|an − L| < .
In this case, we may write
L = lim an
or
n→∞
an → L as n → ∞.
If a sequence does not converge to any real number, then we say it diverges.
1 ∞
Example 2.2.2. The sequence (an )∞
n=1 = n n=1 converges to 0. To prove this, we work from the
definition.
Let > 0 be given. We must prove that there exists n0 ∈ N≥1 such that for all n ≥ n0 ,
|an − 0| < .
By the Archimedean Property (Theorem 1.3.13) there exists n0 such that n0 > 1 . Then if n ≥ n0 ,
1
1
≤
< .
n
n0
Since
1
n
> 0, we have |an − 0| = | n1 | < .
Example 2.2.3. Let us prove that
2n2 + 1
= 2.
n→∞ n2 + n
lim
To prove this, we first rearrange
2n2 + 1
2n2 + 1 − 2(n2 + n)
−
2
=
n2 + n
n2 + n
1 − 2n
= 2
n +n
1
2n
≤ 2
+ 2
n +n n +n
1
2
= 2
+
n +n n+1
1
2
≤ + .
n n
9
Now, given > 0, pick n0 such that n0 > 3 . Then if n ≥ n0 ,
2n2 + 1
3
3
−2 ≤ ≤
< .
2
n +n
n
n0
Proposition 2.2.4 (Uniqueness of limits). Let (an )∞
n=1 be a sequence and let L1 , L2 ∈ R. If
lim an = L1
and
n→∞
lim an = L2
n→∞
then L1 = L2 .
Proof. Suppose for a contradiction that L1 6= L2 , so without loss of generality, L1 < L2 . Then define
1
:= L2 −L
.
2
Since lim an = L1 , there exists n0 such that for all n ≥ n0 ,
n→∞
L1 − < an < L1 + .1
Using the second inequality and the definition of , we have for n ≥ n0 , that
an < L1 + =
L1 + L2
.
2
Likewise, since lim an = L2 , there exists m0 such that for all n ≥ m0 ,
n→∞
L2 − < an < L2 + ,
and from the first inequality we get for n ≥ m0 ,
an > L2 − =
L1 + L2
.
2
Take n := max{n0 , m0 }, so that n ≥ n0 and n ≥ m0 . Then we have
an <
L1 + L2
< an ,
2
a contradiction.
Complementing the definition of limits is the following notion of divergence to ±∞.
∞
Definition 2.2.5. Let (an )∞
n=1 be a sequence of real numbers. We say that (an )n=1 diverges to ∞ if
for every R > 0 there exists n0 ∈ N≥1 such that for all n ≥ n0 ,
an > R.
1
We are using Exercise 1.4.3 here.
10
Likewise, we say that (an )∞
n=1 diverges to −∞ if for every R > 0 there exists n0 ∈ N≥1 such that for
all n ≥ n0 ,
an < −R.
If (an )∞
n=1 diverges to ∞, we may write
∞ = lim an
n→∞
or an → ∞ as n → ∞.
If it diverges to −∞, we may write
−∞ = lim an
n→∞
or an → −∞ as n → ∞.
Again, we caution that ∞ is not a real number, so we must be careful not to treat it as one when
we write something like lim an = ∞.
n→∞
Example 2.2.6. lim n = ∞. To prove this, given R, we just choose any n0 > R.
n→∞
Example 2.2.7. The sequence ((−1)n )∞
n=1 diverges, but it does not diverge to ∞ or −∞ (Exercise
2.2.1). Note that this sequence is in fact bounded.
∞
Proposition 2.2.8. Let (an )∞
n=1 be a sequence which converges to some number L ∈ R. Then (an )n=1
is bounded.
Proof. Since lim an = L, using := 1, there exist n0 such that for all n ≥ n0 ,
n→∞
|an − L| < 1.
Take
M := max{a1 , a2 , . . . , an0 −1 , L + 1}.
(This maximum exists since the set is finite.)
Claim. M is an upper bound for the set {an : n ∈ N≥1 }.
To see this, let n ∈ N≥1 , and we must show an ≤ M . If n < n0 then an is among the list
a1 , . . . , an0 −1 , so that an ≤ M by the definition of M . On the other hand, if n ≥ n0 then by the
choice of n0 , we have an < L + 1 ≤ M . This proves the claim.
Setting
M 0 := min{a1 , . . . , an0 −1 , L − 1},
then the same reasoning shows that M 0 is a lower bound for the set {an : n ∈ N≥1 }. Thus the
sequence is bounded both above and below, as required.
A couple notes.
• The previous proposition can be reformulated as: if a sequence is not bounded then it diverges.
This can be used to show that a sequence which diverges to ±∞ does not converge (a fact
which is suggested by the terminology “diverges to ±∞”).
• The converse of the previous proposition does not hold: a sequence which is bounded might
not converge (Example 2.2.7).
Exercise 2.2.1. Prove that the sequence ((−1)n )∞
n=1 from Example 2.2.7 does not converge.
[Begin by supposing it does converge to some L ∈ R, and reach a contradiction.]
11
Exercise 2.2.2. Prove that
√2
n
∞
converges to 0. Prove this directly from the definition, rather
n=1
than using any theorems or rules (for example from previous calculus courses).
Exercise 2.2.3. Suppose that (an )∞
n=1 is a sequence that converges to some L ∈ R, and define a
by
sequence (bn )∞
n=1
bn := a2n , n ∈ N≥1 .
Prove that (bn )∞
n=1 also converges to L.
∞
Exercise 2.2.4. Suppose that (an )∞
n=1 and (bn )n=1 are sequences that both converge to some L ∈ R,
and define a sequence
(cn )∞
n=1 = (a1 , b1 , a2 , b2 , a3 , b3 , . . . ).
Prove that (cn )∞
n=1 converges to L.
Exercise 2.2.5. [TBB08, Exercise 2.4.15 and 2.4.16] If (an )∞
n=1 is a sequence
√ of nonnegative real
√ ∞
numbers converging to L ≥ 0, prove that the sequence ( an )n=1 converges to L. [Hint. You might
find it easiest to treat the cases L = 0 and L > 0 separately.]
Exercise 2.2.6. Let (an )∞
n=1 be a sequence of nonzero numbers. Prove that if lim an = ∞ then
n→∞
1
n→∞ an
lim
= 0. Is the converse true?
∞
Exercise 2.2.7. Let (an )∞
n=1 be a bounded sequence and let (bn )n=1 be a sequence that converges to
0. Prove that (an bn )∞
n=1 converges to 0.
2.3
Properties of limits
In this section, we will establish many of the tools which one likes to apply in everyday limit computations.
∞
Proposition 2.3.1 (Algebra of limits). Let (an )∞
n=1 and (bn )n=1 be sequences converging to La and
Lb respectively, and let c ∈ R. Then
(i) (an + bn )∞
n=1 converges to La + Lb .
(ii) (can )∞
n=1 converges to cLa .
(iii) (an bn )∞
n=1 converges to La Lb .
(iv) If an 6= 0 for all n and La 6= 0 then
1
an
∞
converges to
n=1
1
.
La
Remark 2.3.2. (i) and (ii) of the above proposition say that the set V of converging sequences forms
a vector space, and that the map (an )∞
n=1 7→ lim an is a linear map from V to R.
n→∞
Proof. (i): Let > 0 be given. Since lim an = La , there exists na such that for all n ≥ na ,
n→∞
|an − La | < 2 . Since lim bn = Lb , there exists nb such that for all n ≥ nb , |bn − Lb | < 2 . Set
n→∞
n0 := max{na , nb }. Then for n ≥ n0 ,
|(an + bn ) − (La + Lb )| ≤ |an − La | + |bn − Lb |
< + = .
2 2
(since n ≥ na and n ≥ nb ).
(ii)-(iv) are Exercise 2.1.2
12
∞
Proposition 2.3.3. Let (an )∞
n=1 and (bn )n=1 be converging sequences. If
an ≤ b n
for all n
then
lim an ≤ lim bn .
n→∞
n→∞
Proof. This is very similar to the proof of Proposition 2.2.4. Suppose for a contradiction that La > Lb .
b
Define := La −L
. Since lim an = La , there exists na such that for all n ≥ na ,
2
n→∞
La + > an > La − =
La + Lb
.
2
Likewise, since lim bn = Lb , there exists nb such that for all n ≥ nb ,
n→∞
Lb − < bn < Lb + =
Using n := max{na , nb }, we get an >
La +Lb
2
La + Lb
.
2
> bn , a contradiction.
Corollary 2.3.4. Let (an )∞
n=1 be a converging sequence such that
m ≤ an ≤ M
for all n.
Then
m ≤ lim an ≤ M.
n→∞
Proof. Since lim M = M , we apply the previous proposition with (bn )∞
n=1 := (M, M, M, . . . ) to get
n→∞
the second inequality. Similarly for the first.
∞
∞
Theorem 2.3.5 (Squeeze Theorem). Let (an )∞
n=1 , (bn )n=1 , (cn )n=1 be sequences such that:
∞
(i) (an )∞
n=1 and (cn )n=1 converge to the same number L, and
(ii) an ≤ bn ≤ cn for all n.
Then (bn )∞
n=1 also converges to L.
∞
Proof. Let > 0 be given. Using the definition of convergence for both (an )∞
n=1 and (bn )n=1 and
taking the maximum n0 from both, there exists n0 such that for all n ≥ n0 ,
L − < an < L + and L − < cn < L + .
Thus,
L − < an ≤ bn ≤ cn < L + for all n ≥ n0 , as required.
Example 2.3.6. Let us show that lim
n→∞
sin(n)
n
= 0. We use an := − n1 , bn :=
sin(n)
,
n
and cn := n1 . Since
−1 ≤ sin(x) ≤ 1 for all x, it follows that
an ≤ b n ≤ c n
for all n.
Moreover, by Example 2.2.2, cn → 0, and then using Algebra of Limits, an → −0 = 0. Hence the
Squeeze Theorem applied and shows that bn → 0.
13
Here is a variant on the Squeeze Theorem for divergence to ±∞.
∞
Proposition 2.3.7. Let (an )∞
n=1 and (bn )n=1 be sequences such that
an ≤ b n
for all n.
Then
(i) If lim an = ∞ then lim bn = ∞.
n→∞
n→∞
(ii) If lim bn = −∞ then lim an = −∞.
n→∞
n→∞
Proof. Exercise 2.3.1
Example 2.3.8. Fix r ∈ R. The sequence (rn )∞
n=1 is an important sequence called the geometric
sequence. Let us analyze its convergence behaviour.
The cases r = 0 and r = 1 are clear, as in these cases, the sequence is constant and so the limit
exists (and equals r).
If r > 1, then set x := r − 1 > 0. By the Binomial Theorem, we have
n 2
n
n
r = (1 + x) = 1 + nx +
x + · · · + xn ≥ nx.
2
Since nx → ∞, it follows by Proposition 2.3.7 that rn → ∞.
If r ∈ (0, 1), then set s := r−1 , so that rn = s1n . Since sn → ∞, it follows by Exercise 2.2.6 that
rn → 0.
If r ∈ (−1, 0) then set s := |r| ∈ (0, 1). We have
−sn ≤ rn ≤ sn
for all n, and by the previous case, sn → 0. Hence −sn → 0 also and then by the Squeeze Theorem,
rn → 0.
If r = −1 then we already saw in Example 2.2.7 that the sequence (rn )∞
n=1 diverges.
n
n ∞
Finally, if r < −1 then since |r| → ∞, we see that the sequence (r )n=1 is bounded neither above
(due to the even terms) nor below (due to the odd terms). It diverges, but doesn’t diverge to ±∞.
Here is an important fact.
Proposition 2.3.9. For any real number α ∈ R, there exists a sequence (an )∞
n=1 of rational numbers
(i.e., an ∈ Q for all n) such that
lim an = α.
n→∞
Proof. We will construct an such that
α−
1
≤ an ≤ α.
n
It will follow from the Squeeze Theorem (Theorem 2.3.5) that
lim an = α.
n→∞
We may find an integer p ∈ Z such that
nα − 1 ≤ p ≤ nα.
14
(2.1)
(For example, let p be the supremum of {k ∈ Z : k ≤ nα}, a set which is bounded above by nα, and
is nonempty by the Archimedean Property. By Exercise 1.3.8, p ∈ Z. Also, we have p ≤ nα + 1, or
else p − 1 would be a smaller upper bound for the set, and on the other hand, p ≥ nα − 1, or else
p + 1 would be in the set, contradicting that p is an upper bound.)
Setting an := np , the inequalities (2.1) clearly hold.
Exercise 2.3.1. Prove Proposition 2.3.7.
∞
Exercise 2.3.2. Let (an )∞
n=1 and (bn )n=1 be two convergent sequences, such that an < bn for all n.
Must it hold that
lim an < lim bn ?
n→∞
n→∞
Give a proof or counterexample.
∞
Exercise 2.3.3. [TBB08, Exercise 2.7.5] Let (an )∞
n=1 and (bn )n=1 be sequences. Which of the following
is true?
∞
∞
(a) If (an )∞
n=1 and (bn )n=1 are both divergent, then so is (an + bn )n=1 .
∞
∞
(b) If (an )∞
n=1 and (bn )n=1 are both divergent, then so is (an bn )n=1 .
∞
∞
(c) If (an )∞
n=1 and (an + bn )n=1 are both convergent, then so is (bn )n=1 .
∞
∞
(d) If (an )∞
n=1 and (an bn )n=1 are both convergent, then so is (bn )n=1 .
∞
1
(e) If (an )∞
is
convergent,
then
so
is
.
n=1
an
n=1
∞
2
(f) If (an )∞
n=1 is convergent, then so is ((an ) )n=1 .
∞
(g) If ((an )2 )n=1 is convergent, then so is (an )∞
n=1 .
∞
(h) If ((an )2 )n=1 is convergent and an ≥ 0 for all n, then so is (an )∞
n=1 .
Exercise 2.3.4. [TBB08, Exercise 2.8.9] Let (an )∞
n=1 be a sequence of positive numbers. Prove that if
an
an
lim an+1 exists and is strictly less than 1 ( lim an+1
< 1) then
n→∞
n→∞
lim an = 0.
n→∞
2.4
Monotone convergence criterion
Here we study sequences which are monotone, meaning that they always tend in the same direction
(increasing or decreasing).
Definition 2.4.1. Let (an )∞
n=1 be a sequence.
(i) We say (an )∞
n=1 is (weakly) increasing if
a1 ≤ a2 ≤ a3 ≤ · · · .
(ii) We say (an )∞
n=1 is strictly increasing if
a1 < a2 < a3 < · · · .
15
(iii) We say (an )∞
n=1 is (weakly) decreasing if
a1 ≥ a2 ≥ a3 ≥ · · · .
(iv) We say (an )∞
n=1 is strictly decreasing if
a1 > a2 > a3 > · · · .
(v) Finally, we say (an )∞
n=1 is monotone if it is either increasing or decreasing.
Note that in some textbooks (including [TBB08]), “increasing” is used where we use “strictly
increasing”, and “nondecreasing” is used where we use “increasing”.2 In our context, the concept of
(weakly) increasing and decreasing sequences is most important, and strictness is secondary.
The following is a powerful result about monotone sequences: the converge exactly when they are
bounded.
Theorem 2.4.2 (Monotone Convergence Criterion). Let (an )∞
n=1 be a monotone sequence. Then it
converges if and only if it is bounded.
Specifically, if (an )∞
n=1 is bounded and increasing, then it converges to sup{an : n ∈ N≥1 }, whereas
if it is bounded and decreasing, then it converges to inf{an : n ∈ N≥1 }.
Proof. If the sequence converges then it is bounded by Proposition 2.2.8. For the converse, we will
do the case that (an )∞
n=1 is bounded and increasing. The decreasing case is exactly the same (or
alternatively, it can be recovered from the increasing case by considering the sequence (−an )∞
n=1 ).
Set L := sup{an : n ∈ N≥1 }. To show convergence, let > 0 be given. Since L is the least upper
bound for {an : n ∈ N≥1 }, L − is not an upper bound, so there exists n0 such that
L − < an0 .
This is our n0 ; now suppose n ≥ n0 . Then since the sequence is increasing,
an ≥ an0 > L − .
On the other hand, since L is an upper bound, an ≤ L. Altogether,
L − < an ≤ L < L + ,
as required.
Example 2.4.3. Consider the sequence
n≤n+1⇒
√1
n
∞
. This sequence is decreasing, since for all n, we have
n=1
√
√
1
1
n≤ n+1⇒ √ ≥ √
.
n
n+1
It is also bounded below by 0. By the Monotone Convergence Criterion, it follows that this sequence
converges to its infimum, which is 0.
∞
Exercise 2.4.1. [Sav17, Exercise 2.3.1] Suppose that (an )∞
n=1 and (bn )n=1 are sequences such that
∞
∞
(an )∞
n=1 is increasing, an ≤ bn for all n, and (bn )n=1 converges. Prove that (an )n=1 converges.
2
This use of “nondecreasing” is confusing. It is natural to think that “nondecreasing” means “not decreasing”, but
∞
there are sequences which are neither increasing nor decreasing, such as ((−1)n )n=1 .
16
Exercise 2.4.2. [TBB08, Exercise 2.9.2] Define a sequence (an )∞
n=1 by
a1 := 1,
√
an+1 := 1 + an ,
n ≥ 1.
Prove that this sequence
is increasing, and determine whether it converges.
√
1+ 5
[Hint. Define γ := 2 , so that 1 + γ = γ 2 . Use induction to prove that an ≤ γ for all n, and then
use this to prove the sequence is increasing.]
∞
∞
Exercise 2.4.3. Let (an )∞
n=1 , (bn )n=1 , and (cn )n=1 be sequences such that
an ≤ b n ≤ c n
for all n.
∞
∞
(a) If (an )∞
n=1 and (cn )n=1 are both increasing, must (bn )n=1 be?
∞
∞
(b) If (an )∞
n=1 and (cn )n=1 are both bounded, must (bn )n=1 be?
∞
∞
(c) If (an )∞
n=1 and (cn )n=1 are both convergenc, must (bn )n=1 be?
2.5
Subsequences
Given a sequence
(a1 , a2 , a3 , . . . )
we may want to forget parts of the sequence and only take certain terms. What we get when we do
this is a subsequence. Here is the formal definition.
∞
Definition 2.5.1. Let (an )∞
n=1 be a sequence. A subsequence is a sequence of the form (ank )k=1 ,
∞
where (nk )k=1 is a strictly increasing sequence with nk ∈ N≥1 for all k.
One needs to be careful not to get confused with the notation of subsequences. Note that we
typically index a subsequence with a different variable (k instead of n in the above definition).
Since the subsequence arises from the sequence, we typically don’t write something like (bn )∞
n=1 for a
subsequence.
If we think of a sequence as a list, a subsequence arises by erasing some of the entries of the list
(specifically, erasing all of the entries that aren’t in {nk : k ∈ N≥1 }).
Since we require the indices of the subsequence, (nk )∞
k=1 , to be strictly increasing, the following
are not examples of subsequences:
(a1 , a1 , a1 , . . . ),
(a2 , a1 , a4 , a3 , a6 , a5 , . . . ).
∞
Example 2.5.2. Let (an )∞
n=1 be a sequence. Then the sequence of even terms (a2n )n=1 is a subsequence.
(We could write this sequence as (ank )∞
k=1 by setting nk := 2k.)
Example 2.5.3. Define (an )∞
n=1 by
an := n +
1
.
n
Here are a couple subsequences of this sequence:
(i) Taking nk := 2k , we have
−1 2
−2 3
−3
(ank )∞
k=1 = (1 + 1, 2 + 2 , 2 + 2 , 2 + 2 , . . . ).
17
(ii) Taking nk := k + 1, we have
(ank )∞
k=1
= k+1+
1
k+1
∞
.
k=1
Proposition 2.5.4. If (an )∞
n=1 converges to L, then any subsequence also converges to L.
Proof. Exercise 2.5.1.
Proposition 2.5.5. Every sequence contains a monotone subsequence.
Proof. Let (an )∞
n=1 be a sequence. We begin by trying to construct an decreasing sequence in a naı̈ve
way: we look for points m of the sequence where am is bigger than all later terms (i.e., am ≥ an for
all n ≥ m). We’ll call such indices m “turn-back points”. The idea is that, if there are infinitely
many of these, then we will get an decreasing sequence by taking them all; if there are only finitely
many, then there must be a increasing subsequence. Let us explain: we have two cases.
Case 1. There are infinitely many turn-back points. In this case, we may take a sequence n1 <
n2 < · · · such that each nk is a turn-back point. In particular, we have an1 ≥ an2 ≥ · · · , so the
subsequence (ank )∞
k=1 is decreasing.
Case 2. There are only finitely many turn-back points. In this case, let M be the last turn-back
point, so that every n > M is not a turn-back point. Take some n1 > M ; since it is not a turn-back
point, there is some n2 > n1 such that an1 < an2 . Likewise, n2 is not a turn-back point, so there is
some n3 > n2 such that an2 < an3 . Continuing in this way, we obtain a subsequence (ank )∞
k=1 that is
strictly increasing.
Here is an important corollary of the previous proposition.
Corollary 2.5.6 (Bolzano–Weierstrass Theorem). Every bounded sequence has a convergent subsequence.
∞
Proof. Let (an )∞
n=1 be a bounded sequence. By the previous proposition, it has a subsequence (ank )k=1
which is monotone. This subsequence is, of course, also bounded. Hence by the Monotone Convergence Criterion, (ank )∞
k=1 converges.
Exercise 2.5.1. Prove Proposition 2.5.4.
∞
∞
Exercise 2.5.2. [TBB08, Exercise 2.11.3] Suppose that (an )∞
n=1 and (bn )n=1 are sequences and (ank )k=1 ,
∞
∞
(bmk )∞
k=1 (respectively) are subsequences. Must (ank + bmk )k=1 be a subsequence of (an + bn )n=1 ?
∞
Exercise 2.5.3. Let (an )∞
n=1 be a sequence. Prove that (an )n=1 is not bounded above if and only if it
∞
has a subsequence (ank )k=1 which diverges to ∞.
Exercise 2.5.4. [TBB08, Exercise 2.11.11] Give an example of a sequence (an )∞
n=1 such that:
(i) For every natural number k ∈ N≥1 , there is a subsequence of (an )∞
n=1 converging to k, and
(ii) For every α ∈ R, if α is a limit of a subsequence of (an )∞
n=1 , then α ∈ N≥1 .
Exercise 2.5.5. Let (an )∞
n=1 be a sequence. Suppose that for every M , there exists a subsequence
1
∞
(ank )∞
such
that
lim
a
nk = M . Prove that there is a subsequence (amk )k=1 such that lim amk = 0.
k=1
k→∞
Is it possible to even arrange that (amk )∞
k=1 is monotone?
18
k→∞
2.6
Cauchy sequences
It is useful (especially in developing further theory) to characterize when a sequence converges, in a
way that doesn’t require knowing what the sequence converges to. This is what we will do in this
section. Note that the Monotone Convergence Theorem does this, in the special case of monotone
sequences: it says that a monotone sequence converges if and only if it is bounded; the condition “it
is bounded” makes no reference to any (potential) value for the limit. But that theorem is limited to
monotone sequences, and we already know that the condition “it is bounded” will not characterize
convergence for non-monotone sequences.
Definition 2.6.1. A sequence (an )∞
n=1 is Cauchy if for all > 0 there exists n0 such that for all
m, n ≥ n0 ,
|am − an | < .
Theorem 2.6.2 (Cauchy Convergence Criterion). Let (an )∞
n=1 be a sequence of real numbers. Then
it converges if and only if it is Cauchy.
Proof. ⇒: Assume that (an )∞
n=1 converges to some value, L ∈ R. We will show that the definition of
a Cauchy sequence holds; so let > 0 be given. By the definition of convergence with /2 in place of
, there exists n0 such that for all n ≥ n0 ,
|an − L| < .
2
Now, if m, n ≥ n0 then
|am − an | ≤ |am − L| + |L − an |
< + = .
2 2
(∆-inequality)
as required.
∞
⇐: Assume that (an )∞
n=1 is Cauchy. First, we must establish that (an )n=1 is bounded; this is
Exercise 2.6.1. Now, we may apply the Bolzano–Weierstrass Theorem (Corollary 2.5.6) to get a
∞
subsequence (ank )∞
k=1 which converges to some value, L ∈ R. Let us show that (an )n=1 converges to
L.
Given > 0, by the definition of a Cauchy sequence, pick n0 such that for all m, n ≥ n0 ,
|am − an | < .
2
Next, since lim ank = L, we may also find k0 such that for all k ≥ k0 ,
k→∞
|ank − L| < .
2
Let us now fix k1 ≥ k0 such that nk ≥ n0 ; this is possible since the sequence (nk )∞
k=1 is strictly
increasing.
For n ≥ n0 , we now have
|an − L| ≤ |an − ank | + |ank − L|
< + = .
2 2
as required.
19
(∆-inequality)
Exercise 2.6.1. Finish the proof of Theorem 2.6.2, by proving (without using that theorem) that every
Cauchy sequence is bounded.
∞
Exercise 2.6.2. [TBB08, Exercise 2.12.1] Show directly that the sequence n1 n=1 is Cauchy.
Exercise 2.6.3. (cf. [TBB08, Example 2.42]) Let (γn )∞
n=1 be a sequence of numbers in (0, 1). Define
∞
a sequence (an )n=1 by a1 := 1, a2 := 2, and
an := γn an−2 + (1 − γn )an−1 ,
for n ≥ 3.
(a) Prove that
|an − an−1 | = γn γn−1 · · · γ3
for all n ≥ 3.
(b) Prove that for all m ≥ n,
|am − an | ≤ |an+1 − an |.
∞
(c) Give an example of a sequence (γn )∞
n=1 for which the associated sequence (an )n=1 does not converge.
(d) Prove that if
lim inf γn < 1
n→∞
then
(an )∞
n=1
does converge.
Exercise 2.6.4. [TBB08, Exercise 2.12.5] Give an example of a sequence (an )∞
n=1 which is not Cauchy,
yet such that for every > 0, there exists n0 such that for all n ≥ n0 ,
|an+1 − an | < .
[Hint. Suppose that (bn )∞
n=1 is another sequence and an is defined by an := b1 + · · · + bn . What does
this condition tell us about the bn ? ]
2.7
lim sup and lim inf
∞
Definition 2.7.1. Let (an )∞
n=1 be a sequence of real numbers. The limit superior of (an )n=1 is
lim sup an := inf{β ∈ R : ∃n0 such that an ≤ β ∀n ≥ n0 }.
n→∞
The limit inferior of (an )∞
n=1 is
lim inf an := sup{β ∈ R : ∃n0 such that an ≥ β ∀n ≥ n0 }.
n→∞
In other words, the limit superior is the infimum of the set of eventual upper bounds for (an )∞
n=1 ,
and similarly for the limit inferior. Using the notational conventions we introduced around inf and
sup, we write the following:
• If (an )∞
n=1 is not bounded above then (since we have no eventual upper bounds), lim sup an =
n→∞
+∞.
• If (an )∞
n=1 is not bounded below then (since we have no eventual lower bounds), lim inf an = −∞.
n→∞
20
• If (an )∞
n=1 converges to −∞ then (since every number, no matter how negative, will be an
eventual upper bound), lim sup an = −∞.
n→∞
• If (an )∞
n=1 converges to ∞ then (since every number, no matter how large, will be an eventual
lower bound), lim inf an = +∞.
n→∞
Example 2.7.2. Define an :=
1
n
+ (−1)n . Let’s first compute lim sup an .
n→∞
1
,
First, we see that any number β > 1 will be an eventual upper bound: as long as n ≥ n0 := β−1
we have that an ≤ β. On the other hand, 1 is not an eventual lower bound (there is no n0 such that
an ≤ 1 for all n ≥ n0 ), since there are arbitrarily large n such that an > 1. Thus,
{β ∈ R : ∃n0 such that an ≤ β ∀n ≥ n0 } = (1, ∞)
and so
lim sup an = inf(1, ∞) = 1.
n→∞
Now we compute lim inf an . Here, −1 is clearly a lower bound, and so any number ≤ −1 is also
n→∞
a lower bound (whence an eventual lower bound). However, for any β > −1, β is not an eventual
1
lower bound: for any n0 , we may find n ≥ n0 such that n is odd and n ≥ β+1
, and so an < β. Thus,
{β ∈ R : ∃n0 such that an ≥ β ∀n ≥ n0 } = (−∞, −1]
and so
lim inf an = sup(−∞, −1] = −1.
n→∞
Proposition 2.7.3. For any sequence (an )∞
n=1 ,
lim inf an ≤ lim sup an .
n→∞
n→∞
Proof. If the sequence isn’t bounded then either lim sup an = ∞ or lim inf an = −∞, and in either
n→∞
n→∞
case, the result is trivial. So let us assume that (an )∞
n=1 is bounded.
Consider the sets used to define lim sup and lim inf:
S := {β ∈ R : ∃n0 such that an ≤ β ∀n ≥ n0 },
T := {α ∈ R : ∃m0 such that an ≥ α ∀n ≥ m0 }.
If β ∈ S and α ∈ T then there is some n0 as in the definition of S and an m0 as in the definition of
T . Then with n := max{m0 , n0 }, we get
α ≤ an ≤ β.
Thus we have shown that every α ∈ T is below every β ∈ S (abusing notation, we could write
T ≤ S).
Rewording this, we have that every α ∈ T is a lower bound for S, and thus for all α ∈ T ,
α ≤ inf S = lim sup an .
n→∞
But then this shows that lim sup an is an upper bound for T , so that
n→∞
lim sup an ≥ sup T = lim inf an ,
n→∞
n→∞
as required.
21
We now give another formula for lim sup and lim inf, which justifies the notation.
Proposition 2.7.4. Let (an )∞
n=1 be a bounded sequence of real numbers. Then
lim sup an = lim sup{an , an+1 , an+2 , . . . }
n→∞
and
n→∞
(2.2)
lim inf an = lim inf{an , an+1 , an+2 , . . . }.
n→∞
n→∞
Proof. We will prove the formula for lim sup; the other one can be proven by essentially the same
argument.
For notational convenience, let us set
bn := sup{an , an+1 , an+2 , . . . }.
Note that since {an , an+1 , an+2 , . . . } ⊆ {an+1 , an+2 , . . . }, it follows that
bn ≥ bn+1
for each n.
∞
Thus the sequence (bn )∞
n=1 is decreasing. It is also bounded, since (an )n=1 is, so by the Monotone
Convergence Criterion, it converges.
Set L := lim bn . We will prove lim sup an = L by two-way inequality.
n→∞
n→∞
≤: For each k, bk is an eventual upper bound: setting n0 := k we have that bk ≤ an for all n ≥ n0 .
Thus, by the definition of lim sup, we have lim sup an ≤ bk . Consequently by Corollary 2.3.4 (with
n→∞
m := lim sup an ),
n→∞
lim sup an ≤ lim bk = L.
n→∞
k→∞
≥: Let β be an eventual upper bound for (an )n→∞ , so that there exists n0 such that an ≤ β for
all n ≥ n0 . Then for all n ≥ n0 , β is an upper bound for {an , an+1 , an+2 , . . . }. Hence,
β ≥ sup{an , an+1 , . . . } = bn .
Since this holds for all n ≥ n0 , it follows from Corollary 2.3.4 (with M = β) that β is greater than
or equal to the limit of (bn0 , bn0 +1 , . . . ), which is L (by Proposition 2.5.4):
β ≥ L.
Thus, L is a lower bound for the set {β ∈ R : ∃n0 such that an ≤ β ∀n ≥ n0 } used in the definition
of lim sup, so
L ≤ inf{β ∈ R : ∃n0 such that an ≤ β ∀n ≥ n0 } = lim sup an .
n→∞
Remark 2.7.5. In the above proof, if we assume only that (an )∞
n=1 is bounded above, then the bn
are still real numbers, and the same arguments still work (although lim bn = −∞ = lim sup an is
n→∞
(an )∞
n=1
n→∞
possible), and so (2.2) still holds. In the case that
is not bounded above, this means in the
notation of the above proof, bn = ∞ for all n. However, if we agree that lim ∞ = ∞, then under
n→∞
this convention, again holds.
In conclusion, adopting the right conventions, the hypothesis that (an )∞
n=1 is bounded is not needed
in the above proposition.
22
∞
Theorem 2.7.6. Let (an )∞
n=1 be a sequence of real numbers. Then (an )n=1 converges if and only if
lim sup an = lim inf an and this value is finite. In this case,
n→∞
n→∞
lim an = lim sup an = lim inf an .
n→∞
n→∞
n→∞
Proof. First, suppose that the sequence converges, and set L := lim an . Given > 0 there exists n0
n→∞
such that for all n ≥ n0 ,
L − < an < L + .
Hence,
L − ≤ inf{an0 , an0 +1 , . . . }
and so from the formula in Proposition 2.7.4,
L − ≤ lim inf an .
n→∞
Since is arbitrary, we find L ≤ lim inf an . Likewise we may infer that L ≥ lim sup an . Hence,
n→∞
n→∞
L ≤ lim inf an
n→∞
≤ lim sup an
(Proposition 2.7.3)
n→∞
≤ L.
We conclude that
lim an = lim sup an = lim inf an .
n→∞
n→∞
n→∞
This shows one direction of the “if and only if”, as well as the final statement of the theorem.
Conversely, suppose that
lim sup an = lim inf an ,
n→∞
n→∞
and that this value is finite. Set L := lim sup an = lim inf an .
n→∞
n→∞
Let > 0 be given. We may find n0 such that
sup{an0 , an0 +1 , . . . } ∈ (L − , L + )
and
inf{an0 , an0 +1 , . . . } ∈ (L − , L + ).
It follows from the first of these that for all n ≥ n0 ,
L + > sup{an0 , an0 +1 , . . . } ≥ an ,
and likewise, L − < an . Thus, for n ≥ n0 , we have |an − L| < , as required.
Exercise 2.7.1. [TBB08, Exercise 2.13.2] Compute lim sup and lim inf for each of the following sequences.
(a) ((−1)n n)∞
n=1 ,
(b) (sin(nπ/8))∞
n=1 ,
23
(c) (n sin(nπ/8))∞
n=1 ,
∞
(d) (n+1) sin(nπ/8)
,
n
n=1
(e) (1 + (−1)n )∞
n=1 ,
(f) (rn )∞
n=1 , consisting of all the rational numbers in the interval (0, 1), arranged in some order. (Does
the order matter?)
Exercise 2.7.2. [TBB08, Exercise 2.13.3] Find a sequence (an )∞
n=1 of rational numbers such that:
√
√
(a) lim sup an = 2 and lim inf an = − 2.
n→∞
n→∞
√
(b) lim sup an = ∞ and lim inf an = − 2.
n→∞
n→∞
(c) lim sup an = π and lim inf an = e.
n→∞
n→∞
Exercise 2.7.3. [TBB08, Exercise 2.13.7] Let (an )∞
n=1 be a bounded sequence, set L := lim sup an , and
n→∞
let > 0. Prove the following.
(a) an > L + for only finitely many n ∈ N≥1 , and
(b) an > L − for infinitely many n ∈ N≥1 .
∞
Exercise 2.7.4. [TBB08, Exercise 2.13.9] Let (an )∞
n=1 and (bn )n=1 be two bounded sequences.
(a) Prove that lim sup(an + bn ) ≤ lim sup an + lim sup bn .
n→∞
n→∞
n→∞
(b) Give an example where
lim sup(an + bn ) 6= lim sup an + lim sup bn .
n→∞
n→∞
n→∞
Exercise 2.7.5. Suppose that (an )∞
n=1 is a Cauchy sequence. Prove directly (without using the Cauchy
Convergence Criterion, though you are allowed to use that (an )∞
n=1 is bounded) that lim sup an =
n→∞
lim inf an . If we then use Theorem 2.7.6, we thus get an alternative proof of the Cauchy Convergence
n→∞
Criterion (Theorem 2.6.2).
∞
Exercise 2.7.6. Let (an )∞
n=1 be a bounded sequence. Prove that there is a subsequence (ank )k=1 which
converges to lim sup an .
n→∞
24
Chapter 3
Series
This chapter is about infinite sums, also called series. Finite sums make sense in any field; however, to
sum infinitely many elements requires taking a limit, so depends on the theory of sequences developed
in the last chapter.
3.1
Definition and basic properties
A series (of real numbers) is an infinite sum
∞
X
an = a1 + a2 + a3 + · · ·
n=1
where (an )∞
n=1 is a sequence of real numbers.
Since addition in R is (a priori) only defined for finitely many numbers at a time1 we need to
define what an infinite sum means for us.
Definition 3.1.1. Let (an )∞
n=1 be a sequence of real numbers. For N ∈ N≥1 , define
sN :=
N
X
an = a1 + · · · + aN ,
n=1
called the N th partial sum of the series
∞
P
an . We say that the series
n=1
∞
P
an converges (to L) if the
n=1
sequence (sN )∞
N =1 converges (to L), and in this case, we may also write
∞
X
an = L.
n=1
If a series does not converge, we say it diverges. If (sN )∞
n=1 diverges to ±∞, we write
∞
X
an = ±∞.
n=1
1
In fact, recall that we only defined addition for two numbers at a time. But by iterating we can of course do
N
P
finitely many numbers:
an = a1 + a2 + · · · + an formally means (· · · ((a1 + a2 ) + a3 ) + · · · + an−1 ) + an .
n=1
25
In other words,
∞
X
an := lim
N
X
N →∞
n=1
an ,
n=1
provided that the limit on the right exists.
We use theory from the previous topic (sequences) to establish results about series.
∞
Proposition 3.1.2. Let (an )∞
n=1 and (bn )n=1 be sequences and let c ∈ R. Suppose that the series
∞
X
an
∞
X
and
n=1
bn
n=1
both converge.
∞
P
(i)
(an + bn ) converges, and
n=1
∞
X
(an + bn ) =
n=1
(ii)
∞
P
∞
X
an +
n=1
∞
X
bn
n=1
can converges, and
n=1
∞
X
∞
X
can = c
an .
n=1
n=1
Proof. Exercise 3.1.1.
The above says that the set
V :=
{(an )∞
n=1
:
∞
X
an converges}
n=1
is a vector space and the function V → R defined by (an )∞
n=1 7→
∞
P
an is a linear map.
n=1
∞
Proposition 3.1.3. Let (an )∞
n=1 and (bn )n=1 be sequences such that an ≤ bn for all n. If the series
∞
X
an
and
n=1
both converge, then
∞
X
∞
X
bn
n=1
an ≤
n=1
∞
X
bn .
n=1
Proof. Exercise 3.1.2.
Often, as in the following proposition, we start an infinite sum at a different index than 1. This
is no problem: we can rewrite such a series as one starting at 1, for example as
∞
X
an =
n=m
∞
X
bn
n=1
where bn = am+n−1 . (This is needed to make sense of such an infinite sum from a theoretical
perspective, but in practice, when it is clear what is going on, we don’t need to do this.)
26
Proposition 3.1.4. Let (an )∞
n=1 be a sequence of real numbers and let m ∈ N≥1 . Then
∞
X
an
n=1
converges if and only if
∞
X
an
n=m
converges.
Proof. Exercise 3.1.3.
Example 3.1.5. Consider the series
∞
X
n=1
1
.
n(n + 1)
Note that
1
1
1
= −
.
n(n + 1)
n n+1
Therefore, we may simplify the partial sum
1 1
1 1
1
1
−
+
−
+ ··· +
−
sN =
1 2
2 3
N
N +1
1
=1−
,
N +1
by cancelling terms in the middle. Therefore,
∞
X
n=1
1
1
= lim 1 −
= 1.
n(n + 1) N →∞
N +1
A series in which the partial sums simplify in this way is called a telescoping series.
Typically, series are not telescoping, so we want more systematic techniques for determining
whether a series converges.
Example 3.1.6. The harmonic series is
∞
X
1
.
n
n=1
Consider the partial sum
s2k = 1 +
1
1
+ ··· + k
2
2
each term ≥1/4
=1+
1
+
2
z }| {
1 1
+
3 4
each term ≥1/8
each term ≥1/2k
z
}|
{
z
}|
{
1
1
1
1
+ + · · · + + · · · + k−1
+ ··· + k
5
8
2
+1
2
We have 2 terms which are ≥ 14 , so their sum is at least 12 . Following this, we have 4 terms which are
≥ 18 , so their sum is at least 12 . Continuing, we can see that each grouping sums to at least 21 . There
are k − 1 such groupings, so we have
s2k ≥ 1 +
1
1
k
+ (k − 1) = 1 + .
2
2
2
27
∞
It follows that (sN )∞
N =1 is not bounded above, so it diverges. In fact, the sequence (sN )N =1 is
increasing, so we see that sN → ∞ and thus
∞
X
1
= ∞.
n
n=1
The previous example is very important, since n1 → 0. Intuitively, one might have expected that
∞
P
a series
an converges if and only if its terms converge to 0; while the harmonic series shows that
n=1
one direction of this is not true, the intuition is correct in the other direction, as the next result
shows.
Proposition 3.1.7. Let (an )∞
n=1 be a sequence of real numbers. If the series
∞
X
an
n=1
converges, then
lim an = 0.
n→∞
Proof. Set L :=
∞
P
an . Consider the partial sum
n=1
sN =
N
X
an ,
n=1
so that lim sN = L. But then we also have lim sN −1 = L, so
N →∞
N →∞
0 = L − L = lim (sN − sN −1 ) = lim aN .
N →∞
N →∞
It was mentioned before the above proposition, but bears repeating: the converse is not true. If
∞
P
a sequence (an )∞
an converges.
n=1 converges to 0, it does not follow that the series
n=1
By taking the contrapositive of Proposition 3.1.7, we have a useful test for divergence of a series:
The Divergence Test. If a sequence (an )∞
n=1 does not converge to 0, then the series
∞
X
an
n=1
does not converge.
Example 3.1.8. A geometric series is one of the form
1 + r + r2 + r3 + · · ·
where r ∈ R. When |r| ≥ 1, the sequence (rn )∞
n=0 does not converge to 0, so by the Divergence Test,
it follows that the corresponding geometric series diverges.
28
On the other hand, for |r| < 1, the sequence (rn )∞
n=0 does converge to 0, so the corresponding
geometric series at least has a chance to converge. We rewrite the partial sum
sN = 1 + r + r 2 + · · · + r N =
1 − rN +1
.
1−r
From this we see that the series does converge,
∞
X
1 − rN +1
1
=
.
N →∞
1−r
1−r
rn = lim
n=0
Exercise 3.1.1. Prove Proposition 3.1.2.
Exercise 3.1.2. Prove Proposition 3.1.3.
Exercise 3.1.3. Prove Proposition 3.1.4. When they converge, what is the relationship between
∞
P
and
∞
P
an
n=1
an ?
n=m
∞
P
Exercise 3.1.4. [TBB08, Exercise 3.4.3] If
(an + bn ) converges, what can you say about the series
n=1
∞
X
an
∞
X
and
∞
P
Exercise 3.1.5. [TBB08, Exercise 3.4.4] If
bn ?
n=1
n=1
(an + bn ) diverges, what can you say about the series
n=1
∞
X
an
∞
X
and
bn ?
n=1
n=1
Exercise 3.1.6. [Leb16, Exercise 2.5.4] Let (an )∞
n=1 be a sequence of real numbers.
∞
P
(a) Prove that if
an converges, then so does
n=1
∞
P
(a2n−1 + a2n ).
n=1
(b) Give an example where the converse of (a) doesn’t hold, that is, such that
converges but
∞
P
∞
P
(a2n−1 + a2n )
n=1
an does not.
n=1
This exercise shows that we must be cautious when we write a series
∞
P
an as
n=1
a1 + a2 + · · · .
3.2
Convergence tests
Here we will establish a number of results which are useful for proving convergence of series in various
different settings.
Proposition 3.2.1 (Boundedness Test). Let (an )∞
n=1 be a sequence of real numbers. Suppose that:
29
(i) an ≥ 0 for all n, and
(ii) There is a bound M ∈ R on the partial sums, so that
N
X
an ≤ M
n=1
for all N ∈ N≥1 .
Then
∞
P
an converges.
n=1
Proof. Since an ≥ 0, the partial sums (sN )∞
N =1 satisfy
sN ≤ sN +1
for all N.
In other words, (sN )∞
N =1 is an increasing sequence. The second condition ensures that this sequence
is bounded above, Therefore, by the Monotone Convergence Criterion (Theorem 2.4.2), it converges.
∞
P
Hence,
an converges.
n=1
∞
Proposition 3.2.2 (Comparison Test). Let (an )∞
n=1 and (bn )n=1 be sequences such that:
0 ≤ an ≤ b n
for all n.
Then:
(i) if
∞
P
bn converges, then so does
n=1
(ii) if
∞
P
∞
P
an .
n=1
an diverges, then so does
n=1
∞
P
bn .
n=1
Warning: it is easy to get confused about the hypothesis of the Comparison Test. If 0 ≤ an ≤ bn ,
∞
∞
∞
P
P
P
and
an converges, then we cannot conclude anything about
bn . Likewise, if
bn diverges, we
n=1
cannot conclude anything about
∞
P
n=1
n=1
an .
n=1
Proof. (ii) is the contrapositive of (i). We will prove (i) now, using the Boundedness Test. Hypothesis
(i) of the Boundedness Test is true since it is a hypothesis here.
∞
P
Set M :=
bn . Since the sequence
n=1
N
X
!∞
bn
n=1
n=1
is increasing and converges to M , we have that M is the supremum of this sequence, and in particular,
N
X
bn ≤ M
n=1
30
for all N . Therefore,
N
X
an ≤
n=1
N
X
bn ≤ M.
n=1
This verifies hypothesis (ii) of the Boundedness Test, so by that test we conclude that
∞
P
an converges.
n=1
Proposition 3.2.3 (Absolute Convergence Test). Let (an )∞
n=1 be a sequence of real numbers. If the
series
∞
X
|an |
n=1
converges, then so does the series
∞
X
an .
n=1
Proof. Write
(an )+ := max{an , 0} and
(an )− := max{−an , 0}.
Then 0 ≤ (an )+ ≤ |an |, so by the Comparison Test (i) (with (an )+ in place of an and |an | in place of
∞
∞
P
P
bn ),
(an )+ converges. By the same argument, the series
(an )− converges. Finally, we observe
n=1
n=1
that an = (an )+ − (an )− , so by linearity,
∞
X
∞
∞
X
X
an =
(an )+ −
(an )−
n=1
n=1
n=1
converges.
Interesting fact 3.2.4. One calls a series
∞
P
an absolutely convergent when the series
n=1
It is nontrivial result (by Dirichlet and Riemann) that a series
∞
P
∞
P
|an | converges.
n=1
an is absolutely convergent if and
n=1
only if any rearrangement of it converges to the same value, i.e.,
∞
X
as(n) =
n=1
∞
X
an
n=1
for any bijection s : N≥1 → N≥1 .
As an illustration of this, consider the alternating harmonic series
1−
1 1
+ − ··· .
2 3
This series converges to some value L ≥ 12 (by the Alternating Series Test, Proposition 3.2.9 below;
the estimate 12 comes by looking at the second partial sum). If rearrangements were allowed, then
31
we could do the following
1 1 1 1 1
+ − + − + ···
2 3 4 5 6
1 1 1 1 1
1
1
1
= 1 − − + − − + ··· +
−
−
+ ···
2k + 1 4k + 2 4k + 3
2 4 3 6 7 1
1
1 1
1
= 1−
− +
−
− + ···
2
4
3 6
8
1 1 1 1
= − + − + ···
2 4 6 8
1
1 1
=
1 − + − ···
2
2 3
L
= .
2
Since L 6= 0, this is a contradiction. This shows that rearranging the alternating harmonic series can
change its value, confirming a special case of the Dirichlet–Riemann result, since we already know
that the alternating harmonic series does not converge absolutely, by Example 3.1.6.
L=1−
Proposition 3.2.5 (Ratio Test). Let (an )∞
n=1 be a sequence of nonzero real numbers.
(i) If
lim sup
n→∞
then
∞
P
an+1
<1
an
an converges (absolutely).
n=1
(ii) If
lim inf
an+1
>1
an
q := lim sup
an+1
< 1.
an
n→∞
then
∞
P
an diverges.
n=1
Proof. (i): Suppose
n→∞
We wish to show that
∞
P
|an | converges, so we will replace an with |an | and show that
n=1
∞
P
an converges.
n=1
This allows us to assume that an ≥ 0 for all n.
Pick r ∈ (q, 1). Then by the definition of lim sup, r is an eventual upper bound for
there exists n0 such that
an+1
≤r
an
for all n ≥ n0 , which we rearrange as
an+1 ≤ an r.
Thus we have,
an0 +1 ≤ an0 r,
an0 +2 ≤ an0 +1 r ≤ an0 r2 ,
32
an+1
.
an
That is,
and continuing in this way, an0 +k ≤ an0 rk for all k ∈ N≥0 . Since r ∈ (0, 1), the geometric series
converges (Example 3.1.8). Hence by the comparison test,
∞
P
∞
P
rk
k=0
an0 +k converges, which is the same as
k=0
saying
∞
X
an converges.
n=n0
By Proposition 3.1.4, it follows that
∞
P
an converges.
n=1
(ii): Suppose
q := lim inf
n→∞
an+1
> 1.
an
Then by the definition of lim inf, since 1 < q, 1 is an eventual lower bound for
n0 such that
|an+1 |
≥1
|an |
for all n ≥ n0 , which we rearrange as
|an+1 | ≥ |an |.
|an+1 |
,
|an |
so there exists
Similarly to what we did in part (i), from this we get |an+k | ≥ |an0 | for all k ∈ N≥0 . We conclude
∞
P
that (an )∞
doesn’t
converge
to
0,
so
by
the
Divergence
Test,
an diverges.
n=1
n=1
Remark 3.2.6. In many examples of series you will see, the limit lim
n→∞
an+1
an
will exist (so will be
equal to the lim inf and the lim sup). However, it often happens that
lim
n→∞
an+1
= 1,
an
and in this case, we cannot conclude anything from the Ratio Test. To see why, recall the two series
from Examples 3.1.5 and 3.1.6:
∞
X
n=1
1
= 1,
n(n + 1)
∞
X
1
= ∞.
n
n=1
However, in both cases, the ratio an+1
converges to 1.
an
an+1
For an example where lim sup an > 1 > lim inf an+1
, define
an
n→∞
n→∞
(
1,
an :=
2,
Then
an+1
=
an
Of course in this case,
∞
P
n even;
n odd.
(
2,
1
,
2
n even;
n odd.
an diverges (by the Divergence Test). See also Exercise 3.2.8.
n=1
33
Proposition 3.2.7 (Root Test). Let (an )∞
n=1 be a sequence of real numbers.
(i) If
lim sup
p
n
|an | < 1
p
n
|an | > 1
n→∞
then
∞
P
an converges (absolutely).
n=1
(ii) If
lim sup
n→∞
then
∞
P
an diverges.
n=1
Proof. This proof is similar to the proof of the Ratio Test.
(i): Suppose
p
q := lim sup n |an | < 1.
n→∞
∞
P
We wish to show that
|an | converges, so we replace an with |an | and we’ll show that
n=1
converges.
Pick r ∈ (q, 1), and using the definition of lim sup, there exists n0 such that
√
n
an ≤ r for all n ≥ n0 .
∞
P
an
n=1
In other words,
0 ≤ an ≤ r n
∞
P
for all n ≥ n0 . Since the geometric series
rn converges, we conclude by the Comparison Test
that so does
∞
P
n=n0
∞
P
an . Thus by Proposition 3.1.4,
n=n0
an converges.
n=1
(ii) Suppose
q := lim sup
p
n
|an | > 1.
n→∞
Using the definition of lim sup, there are infinitely many n such that
p
n
|an | ≥ 1.
For such n, we have
|an | ≥ 1.
We conclude that (an )∞
n=1 doesn’t converge to 0, so by the Divergence Test,
∞
P
an diverges.
n=1
p
Remark 3.2.8. Although n |an | will often converge in examples you will see, it might converge to 1,
in which case the Root Test tells us nothing. Again this can be see through the examples
∞
X
n=1
1
= 1,
n(n + 1)
∞
X
1
= ∞.
n
n=1
However, in both cases, the
√
n
an converges to 1.
34
Our next test concerns alternating series, which is a series in which the signs of the terms alternate
between positive and negative. We set up an alternating series by taking a sequence (an )∞
n=1 of positive
∞
P
numbers and forming
(−1)n+1 an .
n=1
Proposition 3.2.9 (Alternating Series Test). Let (an )∞
n=1 be a sequence of real numbers. Suppose
that:
(i) (an )∞
n=1 is an decreasing sequence, and
(ii) lim an = 0.
n→∞
Then
∞
X
(−1)n+1 an
n=1
converges. Moreover, for any N ,
2N
X
(−1)n+1 an ≤
2N
−1
∞
X
X
(−1)n+1 an .
(−1)n+1 an ≤
n=1
n=1
n=1
Proof. Note that an ≥ 0 for all n, because the sequence is decreasing and converges to 0.
Set
sN :=
N
X
(−1)n+1 an .
n=1
Since
(an )∞
n=1
is a decreasing sequence, we have
sN +2 = sN + aN +1 − aN +2 ≥ sN
sN +2 = sN − aN +1 + aN +2 ≤ sN
if N is even.
if N is odd.
Thus, the subsequence of even terms, (s2N )∞
N =1 is an increasing sequence, and the subsequence of
odd terms, (s2N −1 )∞
is
decreasing.
To
show
that the sequence of even terms (s2N )∞
N =1
N =1 is bounded,
observe that for each N , since a2N +1 ≥ 0,
s2N ≤ s2N + a2N +1 = s2N +1 ≤ s1 .
So, s1 is an upper bound for (s2N )∞
N =1 . By the Monotone Convergence Criterion (Theorem 2.4.2),
∞
(s2N )N =1 converges to L := sup{s2N : N ∈ N≥1 }.
Next, for the subsequence of odd terms, since s2N = s2N −1 − a2N , we get
lim s2N −1 = lim s2N + a2N = lim s2N + lim a2N = L + 0.
N →∞
N →∞
N →∞
35
N →∞
By Exercise 2.2.4, since both the even and odd subsequences converge to the same value, we conclude
that
lim sN = L,
N →∞
i.e.,
∞
P
(−1)n+1 an converges and equals L.
n=1
Finally, since L is the supremum of {s2N : N ∈ N≥1 }, we have
∞
X
n+1
(−1)
an = L ≥ s2N =
n=1
2N
X
(−1)n+1 an .
n=1
By a similar argument as above, we can show that L = inf{s2N −1 : N ∈ N≥1 }, and from this,
∞
2N
−1
X
X
n+1
(−1) an ≤
(−1)n+1 an .
n=1
n=1
In the Alternating Series Test, it is crucial that the sequence (an )∞
n=1 is decreasing – see Exercise
3.2.3.
The last test that we will state uses the integral. Since we haven’t formally defined the integral,
we will not be able to prove this result yet; we will prove it later (Exercise 7.4.4 when we study
integration. However, it is a very powerful test, particularly as one can often use the Fundamental
Theorem of Calculus to easily analyse convergence of an integral.
Proposition 3.2.10 (Integral Test). Let f : [1, ∞) → R be a function. Suppose that:
(i) f (x) ≥ 0 for all x ∈ [1, ∞), and
(ii) f is decreasing: f (x) ≥ f (y) whenever x ≤ y.
Then the series
∞
X
f (n)
n=1
converges if and only if the improper integral
Z
∞
f (x) dx
1
converges.
Exercise 3.2.1. For each of the following, determine whether the series converges.
(a)
∞
P
n=1
(b)
∞
P
n=1
(c)
∞
P
n=1
n
.
2n
n2 −3
.
(n+2)(n+5)
(−1)n n2 −3
.
(n+2)(n+5)
36
∞
P
(d)
n=1
∞
P
(e)
n=1
∞
P
(f)
n=1
∞
P
(g)
n=1
∞
P
(h)
n=1
(n2 +3)n
.
(2n2 −1)n
sin(n)
.
n2 +n
n2 +2
.
n4 +4
n3 +3
.
n6 +6
n!
.
2n
Exercise 3.2.2. [Leb16, Exercise 2.5.7] (Difficult, and this exercise probably belongs better in Sec∞
P
is
a
decreasing
sequence.
If
the
series
an converges, prove that
tion 3.1) Suppose that (an )∞
n=1
n=1
lim nan = 0.
n→∞
Exercise 3.2.3. Give an example of a sequence (an )∞
n=1 such that an ≥ 0 for all n and lim an = 0,
n→∞
such that the alternating series
∞
X
(−1)n+1 an
n=1
does not converge.
Exercise 3.2.4. [TBB08, Exercise 3.5.1] Suppose that (an )∞
n=1 is a sequence of real numbers such that
∞
∞
P
P
an ≥ 0 for all n. If we know that
an converges, does it follow that
a2n converges?
n=1
n=1
∞
(bn )n=1
(an )∞
n=1
are sequences of real numand
Exercise 3.2.5. [TBB08, Exercise 3.4.7] Suppose that
∞
∞
∞
P
P
P
bers such that
an and
bn converge. Does it follow that
an bn converges? What about if we
n=1
n=1
n=1
assume in addition that an ≥ 0 for all n?
∞
Exercise 3.2.6. Suppose that (an )∞
n=1 and (bn )n=1 are sequences of real numbers such that:
(i) bn > 0 for all n,
(ii)
∞
P
bn converges, and
n=1
(iii) The sequence
Prove that
∞
P
an
bn
∞
converges.
n=1
an converges.
n=1
Exercise 3.2.7. [TBB08, Exercise 3.5.11] Show that a series
only if every “subseries”
∞
P
∞
P
an is absolutely convergent if and
n=1
ank converges.
k=1
Exercise 3.2.8. Produce an example of a sequence (an )∞
n=1 of numbers in (0, ∞) such that
an+1
an+1
lim sup
> 1, lim inf
< 1,
n→∞
an
an
n→∞
∞
P
and the series
an converges. Can you even arrange that lim sup an+1
= ∞?
an
n→∞
n=1
37
3.3
Cauchy convergence criterion for series
The following is an important theoretical device, although for examples of sequences you may encounter, it is rarely the right tool to prove convergence.
Proposition 3.3.1 (Cauchy Convergence Criterion). Let (an )∞
n=1 be a sequence of real numbers.
∞
P
Then
converges if and only if, for every > 0 there exists N0 such that, for all N ≥ M ≥ N0 ,
n=1
N
X
an < .
n=M
Proof. Again we use the partial sums
sN :=
N
X
an .
n=1
By the Cauchy Convergence Criterion for sequences (Theorem 2.6.2), (sN )∞
N =1 converges (i.e.,
an
n=1
converges) if and only if (sN )∞
N =1 is Cauchy. Since
sN − sM −1 =
∞
P
N
X
an ,
n=M
it is easy to see that (sN )∞
N =1 is Cauchy if and only if, for every > 0 there exists N0 such that for
all N ≥ M ≥ N0 ,
N
X
an < .
n=M
∞
∞
Exercise 3.3.1. Let (an )∞
n=1 , (bn )n=1 , and (cn )n=1 are sequences such that for all n,
an ≤ b n ≤ c n .
Prove that if
∞
P
n=1
an and
∞
P
cn converge, then so does
n=1
∞
P
n=1
Theorem (Theorem 2.3.5)?
38
bn . Can this be proven using the Squeeze
Chapter 4
Topology of Rd
In this chapter we will look at Euclidean space Rd , enabling us to consider sequences of points in this
space (and, correspondingly, limits of such sequences).
4.1
Norms
The set Rd is a vector space over R: it is equipped with addition,
(a1 , . . . , ad ) + (b1 , . . . , bd ) := (a1 + b1 , . . . , an + bn ),
and scalar multiplication,
c · (a1 , . . . , ad ) := (ca1 , . . . , cad )
(here, (a1 , . . . , ad ), (b1 , . . . , bd ) ∈ Rd and c ∈ R).
A norm encapsulates a way of measuring distances in vector spaces.
Definition 4.1.1. A norm on Rd is a function k · k : Rd → [0, ∞) (written a 7→ kak) satisfying the
following, for a, b ∈ Rd and c ∈ R:
(i) kak = 0 if and only if a = (0, . . . , 0);
(positive definiteness)
(ii) kcak = |c| · kak;
(homogeneity)
(iii) ka + bk ≤ kak + kbk.
(triangle inequality)
For us, the most important norm is the Euclidean norm,
q
k(a1 , . . . , ad )k2 := a21 + · · · + a2d .
Proving properties (i) and (ii) of a norm for k · k2 is a straightforward exercise (Exercise 4.1.1). The
third property takes a bit more work. For this, it is useful to observe that the norm is given by
√
kak2 = a · a,
where we employ the dot product:
(a1 , . . . , ad ) · (b1 , . . . , bd ) := a1 b1 + · · · + ad bd .
Proposition 4.1.2. Let a, b ∈ Rd and let k · k denote the Euclidean norm.
39
(i) (Cauchy–Schwarz Inequality) |a · b| ≤ kak2 · kbk2 .
(ii) (Triangle Inequality) ka + bk2 ≤ kak2 + kbk2 .
(iii) Therefore, k · k2 is a norm on Rd .
Proof. (i): This is clearly true if b = 0, so assume otherwise. Consider
p(t) :=
=
=
=
ka + tbk22
(a + tb) · (a + tb)
a · a + 2t(a · b) + t2 (b · b)
kak22 + 2(a · b)t + kbk22 t2
This is a quadratic polynomial which is always ≥ 0 (evident from the first line), and therefore has at
most one root. Hence its discriminant must be ≤ 0, that is,
2
0 ≥ 2(a · b) − 4kak22 kbk22 .
Rearranging this yields
|a · b|2 ≤ kak22 kbk22 ,
and then taking square roots yields the desired inequality.
(ii): Using the same expansion (with t = 1) we have
ka + bk22 = kak22 + 2a · b + kbk22
≤ kak22 + 2|a · b| + kbk22
≤ kak22 + 2kak2 · kbk2 + kbk22 t2
= (kak2 + kbk2 )2 ,
(Cauchy–Schwarz)
and then taking square roots yields the desired inequality.
(iii): We just proved the triangle inequality, while Exercise 4.1.1 is to show conditions (i) and
(ii).
We will mostly use the Euclidean norm when working with Rd , and will often write k · k for k · k2 ;
however, the next two most important norms are the l1 and l∞ norms, defined as follows.
Example 4.1.3. The l1 -norm on Rd is given by
k(a1 , . . . , ad )k1 := |a1 | + · · · + |ad |.
The l∞ -norm on Rd is given by
k(a1 , . . . , ad )k∞ := max{|a1 |, . . . , |ad |}.
These are norms (Exercise 4.1.2).
Interesting fact 4.1.4. In fact, there are numerous other norms that one can put on Rd ; however, in
a sense that is very relevant to this course, they are all equivalent. To be precise, for any two norms
k · k and k · k0 on Rd , there exist α, β > 0 such that, for all x ∈ Rd ,
αkxk ≤ kxk0 ≤ βkxk.
We won’t prove this in this course. However, when using two norms out of k · k1 , k · k2 , and k · k∞ ,
we can find α and β explicitly: for all a ∈ Rd ,
kak∞ ≤ kak2 ≤ kak1 ≤ dkak∞
(Exercise 4.1.3).
40
Exercise 4.1.1. Show that the Euclidean norm k · k2 satisfies conditions (i) and (ii) of a norm.
Exercise 4.1.2. Show that k · k1 and k · k∞ are norms.
Exercise 4.1.3. Prove that for all a ∈ Rd ,
kak∞ ≤ kak2 ≤ kak1 ≤ dkak∞ .
Exercise 4.1.4. [Sav17, Exercise 4.1.5] Let k · k and k · k0 be norms on Rd . Define k · k00 : Rd → [0, ∞)
by
kak00 := kak + kak0 .
Prove that k · k00 is a norm.
4.2
Convergence in Rd
We now make use of a norm on Rd to define convergence of a sequence in Rd .
d
d
d
Definition 4.2.1. Let (an )∞
n=1 be a sequence in R (that is, an ∈ R for each n), and let L ∈ R .
We say that (an )∞
n=1 converges to L if lim kan − Lk2 = 0. In this case, we may also write
n→∞
lim an = L
n→∞
or an → L as n → ∞.
We have used a particular norm, k · k2 , in the above definition; however, the next proposition says
that we could have used any other norm and arrived at the same concept.
d
d
0
d
Proposition 4.2.2. Let (an )∞
n=1 be a sequence in R and let L ∈ R . Let k · k be a norm on R .
Then an → L if and only if
lim kan − Lk0 = 0.
n→∞
Proof. Exercise 4.2.1 (use Interesting Fact 4.1.4).
When working with sequences in Rd , we sometimes want to look at the components of each term
of the sequence. This gets notationally a bit messy, and we just have to bear with it. If (an )∞
n=1 is a
sequence in Rd , we write
(d)
an = (a(1)
n , . . . , an ) for each n ∈ N≥1 .
Note that we then get d different sequences of real numbers, one for each component:
∞
(d) ∞
a(1)
n n=1 , . . . , an n=1 .
Through these sequences, we can relate convergence in Rd to convergence in R.
d
Proposition 4.2.3. Let (an )∞
n=1 be a sequence in R , with
(d)
an = (a(1)
n , . . . , an )
for each n ∈ N≥1 ,
and let L = (L1 , . . . , Ld ) ∈ Rd . Then
lim an = L (in Rd )
n→∞
if and only if, for each i = 1, . . . , d,
lim a(i)
n = Li
n→∞
41
(in R).
(i)
Proof. ⇒: We assume that lim an = L. Fix i ∈ {1, . . . , d}, and we must show that lim an = Li .
n→∞
Note that from the formula for k · k2 , for any x = (x1 , . . . , xd ) ∈ Rd , we have |xi |2 ≤
n→∞
d
P
x2i ≤ kxk22 ,
i=1
and thus |xi | ≤ kxk2 . Therefore,
−kan − Lk2 ≤ an(i) − Li ≤ kan − Lk2 .
(i)
(i)
By the Squeeze Theorem, it follows that lim an − Li = 0, which is the same as lim an = Li .
n→∞
n→∞
(i)
⇐: We assume that lim an = Li for all i = 1, . . . , d. Then we have
n→∞
2
2
(d)
lim kan − Lk22 = lim (a(1)
n − L1 ) + · · · + (an − Ld )
n→∞
n→∞
= 0.
It follows from Exercise 2.2.5 that kan − Lk2 → 0, as required.
Remark 4.2.4. Since limits are unique in R, a consequence of the previous proposition is that limits
are also unique in Rd .
d
Definition 4.2.5. A sequence (an )∞
n=1 in R is Cauchy if for every > 0 there exists n0 such that
for all m, n ≥ n0 ,
kam − an k2 < .
d
Theorem 4.2.6 (Cauchy Convergence Criterion for Rd ). Let (an )∞
n=1 be a sequence in R . Then it
converges if and only if it is Cauchy.
Proof. ⇒: This proof is almost the same as the argument for sequences in R. Assume that (an )∞
n=1
converges to some L ∈ Rd . Let > 0 be given. There exists n0 such that for all n ≥ n0 ,
kan − Lk2 < .
2
Now, if m, n ≥ n0 then
kam − an k2 ≤ kam − Lk2 + kL − an k2
< + = .
2 2
(∆-inequality)
Hence (an )∞
n=1 is Cauchy.
(1)
(d)
∞
⇐:
Suppose that (an )n=1 is Cauchy. Write an = (an , . . . , an ). Let us show that each sequence
(i)
an
∞
n=1
is Cauchy (for i = 1, . . . , d). Fix i and let > 0 be given. Then since (an )∞
n=1 is Cauchy,
there exists n0 such that for all m, n ≥ n0 , kam − an k2 < . Then for m, n ≥ n0 , we have
(i)
|a(i)
m − an | ≤ kam − an k2 < ,
as required. Thus
(i)
an
∞
is Cauchy, and so it converges by the Cauchy Convergence Criterion for
n=1
R (Theorem 2.6.2); we let Li ∈ R be its limit. Consequently by Proposition 4.2.3, (an )∞
n=1 converges
to (L1 , . . . , Ld ).
Definition 4.2.7. A subset S of Rd is bounded if there exists some M > 0 such that for all x ∈ S,
kxk2 ≤ M.
d
A sequence (an )∞
n=1 in R is bounded if the set of its terms, {an : n ∈ N≥1 } is bounded.
42
Note that in Rd , we do not have a concept of bounded above or below, since there is no ordering.
Next, we generalize the Bolzano–Weierstrass Theorem to Rd . We define a subsequence of a
sequence in Rd in exactly the same way as for a sequence in R.
d
Theorem 4.2.8 (Bolzano–Weierstrass Theorem for Rd ). Let (an )∞
n=1 be a bounded sequence in R .
Then it has a subsequence which converges.
Proof. We prove this by induction in d. The base case is d = 1, and in this case it just the Bolzano–
Weierstrass Theorem for R (Corollary 2.5.6).
d
We now assume that it holds for Rd−1 . Let (an )∞
n=1 be a bounded sequence in R . For each n,
(1)
(d)
write an = (an , . . . , an ) and define
(d−1)
bn := (a(1)
) ∈ Rd−1 .
n , . . . , an
∞
Since kbn k2 ≤ kan k2 , the sequence (bn )∞
n=1 is bounded. By the inductive hypothesis, (bn )n=1 has a
d−1
subsequence
(bnk )∞
k=1 converging to some (L
1 , . .. , Ld−1 ) ∈ R . Next, we consider the subsequence
(d)
∞
(d)
ank
∞
of the last-component sequence an
. This subsequence is also bounded, so it has a
k=1
n=1
∞
(d)
which converges to some Ld ∈ R.
subsubsequence, ankl
l=1
∞
Now, (ankl )∞
l=1 is a subsequence of (an )n=1 . We are done once we prove the following claim.
Claim. (ankl )∞
l=1 converges to (L1 , . . . , Ld ).
(i)
To prove the claim, by Proposition 4.2.3, it is equivalent to show that (ankl )∞
l=1 converges to Li
for each i = 1, . . . , d.
For i = d, we already know this converges. For i < d, since (bnk )∞
k=1 converges to (L1 , . . . , Ld−1 ),
(i)
we know from Proposition 4.2.3 that lim ank = Li . Then by Proposition 2.5.4, it follows that
k→∞
lim
l→∞
(i)
ankl
= Li as well.
Exercise 4.2.1. Prove Proposition 4.2.2 (using Interesting Fact 4.1.4).
d
∞
Exercise 4.2.2. Let (an )∞
n=1 and (bn )n=1 be sequences in R that converge, and let c ∈ R.
(a) Prove that lim (an + bn ) = lim an + lim bn .
n→∞
(b) Prove that lim can = c
n→∞
n→∞
n→∞
lim an .
n→∞
(c) Prove that if (cn )∞
n=1 is a sequence in R which converges, then lim cn an =
n→∞
lim cn
n→∞
(Here we are using scalar multiplication.)
(d) Prove that lim an · bn = lim an · lim bn . (Here we are using dot-product).
n→∞
4.3
n→∞
n→∞
Open and closed sets
Definition 4.3.1. Let a ∈ Rd and let r > 0. The open ball centred at a with radius r is
B(a; r) := {x ∈ Rd : kx − ak2 < r}.
43
lim an .
n→∞
Example 4.3.2. For d = 1,
B(a; r) = {x ∈ Rd : |x − a| < r} = (a − r, a + r).
In fact, every bounded open interval (a, b) is an open ball:
a+b b−a
(a, b) = B
;
.
2
2
For d = 2, the open ball B(a; r) is an open disc (whose boundary is the circle centred at a with
radius r).
For d = 3, the open ball B(a; r) is interior of a sphere – the name “ball” is most appropriate here.
Definition 4.3.3. Let A ⊆ Rd be a set.
(i) A is open if for every x ∈ A, there exists > 0 such that
B(x; ) ⊆ A.
(ii) A is closed if its complement,
Rd \ A = {x ∈ Rd : x 6∈ A}
is open.
It is not the case that a set is either open or closed. For example, the set (0, 1] is neither
open (since 1 ∈ (0, 1] but there is no open ball containing 1 that is in (0, 1]) nor closed (similar
reasoning with 0 ∈ R \ (0, 1]).
Example 4.3.4. Given a ∈ Rd and r > 0, the open ball B(a; r) is an open set. To see this, let
x ∈ B(a; r), so that kx − ak2 < r. Define
:= r − ka − xk2 > 0.
To show that B(x; ) ⊆ B(a; r), let y ∈ B(x; ), so that ky − xk2 < . Then by the triangle inequality,
ky − ak2 ≤ ky − xk2 + kx − ak < + ka − xk2 = r.
Thus, y ∈ B(a; r), as required.
Proposition 4.3.5.
(i) The sets ∅ and Rd are open.
(ii) For any finite collection of open sets, U1 , . . . , Um ⊆ Rd , their intersection,
U1 ∩ · · · ∩ Um ,
is open.
44
(iii) For any arbitrary collection of open sets {Uα : α ∈ I}, their union,
[
Uα ,
α∈I
is open.
Proof. Parts (i) and (ii) are Exercise 4.3.1.
(iii): Set
U :=
[
Uα ,
α∈I
and let x ∈ U . By definition of union, this means that x ∈ Uα for some α ∈ I. Since Uα is open,
there is some > 0 such that B(x; ) ⊆ Uα . Since Uα ⊆ U , it follows that
B(x; ) ⊆ U,
as required.
Example 4.3.6. Although we have that finite intersections of open sets are open, it is not true that
infinite intersections of open sets are open. For example,
∞ \
1 1
{0} =
− ,
,
n n
n=1
but {0} isn’t an open set because it contains no open ball centred at 0.
Example 4.3.7. For any a ∈ R, the sets (a, ∞) and (−∞, a) are open. To see this, we may write
them as unions of open balls:
(a, ∞) =
∞
[
(a, a + n),
(−∞, a) =
∞
[
(a − n, a).
n=1
n=1
Alternatively, here is a proof that (a, ∞) is open using the definition of an open set directly. Let
x ∈ (a, ∞). Then x > a, and we may set r := x − a > 0. We have
B(x; r) = (x − r, x + r) = (a, x + r) ⊂ (a, ∞),
as required.
Example 4.3.8. For any a, b ∈ R, the closed interval [a, b] is closed, since
R \ [a, b] = (−∞, a) ∪ (b, ∞),
which is open by the previous example and since unions of open sets are open.
Proposition 4.3.9. Let F ⊆ Rd . Then F is closed if and only if, for every sequence (an )∞
n=1 in F
(i.e., an ∈ F for all n), if (an )∞
converges
then
n=1
lim an ∈ F.
n→∞
45
Proof. ⇒: We assume that F is closed, i.e., Rd \ F is open. Let (an )∞
n=1 be a sequence in F which
converges, and suppose for a contradiction that L := lim an is not in F . That means L ∈ Rd \ F .
n→∞
d
By openness, there exists > 0 such that B(L; ) ⊆ R \ F . Using the definition of convergence, we
may find n such that kan − Lk2 < . But this means that
an ∈ B(L; ) ⊆ Rd \ F,
so that an 6∈ F , which is a contradiction.
⇐: We assume that for every sequence in F that converges, the limit is in F . We need to show
that Rd \ F is open.
Take a point x ∈ Rd \ F , and suppose for a contradiction
that there is no > 0 such that
1
d
B(x; ) ⊆ R \ F . Then for each n ∈ N≥1 , since B x; n is not contained in Rd \ F , this means that
there must be some point in this ball and in F . Let an be such a point, so an ∈ F and kan − xk2 < n1 .
1
Choosing such an for each n, we arrive at a sequence (an )∞
n=1 in F . Since 0 ≤ kan − xk2 < n , we
get lim kan − xk2 = 0 by the Squeeze Theorem. In other words,
n→∞
lim an = x.
n→∞
By hypothesis, it follows that x ∈ F , which is a contradiction.
Definition 4.3.10. Let A ⊆ Rd be a set and let a ∈ Rd .
(i) a is an interior point of A if there exists > 0 such that B(a; ) ⊆ A. The interior of A is
A◦ := {x ∈ Rd : x is an interior point}.
(ii) a is an accumulation point of A if there is a sequence (an )∞
n=1 in A such that a = lim an . The
n→∞
closure of A is
A := {x ∈ Rd : x is an accumulation point}.
(iii) a is a boundary point of A if it is an accumulation point of A and it is not an interior point.
The boundary of A is
∂A := {x ∈ Rd : x is a boundary point} = A \ A◦ .
(iv) a is an isolated point of A if there exists > 0 such that B(a; ) ∩ A = {a}.
(v) a is a limit point of A if it is an accumulation point and it is not an isolated point of A.
Example 4.3.11. Let A := (0, 1] ∪ {2}.
• The interior points of A are all points in (0, 1):
A◦ = (0, 1).
• The accumulation points of A are all points in [0, 1] ∪ {2}:
A = [0, 1] ∪ {2}.
• The boundary points of A are 0, 1, and 2:
∂A = {0, 1, 2}.
46
• A has only one isolated point, namely 2.
• The limit points of A are all points in [0, 1].
Example 4.3.12. Let A := Q.
• A◦ = ∅.
• A = ∂A = R.
• A has no isolated points.
• Every point of R is a limit point of A.
Example 4.3.13. Let A := Z.
• A◦ = ∅.
• A = ∂A = Z.
• Every point of A is an isolated point.
• A has no limit points.
Exercise 4.3.1. Prove parts (i)-(ii) of Proposition 4.3.5.
Exercise 4.3.2. Prove the following facts about closed sets.
(a) ∅, Rd are closed sets.
(b) For any finite collection of closed sets, F1 , . . . , Fm ⊆ Rd , their union,
F1 ∪ · · · ∪ Fm ,
is closed.
(c) For any arbitrary collection of closed sets {Fα : α ∈ I}, their intersection,
\
Fα ,
α∈I
is closed.
Exercise 4.3.3. [Sav17, Exercise 4.3.1] Prove that a set A ⊆ Rd is bounded if and only if there exists
a ∈ Rd and r > 0 such that A ⊆ B(a; r).
Exercise 4.3.4. [Sav17, Exercise 4.3.2 (b)] Prove that any finite set is closed.
d
d
Exercise 4.3.5. Let (an )∞
n=1 be a sequence in R which converges to L ∈ R . Prove that the set
{an : n ∈ N≥1 } ∪ {L}
is closed.
Exercise 4.3.6. Let A ⊆ Rd with A 6= ∅ and A 6= Rd . Take points a ∈ A and b ∈ Rd \ A, and for
t ∈ [0, 1], define
γ(t) := (1 − t)a + tb.
γ parametrizes a line from a to b – a line that starts in A and ends out of A.
47
(a) Define
T := {t ∈ [0, 1] : γ(t) ∈ A}.
Show that t0 := sup T exists.
(b) Show that with t0 as in part (a), the point γ(t0 ) is in the boundary of A.
(c) Conclude that the only sets in Rd whose boundary is ∅ are ∅ and Rd .
Exercise 4.3.7. Let A ⊆ Rd .
(a) Prove that the closure, A, is closed.
(b) Prove that the interior, A◦ , is open.
(c) Prove that the boundary, ∂A, is closed.
(d) Prove that A◦ = Rd \ (Rd \ A).
d
d
Exercise 4.3.8. Let (an )∞
n=1 be a sequence in R , and define A := {an : n ∈ N≥1 }. Show that if x ∈ R
is a limit point of A, then there is a subsequence (ank )∞
k=1 which converges to x.
Exercise 4.3.9. [Sav17, Exercise 4.3.13] Let A1 , . . . , Ad ⊆ R be closed sets. Show that
A1 × · · · × Ad = {(a1 , . . . , ad ) : ai ∈ Ai for i = 1, . . . , d} ⊆ Rd
is closed.
Exercise 4.3.10. Let A ⊆ Rd and let a ∈ Rd . Prove that the following are equivalent.
(i) a is a limit point of A.
(ii) For every > 0,
B(a; A) ∩ A \ {a} =
6 ∅.
(iii) There exists a sequence (an )∞
n=1 in A \ {a} which converges to a.
Exercise 4.3.11. Construct a set A ⊆ R which has some limit points, but such that none of its limit
points are in A itself.
4.4
Compactness
Definition 4.4.1. Let K ⊆ Rd be a set. We say that K is (sequentially) compact if every sequence
(an )∞
n=1 in K has a subsequence that converges to a point a ∈ K.
In this course, we will frequently abbreviate “sequential compactness” to just “compactness”.
However, there is a different concept which is called “compactness”, which is equivalent to sequential
compactness for subsets of Rd , though the two aren’t equivalent in some more general settings. This
is discussed further in Interesting Fact 4.4.7.
Example 4.4.2. Let F be a finite set,
F = {x1 , . . . , xk }.
Then F is compact. To see this, let (an )∞
n=1 be a sequence in K. Then there must be some i such
that an = xi for infinitely many n. Consequently, we can realize the constant sequence
(xi , xi , . . . )
as a subsequence (ank )∞
k=1 .
48
Example 4.4.3. The set R is not compact. To see this, take the sequence (1, 2, 3, 4, . . . ). This sequence
diverges to ∞, and it is not hard to see that any subsequence of it also diverges to ∞. Hence it has
no subsequence which converges.
∞
Example 4.4.4. The set A := (0, 1] is not compact. To see this, take the sequence n1 n=1 in A. This
sequence converges to 0, and hence so does every subsequence of it (Proposition 2.5.4). Hence, it has
no subsequence converging to a point in A.
Theorem 4.4.5 (Heine–Borel Theorem). Let K be a subset of Rd . Then K is compact if and only
if K is closed and bounded.
Proof. ⇒: Suppose that K is compact. To see that K is closed, suppose for a contradiction that
it is not closed. By Proposition 4.3.9, it follows that there is some sequence (an )∞
n=1 in K which
converges, yet such that
lim an 6∈ K.
n→∞
Then by Proposition 2.5.4 (generalized to Rd ), every subsequence of (an )∞
n=1 converges to the same
point, and hence no subsequence converges to a point in K. This contradicts that K is compact.
Next, to see that K is bounded, again suppose for a contradiction that it is not. Then for each
n ∈ N≥1 , we may find some an ∈ K such that kan k2 ≥ n. This gives us a sequence (an )∞
n=1 . Any
∞
subsequence (ank )k=1 will be unbounded, and therefore won’t converge. Again, this contradicts that
K is compact.
⇐: We assume that K is closed and bounded, and we let (an )∞
n=1 be a sequence from K. Since
∞
K is bounded, so is the sequence (an )n=1 . Therefore by the Bolzano–Weierstrass Theorem (Theorem
d
4.2.8), there is a subsequence (ank )∞
k=1 which converges to some point L ∈ R . Since K is closed and
∞
the sequence (ank )∞
k=1 is in K, it follows from Proposition 4.3.9 that L ∈ K. Thus (an )n=1 has a
subsequence converging to a point in K, as required.
Proposition 4.4.6.
(i) For any finite collection of compact sets, K1 , . . . , Km ⊆ Rd , their union,
K1 ∪ · · · ∪ Km ,
is compact.
(ii) For any arbitrary collection of compact sets {Kα : α ∈ I}, their intersection,
\
Kα ,
α∈I
is compact.
Proof. Exercise 4.4.1
Interesting fact 4.4.7. The correct definition of compactness is as follows: K ⊆ Rd is compact if for
any family {Ui }i∈I of open sets in Rd (indexed by any set I) such that
[
K⊆
Ui
i∈I
({Ui }i∈I is an “open cover” of K), there are i1 , . . . , im ∈ I such that
K ⊆ Ui1 ∪ · · · ∪ Uim
49
({Ui1 , . . . , Uim } is a “finite subcover”). This definition makes sense in a much more general setting
of topological spaces. Metric spaces are certain special topological spaces, which include all subsets
of Rd (and much more), and for these, sequential compactness is equivalent to compactness, a result
which you will see in MAT 3120. The Heine–Borel Theorem, however, does not generalize to this
setting. In the more general setting of topological spaces, compactness and sequential compactness
are different notions, neither implying the other.
Exercise 4.4.1. Prove Proposition 4.4.6
Exercise 4.4.2. In the proof of the Heine–Borel Theorem, we use a generalization of Proposition 2.5.4
to Rd . State and prove this generalization.
Exercise 4.4.3. [Sav17, Exercise 4.4.6] Let A1 , . . . , Ad ⊆ R be compact sets. Show that
A1 × · · · × Ad = {(a1 , . . . , ad ) : ai ∈ Ai for i = 1, . . . , d} ⊆ Rd
is compact.
Exercise 4.4.4. Let K ⊆ Rd be a compact set. Let f : K → R be a function. Suppose that for every
x ∈ K there exists an open set U containing x and some M ∈ R such that for all y ∈ U ,
f (y) ≤ M.
Prove that f (K) = {f (x) : x ∈ K} is bounded above.
[Hint. Proof by contradiction. Suppose it is not bounded above, use this to construct a sequence
(an )∞
n=1 in K with certain properties, and then use a convergent subsequence.]
Exercise 4.4.5. Let K ⊆ Rd . Prove that the following are equivalent.
(i) K is compact.
(ii) Every infinite set E ⊆ K has a limit point z which is in K.
Exercise 4.4.6. Let K ⊆ R be a compact set. Prove that min(K) and max(K) exist (recall that this
means that K contains a lower and upper bound for itself).
[Hint. Use the Heine–Borel Theorem! ]
50
Chapter 5
Continuous functions
In this chapter, we develop another notion of a limit: here it is for a function defined on a subset of
the real numbers, and the concept concerns the behaviour of the function as the variable tends to a
particular point. This concept enables us to talk about continuous functions; it will also (in the next
chapter) play a key role in establishing differentiation.
5.1
The limit of a function
Definition 5.1.1. Let X ⊆ Rd and let a ∈ Rd be a limit point of X. Let f : X → Rm and let
L ∈ Rm . We write
lim f (x) = L
x→a
if for every > 0 there exists δ > 0 such that, if x ∈ X \ {a} and kx − ak2 < δ then kf (x) − Lk2 < .
The idea in the above definition is that when x is close (but not equal) to a, then f (x) should be
close (and possibly equal) to L. It makes sense that we restrict to a being a limit point, because that
is exactly the condition that tells us that there are points (arbitrarily) close to a inside X (Exercise
4.3.10). Recall that a limit point a may or may not be inside X itself; however, if a ∈ X, then the
value f (a) does not matter at all to the limit lim f (x) (or whether this limit exists).
x→a
Example 5.1.2. Define f : R → R by f (x) := 2x + 5. Then for any a ∈ R, we have lim f (x) = 2a + 5.
To see this, set L := 2a + 5, and let > 0 be given. Set δ :=
|x − a| < δ = 2 , then
.
2
x→a
Then if x ∈ R \ {a} and
|f (x) − L| = |2x + 5 − (2a + 5)| = 2|x − a| < 2 = .
2
Example 5.1.3. Define g : R → R by g(x) := x2 . Then for any a ∈ R, we have lim g(x) = a2 .
x→a
To see this, set L := a2 and let > 0 be given. Now, for any x ∈ R, we have
|g(x) − L| = |x2 − a2 | = |x − a| · |x + a|.
We want to make this < . We can make the term |x − a| arbitrarily small, but for the |x + a| term,
it might not be so small but we can at least try to bound it by something fixed. Indeed, if |x − a| < 1
then
|x + a| = |x − a + 2a| ≤ |x − a| + 2|a| < 1 + 2|a|.
So, we now see that we want |x − a| <
.
1+2|a|
51
o
n
Now we set δ := min 1, 1+2|a| . If |x − a| < δ then
|g(x) − L| = |x − a| · |x + 1|
≤ |x − a|(1 + 2|a|)
<
(1 + 2|a|)
1 + 2|a|
= ,
as required.
We next establish uniqueness of limits of a function, in much the same way as we did for sequences.
Proposition 5.1.4. Let X ⊆ Rd and let a ∈ Rd be a limit point. Let f : X → Rm and let L, L0 ∈ Rm .
If lim f (x) = L and lim f (x) = L0 then L = L0 .
x→a
x→a
0
k2
. Using the definition of limit twice
Proof. Suppose for a contradiction that L 6= L0 . Set := kL−L
2
simultaneously, we may find δ > 0 such that for all x ∈ X \ {a} with kx − ak2 < δ,
kf (x) − Lk2 < andkf (x) − L0 k2 < .
Since a is a limit point of X, we may find x ∈ X ∩ B(a; δ) \ {a} (Exercise 4.3.10); that is, x ∈ X,
x 6= a, and kx − ak2 < δ.
Consequently, kf (x) − Lk2 < and kf (x) − L0 k2 < . But then by using the Triangle Inequality,
kL − L0 k2 ≤ kL − f (x)k2 + kf (x) − L0 k2 < + = kL − L1k2 ,
a contradiction.
Next, we shall connect limits of a function with limits of a sequence. This will be instrumental
towards quickly proving basic properties (such as algebra of limits).
Proposition 5.1.5 (Sequential Characterization of Limits). Let X ⊆ Rd and let a ∈ Rd be a limit
point. Let f : X → Rm and let L ∈ Rm . Then lim f (x) = L if and only if for every sequence (an )∞
n=1
x→a
in X \ {a} which converges to a, we have
lim f (an ) = L.
n→∞
Proof. ⇒: We assume that lim f (x) = L. To show that lim f (an ) = L, let > 0 be given. Choose
x→a
n→∞
δ > 0 such that if x ∈ X \ {a} and kx − ak2 < δ then
kf (x) − Lk2 < .
Next, since lim an = a, there exists n0 ∈ N≥1 such that if n ≥ n0 then kan − ak2 < δ.
n→∞
Then for n ≥ n0 , we have an ∈ X \ {a} and kan − ak2 < δ, and thus,
kf (an ) − Lk2 < ,
as required.
⇐: We now assume that for every sequence (an )∞
n=1 in X \ {a} which converges to a, we have
lim f (an ) = L.
n→∞
52
Suppose for a contradiction that it is not true that lim f (x) = L. This means that there is some
x→a
> 0 such that for every δ > 0, there exists x ∈ X \ {a} such that kx − ak2 < δ and kf (x) − Lk2 ≥ .
We will now construct a sequence using this.
Namely, for each n ∈ N≥1 , using δ := n1 , we may choose some an ∈ X \ {a} such that
kan − ak2 <
1
n
and kf (an ) − Lk2 ≥ .
This gives us a sequence (an )∞
n=1 in X \ {a}. Since 0 ≤ kan − ak2 <
ensures that lim kan − ak2 = 0, i.e., lim an = a.
n→∞
1
n
for all n, the Squeeze Theorem
n→∞
By hypothesis, it follows that lim f (an ) = L. But this is a contradiction since we have kf (an ) −
n→∞
Lk2 ≥ for all n.
Proposition 5.1.6 (Algebra of Limits). Let X ⊆ Rd and let a ∈ Rd be a limit point. Let f, g : X →
Rm and γ : X → R be functions which all have limits at a. Let c ∈ R. Then:
(i) lim (f (x) + g(x)) = lim f (x) + lim g(x) .
x→a
x→a
x→a
(ii) lim (cf (x)) = c lim f (x).
x→a
x→a
(iii) lim (γ(x)f (x)) = lim γ(x) lim f (x) .
x→a
x→a
x→a
1
x→a γ(x)
(iv) If γ(x) 6= 0 for all x ∈ X and lim γ(x) 6= 0 then lim
x→a
=
1
.
lim γ(x)
x→a
Proof. Exercise 5.1.1. (Use the Sequential Characterization of Limits and Exercise 4.2.2.)
Theorem 5.1.7 (Squeeze Theorem). Let X ⊆ Rd and let a ∈ Rd be a limit point. Let f, g, h : X → R
be functions which satisfy
f (x) ≤ g(x) ≤ h(x) for all x ∈ X.
If
lim f (x) = lim h(x) = L
x→a
x→a
then
lim g(x) = L
x→a
as well.
Proof. Exercise 5.1.2.
Given a function f : X → Rm , we may define functions f1 , . . . , fm : X → R by
f (a) = (f1 (a), . . . , fm (a)),
a ∈ X.
(Equivalently, fi is the composition of f with the ith coordinate projection Rm → R.) We call
f1 , . . . , fm the component functions of f , and we write f = (f1 , . . . , fm ).
Proposition 5.1.8. Let X ⊆ Rd and let a ∈ Rd be a limit point. Let f = (f1 , . . . , fm ) : X → Rm be
a function and let L = (L1 , . . . , Lm ) ∈ Rm . Then lim f (x) = L if and only if lim fi (x) = Li for each
x→a
x→a
i = 1, . . . , m.
53
Proof. Exercise 5.1.3. (Use the Sequential Characterization of Limits and Proposition 4.2.3.)
The order structure in R allows us to talk of one-sided limits when we have a function of one
variable.
Definition 5.1.9. Let X ⊆ R, let f : X → Rd be a function and let L ∈ Rd . If a ∈ R is a limit
point of X ∩ (a, ∞), then we write
lim+ f (x) = L
x→a
if for every > 0 there exists δ > 0 such that, if x ∈ X, x > a, and
|x − a| < δ
then
|f (x) − L| < .
Likewise, if a ∈ R is a limit point of X ∩ (−∞, a), then we write
lim f (x) = L
x→a−
if for every > 0 there exists δ > 0 such that, if x ∈ X, x < a, and
|x − a| < δ
then
|f (x) − L| < .
Example 5.1.10. Define f : R → R by


−1,
f (x) := 0,


1,
x < 0;
x = 0;
x > 0.
Then one can see that
lim f (x) = 1
x→0+
and
lim f (x) = −1.
x→0−
Exercise 5.1.1. Prove Proposition 5.1.6.
Exercise 5.1.2. Prove Theorem 5.1.7.
Exercise 5.1.3. Prove Proposition 5.1.8.
Exercise 5.1.4. Define f : R \ {0} → R by
f (x) :=
x+1
.
x+3
Prove directly using the definition that
1
lim f (x) = .
x→0
3
54
Exercise 5.1.5. Suppose f : R → R is a function, L ∈ R, and
1
lim f
= L.
n→∞
n
Does it follow that lim+ f (x) = L? (Give a proof or counterexample.)
x→0
m
m
Exercise 5.1.6. Let (an )∞
n=1 be a sequence in R and let L ∈ R . Define a function
1 1
f : 1, , , . . . → Rm
2 3
by
1
= an .
f
n
Prove that lim an = L if and only if lim f (x) = L.
n→∞
x→0
Exercise 5.1.7. Prove that the following limits do not exist.
(i) lim x1 .
x→0
(ii) lim sin
x→0
1
x
.
Exercise 5.1.8. Let X ⊆ R, let a ∈ R be a limit point of X ∩ (a, ∞). Let f : X → Rm and let
L ∈ Rm . Define g := f |X∩(a,∞) : X ∩ (a, ∞) → Rm . Prove that
lim f (x) = L iff
x→a+
lim g(x) = L.
x→a
Exercise 5.1.9. (cf. [Sav17, Exercise 5.1.8]) Let X ⊆ R and let a ∈ R be a limit point of both
X ∩ (a, ∞) and X ∩ (−∞, a). Let f : X → Rd be a function and let L ∈ Rd . Prove that
lim f (x) = L
x→a
if and only if both
lim f (x) = L and
x→a+
5.2
lim f (x) = L.
x→a−
Continuity
We now formulate the notion of a continuous function by making use of the concept of a limit.
Definition 5.2.1. Let X ⊆ Rd and let a ∈ X be a point which is not isolated. Let f : X → Rm . We
say f is continuous at a if
lim f (x) = f (a).
x→a
If a ∈ X is an isolated point, then we say that any function f : X → Rm is continuous at a.
Making use of the definition of limit, we find that f : X → Rm is continuous at a ∈ X if for
any > 0 there exists δ > 0 such that, if x ∈ X and ka − xk2 < δ then kf (a) − f (x)k2 < . This
reformulation says that f is continuous if nearby points get sent to nearby images; it works whether
or not a is an isolated point.
55
Example 5.2.2. Define f : R → R by f (x) := 2x + 5. Then using Example 5.1.2, we have for any
a ∈ R,
lim f (x) = f (a),
x→a
and thus f is continuous at every a ∈ R.
Example 5.2.3. Define f : [0, ∞) → R by
(
sin
f (x) :=
0,
1
x
,
if x > 0;
if x = 0.
Then f is not continuous at 0. To see this, we need to show that lim f (x) = 0 does not hold. Consider
x→0
the sequence
∞
1
∞
(an )n=1 =
π(2n + 0.5) n=1
which converges to 0. Note that the corresponding sequence (f (an ))∞
n=1 is the constant sequence
(1, 1, . . . ),
which does not converge to 0.
Example 5.2.4. Define g : [0, ∞) → R by
(
x sin
g(x) :=
0,
1
x
,
if x > 0;
if x = 0.
This function oscillates similarly to the previous example, but the magnitude of the oscillation is
tempered by the factor of x. We shall prove that this function is continuous at 0.
For this, consider functions f, h : [0, ∞) → R given by f (x) := x, h(x) := −x. Then we have
f (x) ≤ g(x) ≤ h(x) for all x, and
lim f (x) = 0 = lim h(x).
x→∞
x→∞
Therefore, by the Squeeze Theorem, lim g(x) = 0 = g(0), as required.
x→∞
Proposition 5.2.5. Let X ⊆ Rd and let Y ⊆ Rm . Let f : X → Y ⊆ Rm and g : Y → Rn
be functions. Let a ∈ X. Suppose that f is continuous at a and g is continuous at f (a). Then
g ◦ f : X → Rn is continuous at a.
Proof. Let > 0 be given. Since g is continuous at f (a), there exists η > 0 (we’re using η instead of
δ) such that for all y ∈ Y , if ky − f (a)k2 < η then kg(y) − g(f (a))k2 < . Next, since f is continuous
at a, using η in place of in that definition, we obtain δ > 0 such that for all x ∈ X, if kx − ak2 < δ
then kf (x) − f (a)k2 < η.
Now, putting these together, if x ∈ X and kx − ak2 < δ then kf (x) − f (a)k2 < η and so (viewing
f (x) as y),
kg(f (x)) − g(f (a))k2 < ,
as required.
Proposition 5.2.6. Let X ⊆ Rd and let a ∈ Rd be a limit point. Let f, g : X → Rm and γ : X → R
be functions which are all continuous at a. Let c ∈ R. Then:
56
(i) f + g is continuous at a.
(ii) c · f is continuous at a.
(iii) γ · f is continuous at a.
(iv) If γ(x) 6= 0 for all x ∈ X then
1
γ
is continuous at a.
Proof. These follow from Proposition 5.1.6.
Exercise 5.2.1. [Leb16, Exercise 3.2.3] Define f : R → R by
(
x,
if x ∈ Q;
f (x) =
2
x,
if x 6∈ Q.
Prove that f is continuous at 1 and discontinuous at 2.
Exercise 5.2.2. Let X ⊆ Rd , let f : X → Rm , and let a ∈ X. Let r > 0 and define
g := f |X∩B(a;r) : X ∩ B(a; r) → Rm .
Prove that f is continuous at a if and only if g is continuous at a.
Exercise 5.2.3. [Sav17, Exercise 5.2.3] Let f : Rd → Rm . Recall that for B ⊆ Rm the preimage of B
under f is
f −1 (B) := {x ∈ Rd : f (x) ∈ B}.
Prove that f (x) is continuous for all x ∈ Rd if and only if for every open set U of Rm , the preimage
f −1 (U ) is open. Prove also that f is continuous if and only if for every closed set F of Rm , the
preimage f −1 (F ) is closed.
Exercise 5.2.4. Define the Dirichlet function f : R → R by
(
0,
if x 6∈ Q;
f (x) := 1
,
if x = pq in lowest terms (with p ∈ Z and q ∈ N≥1 ).
q
Prove that for any a ∈ R, lim f (x) = 0. From this, determine the points at which f is continuous.
x→a
5.3
Properties of continuous functions
In the previous section, we defined what it means for a function to be continuous at a single point.
Here we consider functions that are “globally” continuous, or continuous at every point of their
domain. Such functions have some strong and useful properties, as we shall see.
Definition 5.3.1. Let X ⊆ Rd and let f : X → Rm be a function. We say that f is continuous (on
X) if f is continuous at a for every a ∈ X.
Theorem 5.3.2. Let K ⊆ Rd be compact and let f : K → Rm be a continuous function. Then its
image, f (K), is also compact.
57
Proof. Let (yn )∞
n=1 be a sequence in f (K); we need to find a subsequence that converges to a point
in f (K). By definition of f (K), we can find xn ∈ K for each n such that yn = f (xn ). Since K is
∞
compact, there is a subsequence (xnk )∞
k=1 (of the sequence (xn )n=1 ) which converges to a point a ∈ K.
Since f is continuous at a, we have
lim f (x) = f (a),
x→a
and therefore by the Sequential Characterization of Limits (Proposition 5.1.5), lim f (xnk ) = f (a).
k→∞
This says that the subsequence (ynk )∞
k=1 converges to the point f (a) ∈ f (K), as required.
This has an important consequence for the problem of optimizing functions.
Corollary 5.3.3 (Extreme Value Theorem). Let K ⊂ Rd be compact and nonempty, and let f :
K → R be a continuous function. Then there exists xmin , xmax ∈ K such that for all x ∈ K,
f (xmin ) ≤ f (x) ≤ f (xmax ).
In other words, the image of f is bounded above and below, and it attains its bounds.
Proof. By the previous theorem, f (K) is compact, By Exercise 4.4.6, min(f (K)) and max(f (K))
exist: that is, there are ymin , ymax ∈ f (K) such that
ymin ≤ z ≤ ymax
for all z ∈ f (K). We may take xmin , xmax ∈ K such that
f (xmin ) = ymin
and f (xmax ) = ymax .
Then it follows that for all x ∈ K,
f (xmin ) ≤ f (x) ≤ f (xmax ).
Remark 5.3.4. If X ⊆ Rd is not compact, then it is possible that f (X) is not bounded, or that it is
bounded but it does not attain its bounds. Here are some examples.
Define f : (0, 1] → R by f (x) := x1 . Then f ((0, 1]) = [1, ∞) is not bounded above.
2
Define g : R → R by g(x) := x2x+1 . Then g(R) = [0, 1), which is bounded, but has no maximum.
Theorem 5.3.5 (Intermediate Value Theorem). Let f : [a, b] → R be a continuous function. Let
y ∈ R be any value between f (a) and f (b). Then there exists z ∈ [a, b] such that f (z) = y.
Proof. We may assume without loss of generality that f (a) ≤ f (b), since in the other case, we may
replace f with −f (and y with −y). Define
S := {x ∈ [a, b] : f (x) ≤ y}.
This set is nonempty (it contains a) and bounded above (by b). Therefore, z := sup(S) exists.
We need only prove that f (z) = y.
To show f (z) ≥ y, assume for a contradiction that f (z) < y. Then using := y − f (z) we can
find δ such that for all x ∈ [a, b] with |x − z| < δ, we have |f (x) − f (z)| < . We may find x ∈ [a, b]
with |x − z| < δ and x > z (since z 6= b), and for such x,
f (x) < f (z) + = f (z) + y − f (z) = y.
58
Hence, x ∈ S, contradicting that z is an upper bound.
Next, to show f (z) ≤ y, assume for a contradiction that f (z) > y. Then using := f (z) − y
we can find δ such that for all x ∈ [a, b] with |x − z| < δ, we have |f (x) − f (z)| < . Then for all
x ∈ (z − δ, z], we have
f (x) > f (z) − = f (z) − (f (z) − y) = y,
so that x 6∈ S. Since z is an upper bound for S, this shows that z − δ is also an upper bound. But
that contradicts that z is the least upper bound.
The Intermediate Value Theorem is extremely powerful, since it gives us criteria (often easy to
check) for an equation to have a solution. For example, for any p ∈ N≥0 and α > 0, the function
f : [0, ∞) → R defined by
f (x) := xp − α
is continuous. We have f (0) = −α < 0 and (since np → ∞ as n → ∞) we can find b ∈ [0, ∞)
such that f (b) > 0. Then the Intermediate Value Theorem immediately tells us that there exists
z ∈ [0, ∞) such that f (z) = 0, i.e., z p = α. In this way, we are immediately able to define pth roots.
Proposition 5.3.6. Let f : [a, b] → R be a continuous function. Then f ([a, b]) = [c, d] for some
c, d ∈ R.
Proof. By the Extreme Value Theorem, there exist xmin , xmax ∈ [a, b] such that
f ([a, b]) ∈ [f (xmin ), f (xmax )].
Set c := f (xmin ) and d := f (xmax ).
We thus have one containment, f ([a, b]) ⊆ [c, d], and we only need to prove the other containment.
Let y ∈ [c, d]. Then by the Intermediate Value Theorem, there exists z between xmin and xmax such
that f (z) = y. Thus, y ∈ f ([a, b]), as required.
Exercise 5.3.1. [TBB08, Exercise 5.7.1] Give an example of a function f : R → R such that for all
a ∈ R, f is not continuous at a, yet such that the conclusion of the Extreme Value Theorem holds,
i.e., there exists xmin , xmax ∈ R such that for all x ∈ R,
f (xmin ) ≤ f (x) ≤ f (xmax ).
Exercise 5.3.2. Let f : R → R be a periodic function, i.e., such that there exists P > 0 such that
f (P + x) = f (x) for all x ∈ R. Suppose that f is continuous. Prove that the conclusion of the
Extreme Value Theorem holds, i.e., there exists xmin , xmax ∈ R such that for all x ∈ R,
f (xmin ) ≤ f (x) ≤ f (xmax ).
Exercise 5.3.3. Let f : [a, b] → [a, b] be a continuous function. Prove that there exists y ∈ [a, b] such
that f (y) = y (such y is called a fixed point).
Exercise 5.3.4. Let I ⊆ R be an interval and let f : I → R be a continuous function. Prove that
f (I) is an interval.
Exercise 5.3.5. Let f : Rd → R be a continuous function. Suppose there exists x+ , x− ∈ Rd such that
f (x− ) < 0 < f (x+ ).
Prove that there exists x0 ∈ Rd such that f (x0 ) = 0. For a bigger challenge, prove that there are
infinitely many such points.
Exercise 5.3.6. [Leb16, Exercise 3.3.3] Let f : (0, 1) → R be a continuous function such that
lim f (x) = lim f (x). Prove that f attains a minimum or f attains a maximum (but possibly not
x→0
x→1
both): that is, either there exists xmin ∈ (0, 1) such that f (xmin ) ≤ f (x) for all x ∈ (0, 1), or there
exists xmax ∈ (0, 1) such that f (x) ≤ f (xmax ) for all x ∈ (0, 1).
59
5.4
Inverses of continuous functions
Recall that a function f : A → B is injective or one-to-one if for all x, y ∈ A,
f (x) = f (y) ⇒ x = y.
A function f : A → B is surjective or onto if
f (A) = B.
When f is both injective and surjective, it is called bijective, and it follows that it has an inverse: a
function f −1 : B → A such that
f −1 ◦ f = idA ,
f ◦ f −1 = idB .
Namely, for y ∈ B, one defines
f −1 (y) := x
where x ∈ A is the unique element satisfying f (x) = y.
Often we are given an injective function f : A → B, and by replacing the codomain B with f (B),
we get a function f : A → f (A) which is surjective for free, and thus bijective.
When we start with a continuous injective function f : A → Rd , it would be nice if its inverse
−1
f were also continuous; unfortunately, this is not true in this level of generality, and we need to
put restrictions on the domain A.
Example 5.4.1. Define f : [0, 1) ∪ [2, 3] → R by
(
t,
f (t) :=
t − 1,
t ∈ [0, 1);
t ∈ [2, 3].
This function is injective, so it has an inverse f −1 : [0, 2] → R which is given by
(
t,
t ∈ [0, 1);
−1
f (t) =
t + 1,
t ∈ [1, 2].
The function f is continuous but f −1 is not.
For positive results, we will focus on maps from an interval into R; however see also Exercise 5.4.2
for a different general result.
Definition 5.4.2. Let X ⊆ R and let f : X → R be a function.
(i) We say f is (weakly) increasing if for x, y ∈ X,
x ≤ y =⇒ f (x) ≤ f (y).
(ii) We say f is strictly increasing if for x, y ∈ X,
x < y =⇒ f (x) < f (y).
(iii) We say f is (weakly) decreasing if for x, y ∈ X,
x ≥ y =⇒ f (x) ≥ f (y).
60
(iv) We say f is strictly decreasing if for x, y ∈ X,
x > y =⇒ f (x) > f (y).
(Note that when X = N≥1 then the above concepts for a function f : N≥1 → R agree with those
already defined in Definition 2.4.1.)
Lemma 5.4.3. Let a < b and let f : [a, b] → R be a continuous function. The following are
equivalent.
(i) f is either strictly increasing or strictly decreasing.
(ii) f is injective.
Proof. (i) ⇒ (ii): This is immediate from the definitions.
(ii) ⇒ (i): Assume without loss of generality that f (a) < f (b), and let’s prove that f is strictly
increasing. If it is not then there exists x1 , x2 ∈ [a, b] such that x1 < x2 and f (x1 ) ≥ f (x2 ).
Consider two cases:
If f (x1 ) ≥ f (b) then we have
f (a) ≤ f (b) ≤ f (x1 ),
so by applying the Intermediate Value Theorem to f |[a,x1 ] (with y := f (b)), there exists z ∈ [a, x1 ]
such that f (z) = f (b). Since f is injective, this would mean that z = b, but z ≤ x1 < x2 ≤ b, so this
is a contradiction.
If f (x1 ) ≤ f (b), then we have
f (x2 ) ≤ f (x1 ) ≤ f (b),
so by applying the Intermediate Value Theorem to f |[x2 ,b] (with y := f (x1 )), there exists z ∈ [x2 , b]
such that f (z) = f (x1 ). Again, since f is injective, this would mean that z = x1 , but z ≥ x2 > x1 ,
so this is a contradiction.
Theorem 5.4.4. Let I ⊆ R be an interval and let f : I → R be an injective continuous function.
Then f −1 : f (I) → R is continuous.
Proof. On every bounded closed subinterval [a, b] of I, f is either strictly increasing or strictly
decreasing. From this it is clear that f is either strictly increasing or strictly decreasing on all of I.
Assume, without loss of generality, that f is strictly increasing.
Let b ∈ f (I); to show that f −1 is continuous at b, we must show lim f −1 (y) = f −1 (b). So, let
y→b
> 0 be given.
Set a := f −1 (b) (so that f (a) = b). Let us assume that a ∈ I ◦ – in the other case, a is an endpoint
of the interval I and then we will need to adjust the following argument slightly. Since there is no
61
harm in decreasing , we may further assume that a − , a + ∈ I. Since f is strictly increasing, we
have
f (a − ) < f (a) < f (a + ).
We then set
δ := min{f (a) − f (a − ), f (a + ) − f (a)} > 0.
We are done once we prove the following claim.
Claim. If y ∈ f (I) and |y − b| < δ, then |f −1 (y) − f −1 (b)| < .
To prove this claim, suppose it is false, so either
f −1 (y) ≤ f −1 (b) − or f −1 (y) ≥ f −1 (b) + .
Set x := f −1 (y) (so that f (x) = y), so the above reads
x ≤ a − or x ≥ a + .
In the first case, since f is strictly increasing, we get
y = f (x) ≤ f (a − ) ≤ f (a) − δ = b − δ.
In the second case, we similarly obtain
y = f (x) ≥ f (a + ) ≥ f (a) + δ = b + δ.
In either case, we get a contradiction to the assumption that |y − b| < δ.
This concludes the proof of the claim.
Exercise 5.4.1. Let p ∈ N≥2 . Prove that the function f : [0, ∞) → R given by f (x) :=
continuous.
√
p
x is
Exercise 5.4.2. Let K ⊆ Rd be a compact set and let f : K → Rm be a continuous injective function.
Prove that f −1 : K → Rd is continuous.
[Hint. Generalize Exercise 5.2.3 to say that if X ⊆ Rd is closed then a function f : X → Rm is
continuous if and only if for every closed set F of Rm , the preimage f −1 (F ) is closed.]
5.5
Uniform continuity
Let X ⊆ Rd and let f : X → Rm be a function. If we unwind the definition of “f is a continuous
function”, it says:
For any x ∈ X and > 0 there exists δ > 0 such that if y ∈ X and |x − y| < δ then |f (x) − f (y)| < .
In this definition, the δ is allowed to depend on and on x. It is crucial that δ is allowed to
depend on , to capture the idea that “close points get sent to close images” (see Exercise 5.5.3).
However, it is less clear, at first, what role the dependence on x plays.
Here we explore this subtlety, by first introducing a variant on continuity where δ is not allowed
to depend on x.
Definition 5.5.1. Let X ⊆ Rd and let f : X → Rm be a function. We say that f is uniformly
continuous (on X) if for any > 0 there exists δ > 0 such that for any x, y ∈ X, if kx − yk2 < δ
then kf (x) − f (y)k2 < .
62
Example 5.5.2. Define f : R → R by f (x) := |x|. This is uniformly continuous.
To see this, given > 0, we set δ := . Then if x, y ∈ R and |x − y| < δ, then
|x| − |y| ≤ |x − y| < δ = .
Example 5.5.3. Define f : R → R by f (x) := x2 Then f is continuous (by Proposition 5.2.6 (iii)) but
not uniformly continuous.
To see that it is not uniformly continuous, suppose for a contradiction that it is. Then using
:= 1 there would be some δ > 0 such that for all x, y ∈ R, if |x − y| < δ then |f (x) − f (y)| < 1.
However, we may take
δ
1
x := , y := x + ,
δ
2
so that |x − y| = 2δ < δ. However,
|f (x) − f (y)| = y 2 − x2
= (y − x)(y + x)
δ
≥ (x + x)
2
δ 2
= · = 1,
2 δ
a contradiction.
The obstruction to uniform continuity in the previous example came from points that are large.
In fact, in the next example, we should that if we eliminate large points from the domain then the
function becomes uniformly continuous.
Example 5.5.4. Consider the restriction of the previous example to [a, b] where a, b ∈ R with a < b.
That is, f : [a, b] → R given by f (x) := x2 . This function is uniformly continuous.
To see this, let > 0 be given. Set M := max{|a|, |b|}, so that M is an upper bound for
. Then for x, y ∈ [a, b] with |x − y| < δ, we have
{|x| : x ∈ [a, b]}. Take δ := 2M
|f (x) − f (y)| = |x2 − y 2 |
= |x − y||x + y|
≤ |x − y|(|x| + |y|)
< δ(M + M )
= ,
as required.
In fact, the previous example is a special case of a general phenomenon: continuous functions on
compact sets are automatically uniformly continuous.
Theorem 5.5.5. Let K ⊆ Rd be compact and let f : K → Rm be continuous. Then f is uniformly
continuous.
Proof. We prove this by contradiction: suppose that f is not uniformly continuous, so there exists
> 0 such that for every δ > 0 there exist x, y ∈ K with kx − yk < δ and kf (x) − f (y)k ≥ .
In particular, for each n, we may choose some xn , yn ∈ X such that kxn − yn k2 < n1 yet kf (xn ) −
∞
f (yn )k2 ≥ . This gives us two sequences, (xn )∞
n=1 and (yn )n=1 , in K.
63
Since K is compact, there is a subsequence (xnk )∞
k=1 which converges to a point a ∈ K. The
∞
subsequence (ynk )k=1 must also converge to a, since
kynk − ak2 ≤ kynk − xnk k2 + kxnk − ak2 → 0,
and so by the Squeeze Theorem, kynk − ak2 → 0.
Since f is continuous and by using the Sequential Characterization of Limits (Proposition 5.1.5),
it follows that
lim f (xnk ) = f (a) = lim f (ynk ),
k→∞
k→∞
and in particular,
lim kf (xnk ) − f (ynk )k2 = 0.
k→∞
But this is a contradiction since kf (xnk ) − f (ynk )k2 ≥ for all k.
Exercise 5.5.1. Determine which of the following functions are uniformly continuous.
(a) f : (0, ∞) → R defined by f (x) := x1 .
(b) f : [1, ∞) → R defined by f (x) := x1 .
(c) f : R → R defined by f (x) :=
1
1+x2
(d) f : (0, ∞) → R defined by f (x) := sin
1
x
.
(e) f : [1, ∞) → R defined by f (x) := ln(x).
(f) f : R → R defined by f (x) := sin(x2 ).
Exercise 5.5.2. Let f : Rd → Rm be continuous. Prove that for any bounded set X ⊂ Rd , the
restricted function f |X is uniformly continuous.
Exercise 5.5.3. Consider modifying the definition of continuity so that δ may not depend on (although it can depend on x): for any x there exists δ > 0 such that for any > 0, if y ∈ X and
|x − y| < δ, then |f (x) − f (y)| < . Show that if X = R and f : X → Rm satisfies this definition
then f must be a constant function.
Exercise 5.5.4. Let X = X1 ∪ X2 ⊆ Rd and let f : X → Rm be a function. Suppose that f |X1 and
f |X2 are both uniformly continuous. Must f be uniformly continuous? [Hint. Consider the case
X1 = (−∞, 0) and X2 = (0, ∞).]
Exercise 5.5.5. Let X ⊆ Rd be bounded and let f : X → Rm be uniformly continuous. Prove that
f (X) is bounded.
Exercise 5.5.6. Let f : Rd → Rm be a function and let L ∈ Rm . Let us write
lim f (x) = L if for
kxk2 →∞
every > 0 there exists M such that if x ∈ Rd and kxk2 ≥ M then kf (x) − Lk2 < .
Prove that if f is continuous and lim f (x) = L then f is uniformly continuous.
kxk2 →∞
Exercise 5.5.7. Let X ⊆ Rd and let f : X → Rm be a function. f is Lipschitz if there is some M > 0
such that for all x, y ∈ X,
kf (x) − f (y)k2 ≤ M kx − yk2 .
(i) Prove that if f is Lipschitz then it is uniformly continuous.
√
(ii) Prove that f : [0, 1] → R defined by f (t) := t is uniformly continuous but not Lipschitz.
64
(iii) Prove that f : [1, ∞) → R defined by f (t) :=
√
t is Lipschitz.
Exercise 5.5.8. (Challenging!) Let X ⊆ Rd and let f : X → Rm be uniformly continuous. Prove that
there is a continuous function g : X → Rm such that g|X = f .
∞
[For a point a ∈ X \ X, take a sequence (xn )∞
n=1 in X that converges to a. Prove that (f (xn ))n=1 is
Cauchy, so define g(a) := lim f (xn ). Prove that this makes g a well-defined function and that it is
n→∞
continuous.]
5.6
Infinite limits and limits at infinity
It is often useful to have the concept of ∞ at our disposal when speaking of limits. We have a few
definitions related to this concept.
Definition 5.6.1. Let A ⊆ Rd and let f : A → Rm .
• If m = 1 and a ∈ Rd is a limit point of A, then we write lim f (x) = ∞ if for every R > 0 there
x→a
exists δ > 0 such that if x ∈ A \ {a} and kx − ak2 < δ then
f (x) > R.
• Similarly, if m = 1 and a ∈ Rd is a limit point of A, then we write lim f (x) = −∞ if for every
x→a
R > 0 there exists δ > 0 such that if x ∈ A \ {a} and kx − ak2 < δ then
f (x) < −R.
• If d = 1, A is not bounded above, and L ∈ Rm , we write lim f (x) = L if for every > 0 there
x→∞
exists R > 0 such that if x ∈ A and x > R then
kf (x) − Lk2 < .
• Similarly, if d = 1, A is not bounded below, and L ∈ Rm , we write lim f (x) = L if for every
x→−∞
> 0 there exists R > 0 such that if x ∈ A and x < −R then
kf (x) − Lk2 < .
• If A is not bounded and L ∈ Rm , we write
lim f (x) = L if for every > 0 there exists R > 0
kxk2 →∞
such that if x ∈ A and kxk2 > R then
kf (x) − Lk2 < .
One can also define things such as
lim f (x) = ∞, by splicing these definitions appropriately
kxk2 →∞
(Exercise 5.6.1).
m
Let f : N≥1 → Rm , and form the associated sequence (an )∞
n=1 . Observe that for L ∈ R ,
lim f (x) = L iff
x→∞
65
lim an = L.
n→∞
Exercise 5.6.1. Splice the definitions in order to define lim f (x) = ∞, lim f (x) = ∞, lim f (x) =
x→∞
−∞, lim f (x) = −∞,
x→−∞
lim f (x) = −∞, and
kxk2 →∞
lim
kxk2 →−∞
x→−∞
x→∞
f (x) = −∞.
Exercise 5.6.2. Let A ⊆ Rd , B ⊆ Rm , and let f : A → Rm , g : B → Rn , such that f (A) ⊆ B. Let
a ∈ Rd be a limit point of A. Suppose that lim kf (x)k2 = ∞ (this makes sense since x 7→ kf (x)k2 is
x→a
a real-valued function) and
lim g(y) = L ∈ Rn . Prove that
kyk2 →∞
lim g(f (x)) = L.
x→a
Exercise 5.6.3. Let f : Rd → Rm be a continuous function. Suppose that
that f is bounded.
[Use the Extreme Value Theorem.]
66
lim f (x) exists. Prove
kxk2 →∞
Chapter 6
Differentiation
We will now use the concept of a limit in order to define differentiability and the derivative of a
real-variable function. Differentiation is a fundamental operation in calculus; the machinery built in
the previous chapter will allow us to rigorously establish the key properties of the derivative.
6.1
The derivative
Definition 6.1.1. Let X ⊆ R, let f : X → R be a function, let a ∈ X be a non-isolated point. We
write
f (x) − f (a)
f 0 (a) := lim
x→a
x−a
(either as a real number, as ±∞, or as “not existing”). When this limit exists and is finite, we say f
is differentiable at a. If all points of X are non-isolated and f is differentiable at every point of X,
df
.
then we say f is differentiable on X. In this case, we get a function f 0 : X → R, also written as dx
As you have seen in Calculus, f 0 (a) is the slope of the tangent line to the graph of y = f (x) at
the point (a, f (a)).
Example 6.1.2. Define f : R → R by f (x) = x2 . Then for a ∈ R,
f (x) − f (a)
x→a
x−a
2
x − a2
= lim
x→a x − a
(x + a)(x − a)
= lim
x→a
x−a
= lim x + a = 2a.
f 0 (a) = lim
x→a
Thus f is differentiable on R.
Example 6.1.3. Define f : R \ {0} → R by
f (x) :=
67
1
.
x
Then for a ∈ R \ {0},
f (x) − f (a)
x→a
x−a
1
−1
= lim x a
x→a x − a
a−x
1
= lim
·
x→a xa
x−a
−1
= lim
x→a xa
−1
= 2.
a
f 0 (a) = lim
Example 6.1.4. Define f : R → R by f (x) := |x|. If a > 0, note that for x close to a we have
f (x) = x. Thus,
x−a
f (x) − f (a)
= lim
= 1.
f 0 (a) = lim
x→a x − a
x→a
x−a
If a < 0, for x close to a we have f (x) = −x, and thus,
f (x) − f (a)
−x − (−a)
= lim
= −1.
x→a
x→a
x−a
x−a
f 0 (a) = lim
However for a = 0, we have
lim+
x→0
f (x) − f (0)
x−0
= lim+
= 1,
x→0 x − 0
x−0
whereas
f (x) − f (0)
−x − 0
= lim+
= −1.
x→0
x→0
x−0
x−0
Since these one-sided limits are different, f 0 (0) does not exist (Exercise 5.1.9).
lim−
Proposition 6.1.5. Let X ⊆ R, let f : X → R be a function, let a ∈ X be a non-isolated point. If
f is differentiable at a then f is continuous at a.
Proof. We need to show that lim f (x) = f (a). We have
x→a
f (x) − f (a)
lim f (x) − f (a) = lim
· (x − a)
x→a
x→a
x−a
f (x) − f (a) = lim
lim x − a
x→a
x→a
x−a
= f 0 (a) · 0 = 0.
(Algebra of Limits)
In the above, we were able to use Algebra of Limits (Proposition 5.1.6) because the limits exist.
We therefore have lim f (x) = f (a), as required.
x→a
The following example shows that, even if f is differentiable everywhere, the function f 0 might
not be continuous.
68
Example 6.1.6. Define f : R → R by
(
x2 sin
f (x) :=
0,
1
x
,
x 6= 0;
x = 0.
This function is differentiable everywhere; for a 6= 0, this will follow from the rules of differentiation
(Chain Rule, Product Rule, Quotient Rule), and we have
1
−1
1
2
0
+ a cos
· 2
f (a) = 2a sin
a
a
a
1
1
= 2a sin
− cos
.
a
a
For a = 0, we have
f (x) − f (0)
f 0 (a) = lim
x→0
x
− 0
1
= lim x sin
x→0
x
=0
by Example 5.2.4.
Similarly,
1
1
lim f (x) = lim 2x sin
− cos
x→0
x→0
x
x
1
,
= 0 − lim cos
x→0
x
0
but this limit does not exist (just like Exercise 5.1.7 (ii)). Hence, f 0 is not continuous at 0.
If f is differentiable and f 0 is differentiable, we write the derivative of f 0 as f 00 or f (2) . Similarly
we define f 000 = f (3) , and so on.
Exercise 6.1.1. Let n ∈ N≥1 and define f : R → R by f (x) := xn . Directly compute f 0 (a) for a ∈ R.
Exercise 6.1.2. [TBB08, Exercise 7.2.1] Let X ⊆ R, let f : X → R be a function, let a ∈ X be a
non-isolated point. Show that
f (a + h) − f (a)
f 0 (a) = lim
.
h→0
h
Exercise 6.1.3. [TBB08, Exercise 7.2.3] Determine which of the following functions f : R → R are
differentiable at 0.
(i) f (x) := x|x|.
(
x sin
(ii) f (x) :=
0,
(
x2 ,
(iii) f (x) :=
0,
1
x
,
x 6= 0;
x = 0.
x ∈ Q;
x 6∈ Q.
Exercise 6.1.4. [Leb16, Exercise 4.1.9] Let X ⊆ R be a set in which all points are non-isolated.
Suppose that f : X → R is differentiable function which is Lipschitz (see the definition in Exercise
5.5.7). Prove that f 0 : X → R is bounded.
Exercise 6.1.5. [TBB08, Exercise 7.2.13] Let f : R → R be strictly increasing and differentiable.
Does this imply that f 0 (x) ≥ 0 for all x ∈ R? Does it imply that f 0 (x) > 0 for all x ∈ R?
69
6.2
Computation rules for derivatives
In Calculus, you have seen a number of tools for computing derivatives. In this section, we shall
prove these rules.
Proposition 6.2.1. Let X ⊆ R, let f, g : X → R be functions, let a ∈ X be a non-isolated point.
Suppose that f and g are both differentiable at a, and let c ∈ R. Then
(i) (cf )0 (a) = c(f 0 (a)) and (f + g)0 (a) = f 0 (a) + g 0 (a);
(Linearity)
(ii) (f g)0 (a) = f 0 (a)g(a) + f (a)g 0 (a).
(Product Rule)
We don’t do the Quotient Rule here; it can be proven using the Chain Rule and the Product Rule
(Exercise 6.2.2).
Proof. Part (i) is Exercise 6.2.1
(iii): We compute
f (x)g(x) − f (a)g(a)
(f g)(x) − (f g)(a)
= lim
x→a
x→a
x−a
x−a
f (x)g(x) − f (a)g(x) + f (a)g(x) − f (a)g(a)
= lim
x→a
x − a f (x) − f (a)
g(x) − g(a)
= lim
lim g(x) + f (a) lim
x→a
x→a
x→a
x−a
x−a
0
0
= f (a)g(a) + f (a)g (a),
lim
using Algebra of Limits in the third line, and that g is continuous at a (Proposition 6.1.5) for the
last equality.
Proposition 6.2.2 (Chain Rule). Let X, Y ⊆ R, let f : X → R and g : Y → R be functions, let
a ∈ X be a non-isolated point. Suppose that f (X) ⊆ Y and that f (a) is a non-isolated point of Y .
Suppose also that f is differentiable at a and g is differentiable at f (a). Then g ◦ f is differentiable
at a, and
(g ◦ f )0 (a) = g 0 (f (a))f 0 (a).
Proof. For y ∈ Y , define
(
h(y) :=
g(y)−g(f (a))
,
y−f (a)
0
g (f (a)),
y 6= f (a);
y = f (a).
Since lim h(y) = g 0 (f (a)) = h(f (a)), h is continuous at f (a).
y→f (a)
Claim. For all x ∈ X \ {a},
g(f (x)) − g(f (a))
f (x) − f (a)
= h(f (x))
.
x−a
x−a
To see this claim, we consider two cases. If f (x) 6= f (a), then we have
g(f (x)) − g(f (a))
g(f (x)) − g(f (a)) f (x) − f (a)
=
·
x−a
f (x) − f (a)
x−a
f (x) − f (a)
= h(f (x))
.
x−a
70
On the other hand, if f (x) = f (a), then both sides are 0, so the claim holds.
Now, using the claim, we compute
g(f (x)) − g(f (a))
x−a
f (x) − f (a)
= lim h(f (x))
x→a
x−a
0
= h(f (a))f (a)
= g 0 (f (a))f 0 (a).
(g ◦ f )0 (a) = lim
x→a
In the second-last line we used the fact that h ◦ f is continuous at a, which is a consequence of
Proposition 5.2.5.
Proposition 6.2.3 (Inverse Rule). Let X ⊆ R be an interval, let f : X → R be a continuous,
injective function. Let a ∈ X. If f is differentiable at a and f 0 (a) 6= 0 then f −1 : f (X) → R is
differentiable at f (a) and
1
(f −1 )0 (f (a)) = 0 .
f (a)
Proof. For convenience, set b := f (a).
Similarly to the previous proof, we define h : X → R by
(
f (x)−f (a)
,
x=
6 a;
x−a
h(x) :=
0
f (a),
x = a.
This function is continuous, and h(a) 6= 0. It follows that h is nonzero everywhere on some open
interval containing a; so by shrinking X, we may assume that h(x) 6= 0 for all x ∈ X.
Now we observe that, for y ∈ f (X) \ {b}, we have
f −1 (y) − f −1 (b)
f −1 (y) − a
=
y−b
f (f −1 (y)) − f (a)
1
.
=
−1
h(f (y))
Since h and f −1 are continuous (the latter by Theorem 5.4.4), so is their composition. Hence by the
Algebra of Limits, we get
f −1 (y) − f −1 (b)
1
1
1
1
=
=
=
= 0 .
−1
−1
y→b
y−b
lim h(f (y))
h(f (b))
h(a)
f (a)
lim
y→b
Exercise 6.2.1. Prove Proposition 6.2.1 (i).
Exercise 6.2.2. Let X ⊆ R, let f, g : X → R be a function, let a ∈ X be a non-isolated point.
Suppose that f and g are both differentiable at a, and that g(x) 6= 0 for all x ∈ X, and g 0 (a) 6= 0.
0
0 (a)
(a) Using the Chain Rule, prove that g1 (a) = −g
.
g(a)2
(b) Using (a) and the Product Rule, prove the Quotient Rule:
71
0
f
g
(a) =
f 0 (a)g(a)−f (a)g 0 (a)
.
g(a)2
Exercise 6.2.3. (cf. [Leb16, Exercise 4.1.11]) Let X ⊆ R, let f, g : X → R be a function, let a ∈ X be
a non-isolated point. Suppose that f is bounded and that g(a) = g 0 (a) = 0. Consider the function
h = f g.
(a) Prove that h0 (a) = 0.
(b) Is it possible to use the product rule to do part (a)?
Exercise 6.2.4. [TBB08, Exercise 7.3.17] Using the fact that the derivative of sin(x)
is cos(x), find a
formula for the derivative of sin−1 (x) (considered as a function [−1, 1] → − π2 , π2 ).
6.3
Optimizing differentiable functions
Derivatives are often used for optimizing functions: finding their maximum and/or minimum (extrema). The precise relationship between derivatives and extrema has to do with the concept of
“local extrema”: the minimum or maximum of a point in a (possibly) small neighbourhood.
Definition 6.3.1. Let X ⊆ R, let f : X → R, and let a ∈ X be an interior point.
(i) a is a local minimum of f if there exists r > 0 such that (a − r, a + r) ⊆ X and
f (a) ≤ f (x)
for all x ∈ (a − r, a + r).
(ii) a is a local maximum of f if there exists r > 0 such that (a − r, a + r) ⊆ X and
f (a) ≥ f (x)
for all x ∈ (a − r, a + r).
Theorem 6.3.2. Let X ⊆ R, let f : X → R, and let a ∈ X be an interior point. If f has a local
maximum or local minimum at a and f is differentiable at a, then f 0 (a) = 0.
Proof. Assume that f has a local minimum at a (the other case is almost identical). Let r > 0 be
such that (a − r, a + r) ⊆ X and
f (a) ≤ f (x) for all x ∈ (a − r, a + r).
Since f is differentiable, the limit
f (x) − f (a)
x→a
x−a
exists, and by the Sequential Characterization of Limits, for any sequence (xn )∞
n=1 in X converging
to a, we will have
f (xn ) − f (a)
f 0 (a) = lim
.
n→∞
xn − a
lim
72
Now, let us first take some sequence (xn )∞
n=1 in (a, a + r) converging to a from the right: for
example,
r
.
xn := a +
n+1
Then we have xn − a > 0 and f (xn ) − f (a) ≥ 0. Thus,
f (xn ) − f (a)
≥ 0,
xn − a
which implies that
f (xn ) − f (a)
≥ 0.
n→∞
xn − a
However, we may similarly take some sequence (xn )∞
n=1 in (a − r, a) converging to a from the left:
for example,
r
xn := a −
.
n+1
In this case we have xn − a < 0 and f (xn ) − f (a) ≥ 0, so that
f 0 (a) = lim
f (xn ) − f (a)
≤ 0,
xn − a
and thus,
f (xn ) − f (a)
≤ 0.
n→∞
xn − a
Putting these two inequalities together, we obtain f 0 (a) = 0 as required.
f 0 (a) = lim
Suppose f : [a, b] → R is continuous. By the Extreme Value Theorem, it attains a maximum
and a minimum. Let’s say we want to find a point xmax ∈ [a, b] where it achieves its maximum; the
point xmax will in particular be a local maximum, so if xmax is an interior point of [a, b] and f is
differentiable at xmax , then by the above theorem, f 0 (xmax ) = 0. It is possible, however, that xmax
doesn’t satisfy the hypotheses of the above theorem (it might not be differentiable, or it might not
be an interior point). Altogether, we get the following three possibilities:
(i) f is differentiable at xmax and f 0 (xmax ) = 0,
(ii) xmax = a or xmax = b (it is not an interior point), or
(iii) f is not differentiable at xmax .
Exercise 6.3.1. Give examples of continuous functions f : [a, b] → R which attain a maximum at a
unique point xmax such that
(a) in the first example, f is not differentiable at xmax ,
(b) in the second example, xmax is a and f is differentiable at xmax ,
Exercise 6.3.2. [TBB08, Exercise 7.5.1] Give an example of a differentiable function f : R → R such
that f 0 (0) = 0 but 0 is not a local maximum nor minimum.
Exercise 6.3.3. [TBB08, Exercise 7.5.2] Define f : R → R by
(
x 6= 0;
x4 2 + sin x1 ,
f (x) :=
0,
x = 0.
73
(i) Prove that f is differentiable on R.
(ii) Prove that f has a minimum at 0.
(iii) Prove that for every r > 0, f 0 ((−r, r)) contains both (strictly) positive and negative numbers.
6.4
The Mean Value Theorem
The Mean Value Theorem tells us that the average change of a differentiable function is attained as
the derivative of the function at some point. This will be derived from Rolle’s Theorem, which is the
special case in which the average change of a function is zero.
Theorem 6.4.1 (Rolle’s Theorem). Let f : [a, b] → R be a continuous function that is differentiable
on (a, b). If f (a) = f (b) then there exists x0 ∈ (a, b) such that
f 0 (x0 ) = 0.
Proof. If f is constant, then f 0 (x0 ) = 0 for all x0 ∈ (a, b). Otherwise, by the Extreme Value Theorem,
let xmin , xmax be the minimal and maximal points of f , so that
f (xmin ) ≤ f (x) ≤ f (xmax )
for all x ∈ [a, b]. Since x is not constant, we know that either
f (xmin ) < f (a) = f (b) or f (xmax ) > f (a) = f (b),
(or both) and in particular, either xmin ∈ (a, b) or xmax ∈ (a, b). We set x0 equal to one of these two
points which is in (a, b).
Then f is differentiable at x0 , and since x0 is a global extreme point for f , it follows from Theorem
6.3.2 that f 0 (x0 ) = 0, as required.
Next we do a generalization of the Mean Value Theorem. This generalization can be used to
prove L’Hôpital’s rule (Exercise 6.4.5).
Theorem 6.4.2 (Cauchy’s Mean Value Theorem). Suppose that f, g : [a, b] → R are continuous
functions that are differentiable on (a, b). Then there exists x0 ∈ (a, b) such that
(f (b) − f (a))g 0 (x0 ) = (g(b) − g(a))f 0 (x0 ).
Proof. Define h : [a, b] → R by
h(x) := (f (b) − f (a))g(x) − (g(b) − g(a))f (x).
Our strategy is to apply Rolle’s Theorem to h.
The function h is continuous, and it is differentiable on (a, b) with
h0 (x) = (f (b) − f (a))g 0 (x) − (g(b) − g(a))f 0 (x).
Also, we compute
h(a) = (f (b) − f (a))g(a) − (g(b) − g(a))f (a)
= f (b)g(a) − f (a)g(a) − g(b)f (a) + g(a)f (a)
= f (b)g(a) − g(b)f (a).
74
Likewise, h(b) = f (b)g(a) − g(b)f (a).
Hence by Rolle’s Theorem, there exists x0 ∈ (a, b) such that h0 (x0 ) = 0. Looking at the computation for h0 (x), it follows that
(f (b) − f (a))g 0 (x0 ) = (g(b) − g(a))f 0 (x0 ).
Corollary 6.4.3 (Mean Value Theorem). Let f : [a, b] → R be a continuous function that is differentiable on (a, b). Then there exists x0 ∈ (a, b) such that
f 0 (x0 ) =
f (b) − f (a)
.
b−a
Proof. Define g : [a, b] → R by g(x) := x, then apply Cauchy’s Mean Value Theorem to f, g to get
x0 ∈ (a, b) such that
(f (b) − f (a))g 0 (x0 ) = (g(b) − g(a))f 0 (x0 ).
Since g 0 (x0 ) = 1 and g(b) − g(a) = b − a, this rearranges to become
f 0 (x0 ) =
f (b) − f (a)
.
b−a
Exercise 6.4.1. [TBB08, Exercise 7.6.4] Suppose that f : R → R is a differentiable function and there
is some c > 0 such that f 0 (x) ≥ c for all x ∈ R. Prove that lim f (x) = ∞.
x→∞
Exercise 6.4.2. [TBB08, Exercise 7.6.5] Let f : R → R be a twice-differentiable function (i.e., f 0 (x)
and f 00 (x) exist everywhere). Prove that if f has at least three zeroes (three distinct points x1 , x2 , x3
such that f (x1 ) = f (x2 ) = f (x3 ) = 0) then there exists x0 ∈ R such that f 00 (x0 ) = 0.
Exercise 6.4.3. Suppose that f : [a, b] → R is a continuous function which is differentiable on (a, b),
such that f 0 (x) = 0 for all x ∈ (a, b). Prove that f is the constant function.
Exercise 6.4.4. Let I ⊆ R be an interval and let f : I → R be a differentiable function. Recall from
Exercise 6.1.4 that if f is Lipschitz (see Exercise 5.5.7 for the definition) then f 0 is bounded. Use the
Mean Value Theorem to prove the converse: that if f 0 is not Lipschitz then f is not bounded.
Exercise 6.4.5. Let I be an interval containing a as an interior point, and let f, g : I → R be
continuous functions, such that g(x) > 0∀x ∈ I \ {a}, g 0 (x) 6= 0∀x ∈ I \ {a}, and f (a) = g(a) = 0.
(i) Let > 0. Suppose that b ∈ I satisfies
f (b) ≥ g(b)(L + ).
Prove that there is some x ∈ I such that
f 0 (x) ≥ g 0 (x)(L + ).
[Use Cauchy’s Mean Value Theorem.]
(ii) Prove that if
g(x)(L − ) < f (x) < g(x)(L + ) for all x ∈ I.
then
g 0 (x)(L − ) < f 0 (x) < g 0 (x)(L + ) for all x ∈ I.
75
(iii) Prove that if
f 0 (x)
exists,
x→a g 0 (x)
L := lim
then
lim
x→a
f (x)
= L.
g(x)
(This is one case of L’Hôpital’s Rule.)
Exercise 6.4.6. [TBB08, Exercise 7.6.15] Let f : [0, ∞) → R be a function which is differentiable,
N
P
such that f 0 (x) is decreasing and positive. By writing f (N ) − f (0) as
f (n) − f (n − 1), prove that
n=1
the series
∞
X
f 0 (n)
n=1
converges if and only if f is bounded.
[You may not use the Integral Test. In fact, this is essentially a restatement of the Integral Test
through the Fundamental Theorem of Calculus, but the task here is to prove it by just using the Mean
Value Theorem.]
76
Chapter 7
Riemann integration
In this chapter we define integration and study its properties. Integrating a positive function can be
thought of as measuring the area under its graph (and above the x-axis). We will prove some important properties of the integral, including the Fundamental Theorem of Calculus (relating integration
to differentiation).
7.1
The Riemann integral
To formally define the integral, we proceed by estimating this area. Fix a < b.
A partition of an interval [a, b] is a finite set {t0 , t1 , . . . , tn } such that
a = t0 < t1 < · · · < tn = b.1
A partition breaks up the interval [a, b] into n subintervals
[t0 , t1 ], [t1 , t2 ], . . . , [tn−1 , tn ].
Let P = {t0 , t1 , . . . , tn } be a partition and let f : [a, b] → R be a bounded function. For
i = 1, . . . , n, define
mi (P, f ) := inf f ([ti−1 , ti ]) = inf{f (t) : t ∈ [ti−1 , ti ]},
Mi (P, f ) := sup f ([ti−1 , ti ]) = sup{f (t) : t ∈ [ti−1 , ti ]}.
Definition 7.1.1. Let P = {t0 , t1 , . . . , tn } be a partition and let f : [a, b] → R be a bounded function.
The lower Darboux sum of f for P is
L(P, f ) :=
n
X
mi (P, f )(ti − ti−1 ).
i=1
The upper Darboux sum of f for P is
U (P, f ) :=
n
X
Mi (P, f )(ti − ti−1 ).
i=1
1
Of course, sets are not ordered: {t0 , t1 , t2 , t3 } = {t1 , t3 , t0 , t2 }. However, when we write out a partition this way
as {t0 , t1 , . . . , tn }, we will understand that the elements are in order (unless otherwise noted).
77
The idea is the following. Suppose that f is a positive function. We start with a partition P ,
which corresponds to a way of breaking up [a, b] into intervals [t0 , t1 ], [t1 , t2 ], . . . , [tn−1 , tn ]. If we then
ask for the smallest rectangles which have these intervals on one side, and which cover the graph of
f , then the rectangles we will come up with are precisely
[t0 , t1 ] × [0, M1 (P, f )], . . . , [ti−1 , ti ] × [0, Mi (P, f )], . . . [tn−1 , tn ] × [0, Mn (P, f )].
The area of these rectangles is precisely the upper Darboux sum, and in this sense, the upper
Darboux sum gives an estimation from above of the area under f . Likewise, the lower Darboux sum
corresponds to the largest rectangles that can be fit under the graph of f , so the lower Darboux sum
gives an estimation from below of the area under f .
Example 7.1.2. Let f : [0, 1] → R be given by f (t) := t, and let P := t0 := 0, t1 := 31 , t2 := 1 . Then
m1 (P, f ) = 0,
1
m2 (P, f ) = ,
3
1
M1 (P, f ) = ,
3
M2 (P, f ) = 1.
Hence,
L(P, f ) = m1 (P, f )(t1 − t0 ) + m2 (P, f )(t2 − t1 ) = 0
and
1
U (P, f ) = M1 (P, f )(t1 − t0 ) + M2 (P, f )(t2 − t1 ) =
3
1
1
1
2
−0 +
1−
=
3
3
3
9
1
1
7
−0 +1 1−
= .
3
3
9
It is clear that mi (P, f ) ≤ Mi (P, f ) always holds, and therefore,
L(P, f ) ≤ U (P, f )
for any partition P .
Definition 7.1.3. Let P, P 0 be partitions. We say that P 0 refines P if P ⊆ P 0 (as sets).
Lemma 7.1.4. Let f : [a, b] → R be a bounded function and let P, P 0 be partitions of [a, b] such that
P 0 refines P . Then
L(P, f ) ≤ L(P 0 , f ) and U (P 0 , f ) ≤ U (P, f ).
Proof. Since P refines P 0 , it is obtained from P by adding finitely many points. We will prove that
if P 0 contains exactly one more point than P , then
L(P, f ) ≤ L(P 0 , f ) and U (P 0 , f ) ≤ U (P, f ).
The general case will then follow by induction, using this special case as the inductive step.
78
We will only prove L(P, f ) ≤ L(P 0 , f ), as the case for upper sums is similar.
Let P = {t0 , t1 , . . . , tn } and let P 0 = {t0 , . . . , tk−1 , s, tk , . . . , tn }, with
a = t0 < t1 < · · · < tk−1 < s < tk < · · · < tn = b.
Let us compare
L(P, f ) = m1 (P, f )(t1 − t0 ) + · · · + mk (P, f )(tk − tk−1 ) + · · · + mn (P, f )(tn − tn−1 )
with
L(P 0 , f ) = m1 (P 0 , f )(t1 − t0 ) + · · · + mk (P 0 , f )(s − tk−1 ) + mk+1 (P 0 , f )(tk − s)
+ · · · + mn+1 (P 0 , f )(tn − tn−1 ).
Note that for i < k, we have
mi (P, f ) = mi (P 0 , f ),
and likewise, for i > k,
mi (P, f ) = mi+1 (P 0 , f ).
Hence,
L(P 0 , f ) − L(P, f ) = mk (P 0 , f )(s − tk−1 ) + mk+1 (P 0 , f )(tk − s) − mk (P, f )(tk − tk−1 ).
Nett, observe that since f ([tk−1 , s]) ⊆ f ([tk−1 , tk ], we have
mk (P, f ) = inf f ([tk−1 , tk ]) ≤ inf f ([tk−1 , s]) = mk (P 0 , f ),
and likewise,
mk (P, f ) ≤ mk+1 (P 0 , f ).
Thus,
L(P 0 , f ) − L(P, f )
= mk (P 0 , f )(s − tk−1 ) + mk+1 (P 0 , f )(tk − s) − mk (P, f )(tk − tk−1 )
≥ mk (P, f )(s − tk−1 ) + mk (P, f )(tk − s) − mk (P, f )(tk − tk−1 )
= 0,
as required.
Corollary 7.1.5. Let f : [a, b] → R be a bounded function and let P, P 0 be partitions of [a, b]. Then
L(P, f ) ≤ U (P 0 , f ).
Proof. Let P 00 = P ∪ P 0 (as sets), so that P 00 is a partition of [a, b] which refines both P and P 0 .
Then
L(P, f ) ≤ L(P 00 , f )
≤ U (P 00 , f )
≤ U (P 0 , f )
(previous lemma)
(previous lemma).
79
Definition 7.1.6. A bounded function f : [a, b] → R is (Riemann) integrable if
sup{L(P, f ) : P a partition of [a, b]} = inf{U (P, f ) : P a partition of [a, b]}.
In this case, we set
b
Z
Z
b
f
f (t) dt or
a
a
equal to this number, called the integral of f (on [a, b]).
In other words,
b
Z
f (t) dt = I
a
if we can approximate I both from above and from below arbitrarily well by upper and lower Darboux
sums (respectively).
If [a, b] ⊆ A ⊆ R and f : A → R is a function such that f |[a,b] is integrable, then we continue to
write
Z
b
f (t) dt
a
for the value of the integral of f |[a,b] . We also write
Z
a
Z
b
f (t) dt = −
f (t) dt.
b
a
To shorten notation, we will sometimes write
sup L(P, f )
P
to mean
sup{L(P, f ) : P a partition of [a, b]},
and similarly
inf U (P, f )
P
for
inf{U (P, f ) : P a partition of [a, b]}.
Note that by Corollary 7.1.5, it follows that
sup L(P, f ) ≤ inf U (P, f ).
P
P
Example 7.1.7. Let f : [0, 1] → R be defined by f (t) := t. This is integrable and
To see this, consider the partition P = {0, n1 , n2 , . . . , nn = 1}. Then
i−1 i
i−1 i
i−1
mi (f, P ) = inf f
,
= inf
,
=
,
n n
n n
n
80
R1
0
f (t) dt = 21 .
and so
L(f, P ) =
n
X
mi (f, P )
i=1
i
i−1
−
n
n
n
X
i−1 1
=
·
n
n
i=1
Similarly, Mi (f, P ) =
i
n
=
n
1 X
(i − 1)
n2 i=1
=
n(n − 1)
n−1
.
=
2n2
2n
and so
U (f, P ) =
=
n
X
i=1
n
X
i=1
Mi (f, P )
i−1
i
−
n
n
i 1
·
n n
=
n
1 X
i
n2 i=1
=
n(n + 1)
n+1
=
.
2n2
2n
It follows that
sup L(P, f ) ≥ sup
P
1
= ,
2
1
= .
2
n−1
: n ∈ N≥1
2n
and on the other hand,
inf U (P, f ) ≤ inf
P
n+1
: n ∈ N≥1
2n
Therefore, these must both equal 12 , which means that
Z
0
1
1
f (t) dt = .
2
Example 7.1.8. Consider the function f : [0, 1] → R defined by
(
0,
t ∈ Q;
f (t) :=
1,
t 6∈ Q.
For any partition P , we will show that
L(P, f ) = 0 and U (P, f ) = 1.
Consequently,
sup L(P, f ) = 0 6= 1 = inf U (P, f ),
P
P
so that f is not integrable.
81
Let P = {t0 , . . . , tn } be a partition. Then for each i, the set [ti−1 , ti ] contains both rational and
irrational numbers. Hence,
f ([ti−1 , ti ]) = {0, 1},
so that mi (P, f ) = 0 and Mi (P, f ) = 1. From this we get
U (P, f ) =
n
X
Mi (P, f )(ti − ti−1 )
i=1
=
n
X
(ti − ti−1 )
i=1
= tn − t0
= 1.
(by telescoping)
On the other hand,
L(P, f ) =
n
X
mi (P, f )(ti − ti−1 ) =
n
X
i=1
0(ti − ti−1 ) = 0.
i=1
Example 7.1.9. Consider the function f : [0, 2] → R defined by
(
0,
t = 1;
f (t) :=
1,
t 6= 1.
R1
This is integrable and 0 f (t) dx = 2.
To see this, given > 0, consider the partition
P := {t0 = 0, t1 = 1 − , t2 = 1 + , t3 = 2}.
We compute
m1 (P, f ) = 1, m2 (P, f ) = 0, m3 (P, f ) = 1,
and therefore
L(f, P ) =
3
X
mi (f, P )(ti − ti−1 )
i=1
= 1(1 − − 0) + 0(1 + − (1 − )) + 1(2 − (1 + ))
= 1 − + 1 − = 2 − 2.
Therefore,
sup L(f, P ) ≥ 2 − 2,
P
and since > 0 is arbitrary,
sup L(f, P ) ≥ 2.
P
On the other hand, taking the “trivial” partition P := {0, 2}, we have M1 (f, P ) = 1, and so
U (f, P ) = M1 (f, P )(2 − 0) = 2,
82
and therefore
inf U (f, P ) ≤ 2.
P
Hence, supP L(f, P ) = inf P U (f, P ) = 2, so f is integrable and
Z 1
f (t) dt = 2.
0
Here is a way of rephrasing integrability.
Proposition 7.1.10. Let f : [a, b] → R be a bounded function. Then f is integrable if and only if,
for every > 0 there exists a partition P such that
U (P, f ) − L(P, f ) < .
Proof. ⇒: Assume that f is integrable and let > 0. Set
I := sup L(P, f ) = inf U (P, f ).
P
P
It follows that there are partitions PL and PU such that
L(PL , f ) > I − ,
2
U (PU , f ) < I + .
2
Let P = PL ∪ PU , so by Lemma 7.1.4,
L(P, f ) ≥ L(PL , f ) and U (P, f ) ≤ U (PU , f ).
Therefore,
− I+
= .
2
2
⇐: For any > 0 there is a partition P such that U (P, f ) − L(P, f ) < .
U (P, f ) − L(P, f ) ≤ U (PU , f ) − L(PL , f ) < I −
0 ≤ inf U (Q, f ) − sup L(Q, f ) ≤ U (P, f ) − L(P, f ) < .
Q
Q
Since > 0 is arbitrary, it follows that inf Q U (Q, f ) = supQ L(Q, f ), so that f is integrable.
Theorem 7.1.11. If f : [a, b] → R is continuous then f is integrable.
Proof. Let > 0 be given. Since f is continuous and [a, b] is compact, by Theorem 5.5.5, there exists
δ > 0 such that if x, y ∈ [a, b] and |x − y| < δ then |f (x) − f (y)| < b−a
.
Let P = {t0 , . . . , tn } be a partition such that |ti − ti−1 | < δ for all i. By Proposition 5.3.6,
f ([ti−1 , ti ]) is a closed interval, and then by the uniform continuity condition, we see that its maximum
and minimum cannot be separated by more than /(b − a).2 In other words,
Mi (P, f ) − mi (P, f ) <
2
.
b−a
In fact, we don’t really need to use Proposition 5.3.6 here:
Mi (P, f ) − mi (P, f ) = sup f ([ti−1 , ti ]) − inf f ([ti−1 , ti ]) = sup{f (x) − f (y) : x, y ∈ [ti−1 , ti ]} ≤
83
.
b−a
Consequently,
U (P, f ) − L(P, f ) =
n
X
(Mi (P, f ) − mi (P, f ))(ti − ti−1 )
i=1
<
n
X
i=1
=
(ti − ti−1 )
b−a
(tn − t0 ) = .
b−a
This verifies the condition in the previous proposition, so we conclude that f is integrable.
Exercise 7.1.1. Define f : [−1, 1] → R by f (x) := x2
(i) Let P := −1, − 21 , 13 , 1 . Compute L(P, f ) and U (P, f ).
(ii) Find a partition P such that U (P, f ) − L(P, f ) < 21 .
Exercise 7.1.2. [Leb16, Exercise 5.1.5] Define f : [−1, 1] → R by
(
1,
x > 0;
f (x) :=
0,
x ≤ 1.
Prove that f is integrable and compute
R1
−1
f (t) dt.
Exercise 7.1.3. Let f : [0, 1] → R be an increasing function.
(a) Prove that for any partition P and any i, Mi (P, f ) = mi+1 (P, f ).
(b) For n ∈ N≥1 , set Pn := {0, n1 , n2 , . . . , nn }. Find a short expression for U (Pn , f ) − L(Pn , f ).
(c) Prove that f is Riemann integrable.
Exercise 7.1.4. Recall the Dirichlet function from Exercise 5.2.4:
(
0,
if x 6∈ Q;
f (x) := 1
,
if x = pq in lowest terms (with p ∈ Z and q ∈ N≥1 ).
q
Let > 0.
(a) Prove that for any partition P of [0, 1],
L(P, f ) = 0.
(b) Prove that there is a number N such that for any partition P of [0, 1], the set
{i : Mi (P, f ) ≥ }
has size at most N .
(c) Let N be as in part (b). Suppose that P = {x0 , x1 , . . . , xn } is a partition of [0, 1], and let
δ := max{x1 − x0 , x2 − x1 , . . . , xn − xn−1 }.
Prove that
U (f, P ) ≤ + N δ.
84
(d) Prove that there is a partition P such that
U (f, P ) < 2.
(e) Prove that f is integrable and compute
R1
0
f (t) dt.
Exercise 7.1.5. [Leb16, Exercise 5.1.7] Let f : [a, b] → R be Riemann integrable. Prove that given
> 0 there is a partition P = {t0 , . . . , tn } of [a, b] such that, for any ξ1 ∈ [t0 , t1 ], . . . , ξn ∈ [tn−1 , tn ],
Z b
n
X
f (t) dt −
f (ξi )(ti − ti−1 ) < .
a
[The sum
n
P
i=1
f (ξi )(ti − ti−1 ) is called a Riemann sum for f .]
i=1
Exercise 7.1.6. [Leb16, Exercise 5.1.8] Let α > 0, β ∈ R. Suppose that f : [β, α + β] → R is Riemann
integrable. Define g : [0, 1] → R by
g(x) := f (αx + β).
R1
R α+β
Prove that g is Riemann integrable and compute 0 g in terms of β f .
Exercise 7.1.7. Let f : [a, b] → R be a bounded function and let P be a partition of [a, b]. Show that
for c > 0,
U (P, cf ) = cU (P, f ) and L(P, cf ) = cL(P, f ).
What happens if c < 0?
7.2
Properties of the integral
Proposition 7.2.1 (Additive Property). Let f : [a, b] → R be a bounded function and let c ∈ (a, b).
Then f is integrable if and only if f |[a,c] and f |[c,b] are both integrable. In this case,
Z b
Z c
Z b
f (t) dt.
f (t) dt +
f (t) dt =
a
c
a
Proof. ⇒: Let > 0 be given. There exists a partition P of [a, b] such that U (P, f ) − L(P, f ) < . By
possibly adding a point (which leads to a refinement of the original P ), we may assume that c ∈ P .
Break up the partition P into two partitions corresponding to [a, c] and [c, b]: define
P1 := P ∩ [a, c] and P2 := P ∩ [c, b].
Then one sees that
U (P, f ) = U (P1 , f |[a,c] ) + U (P2 , f |[c,b] )
and likewise
L(P, f ) = L(P1 , f |[a,c] ) + L(P2 , f |[c,b] ).
Since U (P2 , f |[c,b] ) − L(P2 , f |[c,b] ) ≥ 0, it follows that
U (P1 , f |[a,c] ) − L(P1 , f |[a,c] ) ≤ U (P, f ) − L(P, f ) < .
Likewise,
U (P2 , f |[c,b] ) − L(P2 , f |[c,b] ) ≤ U (P, f ) − L(P, f ) < .
Since is arbitrary, we see that both f |[a,c] and f |[c,b] are integrable.
Rb
Rc
Rb
⇐ and showing a f = a f + c f are Exercise 7.2.1
85
Proposition 7.2.2 (Linearity of the integral). Let f, g : [a, b] → R be bounded integrable functions
and let c ∈ R. Then cf + g is integrable, and
Z b
Z b
Z b
g(t) dt.
f (t) dt +
cf (t) + g(t) dt = c
a
a
a
Proof. The case c = 0 is trivial. For c > 0, it is an exercise (Exercise 7.2.2) to show that
mi (P, cf + g) ≥ cmi (P, f ) + mi (P, g) and Mi (P, cf + g) ≤ cMi (P, f ) + Mi (P, g).
It follows that
L(P, cf + g) ≥ cL(P, f ) + L(P, g) and U (P, cf + g) ≤ cU (P, f ) + U (P, g).
Consequently,
sup L(P, cf + g) ≥ sup cL(P, f ) + L(P, g)
P
P
≥ c sup L(P, f ) + sup L(P, g)
P
P
Z b
Z b
=c
f+
g.
a
a
and likewise
b
Z
inf U (P, cf + g) ≤ c
Z
b
g.
f+
P
a
a
Rb
Rb
Rb
It follows that cf + g is integrable with a cf + g = c a f + a g.
For the case c < 0, it suffices (by combining with the case we just did) to show that −f is
Rb
Rb
integrable and a −f = − a f . This is Exercise 7.2.3.
Proposition 7.2.3. Let f, g : [a, b] → R be integrable. If f (t) ≤ g(t) for all t ∈ [a, b] then
Z
b
Z
f (t) dt ≤
b
g(t) dt.
a
a
Proof. Let P be a partition. Then it is straightforward to see that mi (P, f ) ≤ mi (P, g) for all i, and
thus
L(P, f ) ≤ L(P, g).
Therefore,
Z
b
Z
f = sup L(P, f ) ≤ sup L(P, g) =
a
P
P
b
g.
a
Corollary 7.2.4. Let f : [a, b] → R be integrable. If m, M ∈ R are such that
m ≤ f (t) ≤ M
for all x ∈ [a, b] then
Z
m(b − a) ≤
b
f (t) dt ≤ M (b − a).
a
86
Exercise 7.2.1. Prove the remaining part of Proposition 7.2.1: that if f : [a, b] → R is a bounded
function such that f |[a,c] and f |[c,b] is integrable then f is integrable and
Z
b
Z
f (t) dt =
a
c
Z
f (t) dt +
a
b
f (t) dt.
c
Exercise 7.2.2. Let f, g : [a, b] → R be bounded functions, let P be a partition of [a, b], and let c > 0.
(a) Show that mi (P, f + g) ≥ mi (P, f ) + mi (P, g) and Mi (P, f + g) ≤ Mi (P, f ) + Mi (P, g) for all i.
(b) Show that mi (P, cf ) = cmi (P, f ) and Mi (P, cf ) = cMi (P, f ).
(c) Show that mi (P, cf + g) ≥ cmi (P, f ) + mi (P, g) and Mi (P, cf + g) ≤ cMi (P, f ) + Mi (P, g).
(d) Show that mi (P, −f ) = −Mi (P, f ).
Exercise 7.2.3. Let f : [a, b] → R be an integrable function. Prove that −f is integrable and
b
Z
b
Z
−f (t) dt = −
f (t) dt.
a
a
Exercise 7.2.4. Let f : [a, b] → R be a continuous function such that f (t) ≥ 0 for all t ∈ [a, b].
Rb
Suppose that a f (t) dt = 0. Prove that f (t) = 0 for all t ∈ [a, b].
[Hint. If f (x) > 0 then use continuity to obtain γ and c < d such that f (t) ≥ γ for all t ∈ [c, d].]
Rb
Exercise 7.2.5. Let f : [a, b] → R be continuous. Prove that if a f (t)g(t) dt = 0 for every continuous
g : [a, b] → R, then f (t) = 0 for all t ∈ [a, b].
[Hint. Find a way to use the previous exercise.]
Exercise 7.2.6. Let f : [a, b] → R be an integrable function (hence bounded). Define F : [a, b] → R
by
Z x
F (x) :=
f (t) dt.
a
Prove that F is Lipschitz (see Exercise 5.5.7) and therefore (uniformly) continuous.
Exercise 7.2.7. Let f : [a, b] → R be an integrable function. Prove that |f | is integrable and that
Z
b
Z
a
7.3
b
|f (t)| dt.
f (t) dt ≤
a
The Fundamental Theorem of Calculus
Theorem 7.3.1 (Fundamental Theorem of Calculus). Let f : [a, b] → R be an integrable function.
Define F : [a, b] → R by
Z x
F (x) :=
f (t) dt.
a
For any x ∈ [a, b], if f is continuous at x then F is differentiable at x and
F 0 (x) = f (x).
87
Proof. Fix x ∈ [a, b] such that f is continuous at x. We will show that (if x < b) lim+
y→x
F (y)−F (x)
y−x
= f (x)
using the δ- definition of limit (with ≤ in place of <). A similar argument will show that (if x > a)
(x)
lim− F (y)−F
= f (x), and thus by Exercise 5.1.9,
y−x
y→x
F (y) − F (x)
= f (x).
y→x
y−x
Ry
Note that by the Additive Property, F (y) − F (x) = x f .
Given > 0, choose δ > 0 such that if t ∈ [a, b] and |t − x| ≤ δ then |f (t) − f (x)| ≤ . Let
y ∈ (x, x + δ]. Then for t ∈ [x, y], we have
F 0 (x) = lim
f (x) − ≤ f (t) ≤ f (x) + .
Hence by Corollary 7.2.4 (applied to f |[x,y] ),
y
Z
f ≤ (f (x) + )(y − x),
(f (x) − )(y − x) ≤
x
so that
f (x) − ≤
This proves that lim+
y→x
F (y)−F (x)
y−x
F (y) − F (x)
≤ f (x) + .
y−x
= f (x), as required.
The following is the form in which the Fundamental Theorem of Calculus is typically used: it
enables computing integrals via antidifferentiation.
Theorem 7.3.2. Let F : [a, b] → R be a differentiable function, such that F 0 : [a, b] → R is
continuous. Then
Z b
F 0 (t) dt = F (b) − F (a).
a
Proof. Define G : [a, b] → R by
Z
x
G(x) :=
F 0 (t) dt.
a
Then by the Fundamental Theorem of Calculus, G0 = F 0 . It follows from Exercise 6.4.3 that G − F
is a constant function, i.e.,
F (x) = G(x) + C
for some C ∈ R. Using x = a, we find
F (a) = G(a) + C = 0 + C,
so that C = F (a). Hence,
Z
F (b) − F (a) = G(b) =
b
F 0,
a
as required.
Interesting fact 7.3.3. The above theorem can be generalized somewhat: instead of assuming that F 0
is continuous, it suffices to assume that F 0 exists (i.e., F is differentiable). The proof for this case is
a bit longer (and can’t be reduced to Theorem 7.3.1).
88
Exercise 7.3.1. [Sav17, Exercise 7.3.1] Let f : [a, b] → R be continuous and define F : [a, b] → R by
Z b
f (t) dt.
F (x) :=
x
Prove that F is differentiable and F 0 (x) = −f (x) for all x ∈ [a, b].
Exercise 7.3.2. [Leb16, Exercise 5.3.2] Define G : R → R by G(x) :=
R x2
0
sin(t2 ) dt. Compute G0 (x).
Exercise 7.3.3. [Sav17, Exercise 7.3.6] Let f : [a, b] → R be continuous, where a < b. Prove that if
Rx
Rb
f (t) dt = x f (t) dt for all x ∈ [a, b], then f (t) = 0 for all t ∈ [a, b].
a
Exercise 7.3.4. Prove the Integration by Parts formula: if f, g : [a, b] → R are differentiable functions
for which f 0 , g 0 : [a, b] → R are continuous, then
Z b
Z b
0
f (t)g (t) dt = f (b)g(b) − f (a)g(a) −
f 0 (t)g(t) dt.
a
a
Exercise 7.3.5. Prove the Change of Variables formula: if f : [c, d] → R is a continuous function and
g : [a, b] → R is a differentiable function such that g([a, b]) ⊆ [c, d] and g 0 : [a, b] → R is continuous,
then
Z
Z
g(b)
b
f (g(t))g 0 (t) dt =
f (t) dt.
g(a)
a
[Hint. Define F (t) :=
compute F 0 and G0 .]
7.4
Rx
a
f (g(t))g 0 (t) dt and G(t) :=
R g(x)
g(a)
f (t) dt. Use FTC and the Chain Rule to
Improper integrals
So far we have only defined integration for bounded functions on bounded intervals. It is desirable
to extend the theory to unbounded functions and/or unbounded intervals. The approach of approximating by Darboux sums, however, will not work: if f is not bounded above then for every partition
P , Mi (P, f ) will be infinite for some i, and thus
U (P, f ) = ∞.
Instead, to deal with this case, we use limits.
Definition 7.4.1. Let f : (a, b] → R be a function such that, for every x ∈ (a, b], f |[x,b] is Riemann
integrable. Then we define
Z b
Z b
f (t) dt,
f (t) dt := lim+
a
x→a
x
provided that this limit exists. Likewise, if f : [a, b) → R is such that f |[a,x] is integrable for all
x ∈ [a, b), then
Z b
Z x
f (t) dt := lim−
f (t) dt,
a
x→b
a
provided that this limit exists.
Let f : [a, ∞) → R be a function such that, for every x ∈ [a, ∞), f |[a,x] is integrable. Then we
define
Z ∞
Z x
f (t) dt := lim
f (t) dt.
a
x→∞
89
a
Likewise, if f : (−∞, b] → R is a function such that f |[x,b] is integrable for all x ∈ (−∞, b] then
Z b
Z b
f (t) dt := lim
f (t) dt.
x→−∞
−∞
x
Whenever one of the above limits exist, we say that the improper integral converges.
Note that if f : [a, b] → R is integrable then by Exercise 7.2.6,
Z b
Z x
f (t) dt = lim−
f (t) dt
x→b
a
(and similarly, one can prove
Rb
a
Rb
f (t) dt = lim+
x
x→a
a
f (t) dt), so there is no ambiguity in the above
notation.
The above definition allows integrals to be defined where there is unboundedness at one endpoint.
To allow unboundedness at both endpoints, we cut in the middle and combine. For example, suppose
f : (a, ∞) → R is a function for which f |[x,y] is Riemann integrable for all a < x < y. Then we choose
some c ∈ (a, ∞) and define
Z ∞
Z c
Z ∞
f (t) dt :=
f (t) dt +
f (t) dt,
a
a
c
provided both of these exist (using the above definition).
Exercise 7.4.1. [TBB08, Exercise 8.5.3] For what values of p is the integral
may use the Fundamental Theorem of Calculus.)
R∞
1
1
tp
dt convergent? (You
Exercise 7.4.2. [Sav17, Exercise 7.4.5] Suppose that f : [0, ∞) → R is a function such that
converges. Prove that for every > 0 there exists M such that if a > b > M then
Z b
f (t) dt < .
R∞
0
f (t) dt
a
Exercise 7.4.3. Define f : R → R by f (t) := t. Show that the limit
Z x
lim
f (t) dt
x→∞
exists, but explain why
R∞
−∞
−x
f (t) dt doesn’t converge.
Exercise 7.4.4. In this question, we prove the Integral Test (Proposition 3.2.10). Let f : [1, ∞) → R
be an decreasing function.
(i) Prove that for all n ∈ N≥1 ,
N
X
Z
n=2
(ii) Prove that if
R∞
1
N +1
f (n) ≤
f (t) dt converges, then
f (t) dt ≤
1
N
X
f (n).
n=1
∞
P
f (n) converges.
n=1
(iii) Prove that if F : [1, ∞) → R is a bounded increasing function then lim F (x) = sup F ([0, ∞).
x→∞
(iv) Using (c), prove that if
∞
P
f (n) converges then
n=1
90
R∞
1
f (t) dt converges.
Chapter 8
Sequences and series of functions
Our aim in this chapter is to make sense of and analyze infinite sums of functions:
∞
P
fn , where
n=1
fn : X → Rm . Inherent in trying to define such a thing is a notion of limit, and therefore it makes
sense to start with sequences of functions and develop the notion of a limit in that setting. From
there, we define convergence of series of functions in terms of partial sums.
8.1
Pointwise limits
The most obvious way to define a limit of functions is the pointwise limit. We will illustrate with
examples that, unfortunately, pointwise limits do not behave well with respect to calculus operations.
Definition 8.1.1. Let X be a set, let (fn : X → Rm )∞
n=1 be a sequence of functions, and let f : X →
m
∞
R . We say that (fn )n=1 converges pointwise to f if for every x ∈ X,
lim fn (x) = f (x).
n→∞
In this case, we may write
pw− lim fn = f.
n→∞
Example 8.1.2. Define fn : [0, 1] → R by fn (x) := xn . Then for x ∈ [0, 1],
(
0,
x ∈ [0, 1);
lim fn (x) = lim xn =
n→∞
n→∞
1,
x = 1.
Hence, (fn )∞
n=1 converges pointwise to f : [0, 1] → R defined by
(
0,
x ∈ [0, 1);
f (x) :=
1,
x = 1.
Note that each fn is continuous, but the pointwise limit f is not.
n
Example 8.1.3. Define fn : [0, 1] → R by fn (x) := xn . For x ∈ [0, 1], we can see (using the Squeeze
Theorem) that
lim fn (x) = 0,
n→∞
(fn )∞
n=1
so that
converges pointwise to the constant zero function (which we’ll write as g here). How
do the derivatives behave? fn0 is differentiable and fn0 (x) = xn−1 . Hence, (fn0 )∞
n=1 converges pointwise
91
to the function f from the previous example – which is not the same as g 0 (which is also the zero
function). This shows that
0
pw− lim fn 6= pw− lim fn0 .
n→∞
n→∞
Example 8.1.4. Define fn : [0, 1] → R such that
1
fn (0) = 0, fn
= n,
n
2
fn
= 0,
n
fn (1) = 0,
and fn is linear in between.
An explicit formula for fn is

2

n x,
fn (x) := 2n − n2 x,


0,
x ∈ [0, n1 ];
x ∈ [ n1 , n2 ];
x ∈ [ n2 , 1].
Claim. The sequence (fn )∞
n=1 converges pointwise to the zero function g.
To see this, note that for x = 0, we have fn (x) = 0 for all n, and thus lim fn (0) = 0. On the
n→∞
other hand, for x > 0, we have fn (x) = 0 provided n > x2 , and thus (fn (x))∞
n=1 is eventually zero,
which implies its limit is 0.
R1
However, to compute 0 fn (t) dt, note that this gives the area of a triangle with base length n2
and height n, so
Z
1
fn (t) dt = 1.
0
The integral of g = pw− lim fn , however, is 0. This shows that
n→∞
Z
1
Z
lim fn (t) dt 6= lim
0 n→∞
n→∞
1
fn (t) dt.
0
Exercise 8.1.1. Give an example of a sequence of differentiable functions (fn : R → R)∞
n=1 which
converge pointwise to the function f (x) := |x|, which is not differentiable at 0.
Exercise 8.1.2. (cf. [TBB08, Exercise 9.2.1]) Define fn : R → R by fn (x) :=
pointwise limit of the sequence (fn )∞
n=1 .
x2n
.
1+x2n
Determine the
Exercise 8.1.3. [TBB08, Exercise 9.2.3] There is a sequence (xn )∞
n=1 in [0, 1] such that
{xn : n ∈ N≥1 } = Q ∩ [0, 1].
92
(In other words, Q ∩ [0, 1] is countable.) Define fn : [0, 1] → R by
(
x,
x ∈ {x1 , . . . , xn };
fn (x) :=
0,
otherwise,
and
(
x,
f (x) :=
0,
x ∈ Q;
otherwise,
Show that fn converges pointwise to f , and that each fn is integrable. (From Example 7.1.8, we
know that f is not integrable.)
Exercise 8.1.4. [TBB08, Exercise 9.2.7] Let (fn : [a, b] → R)∞
n=1 be a sequence which converges
pointwise to f : [a, b] → R. Which of the following are true:
(a) If each fn is (weakly) increasing then so is f .
(b) If each fn is strictly increasing then so is f .
(c) If each fn is bounded then so is f .
(d) If each fn is constant then so is f .
(e) If fn (x) ≥ 0 for all x ∈ [a, b] then f (x) ≥ 0 for all x ∈ [a, b].
(f) If each fn is linear (meaning, of the form fn (x) = αn x + βn ) then so is f .
(g) If each fn is convex (meaning fn (tx + (1 − t)y) ≤ tfn (x) + (1 − t)fn (y) for all x, y ∈ [a, b] and all
t ∈ [0, 1]) then so is f .
8.2
Uniform convergence
In this section, we define a stronger notion of convergence of a sequence of functions. Using this
notion, we can overcome the bad behaviour with mere pointwise convergence, seen in the last section.
To contextualize the definition of uniform convergence, we unwind the definition of “a sequence
m
(fn : X → Rm )∞
n=1 converges pointwise to f : X → R ” (using the δ- definition of convergence in
Rm ), we get:
For any x ∈ X and > 0 there exists n0 ∈ N≥1 such that for all n ≥ n0 then kfn (x) − f (x)k2 < .
Here, n0 is allowed to depend on both x and (but not, of course, on n). The idea of “uniformizing”
this definition which we will do presently is similar to what we did when we defined uniform continuity
(Section 5.5): we modify the definition so that n0 is not allowed to depend on x.
Definition 8.2.1. Let X be a set, let (fn : X → Rm )∞
n=1 be a sequence of functions, and let f : X →
m
∞
R . We say that (fn )n=1 converges uniformly to f if for every > 0 there exists n0 such that for all
n ≥ n0 and x ∈ X,
kfn (x) − f (x)k2 < .
In this case, we may write
u− lim fn = f.
n→∞
93
By comparing this definition to the characterization of pointwise convergence above, it is clear
that
u− lim fn = f implies pw− lim fn = f.
n→∞
n→∞
We also define uniform convergence of a series of functions.
Definition 8.2.2. Let X be a set, let (fn : X → Rm )∞
n=1 be a sequence of functions, and let f : X →
m
R . We define
N
∞
X
X
fn ,
u−
fn := u− lim
N →∞
n=1
n=1
provided that this limit exists. When this limit exists, we say that the series
∞
P
fn converges uniformly
n=1
(on X).
sin(nx)
.
n
1
n0 > .
Example 8.2.3. Define fn : R → R by fn (x) :=
To see this, let > 0 be given. Then pick
|fn (x) − 0| =
Then u− lim fn = 0 (the zero function).
n→∞
If n ≥ n0 and x ∈ R then
| sin(nx)|
1
≤ < .
n
n
Example 8.2.4. Let b ∈ (0, 1) and define fn : [0, b] → R by fn (x) := xn . Then u− lim fn = 0 (the
n→∞
zero function).
To see this, let > 0 be given. Since lim bn = 0, there exists n0 such that if n ≥ n0 then |bn | < .
n→∞
Now, if n ≥ n0 and x ∈ [0, b] then x ≤ b so that xn ≤ bn , and thus
|fn (x) − 0| = xn < .
n
Example 8.2.5. The sequence of functions (fn : [0, 1] → R)∞
n=1 defined by fn (x) := x does not
converge uniformly to the function g in Example 8.1.2. This will follow from Theorem 8.3.1 below:
if u− lim fn = g then this theorem would tell us that g is continuous, but g is not.
n→∞
In practice, we will (more) often be interested in whether series of functions converge uniformly,
and for this, there is the following useful test.
Theorem 8.2.6 (The Weierstrass M -test). Let X be a set, let (fn : X → R)∞
n=1 be a sequence of
∞
functions, and let (Mn )n=1 be a sequence of non-negative real numbers. Suppose that the following
hold:
(i) |fn (x)| ≤ Mn for all x ∈ X, and
(ii)
∞
P
Mn converges.
n=1
Then
∞
P
fn converges uniformly.
n=1
Proof. For each x ∈ X, by the Comparison Test, the series
∞
X
|fn (x)|
n=1
94
converges, so that
∞
P
fn (x) converges absolutely. Define
n=1
g(x) :=
∞
X
fn (x).
n=1
We will show that g = u−
Let > 0. Since
∞
P
∞
P
fn .
n=1
Mn converges, we can find N0 such that
n=1
∞
X
Mn −
n=1
N0
X
Mn < .
n=1
Then if N ≥ N0 and x ∈ X,
g(x) −
N
X
fn (x) =
n=1
=
≤
≤
∞
X
fn (x) −
n=1
∞
X
N
X
fn (x)
n=1
fn (x)
n=N +1
∞
X
|fn (x)|
n=N +1
∞
X
Mn
n=N +1
< ,
as required.
Exercise 8.2.1. (cf. [TBB08, Exercise 9.3.1]) For n ∈ N≥1 , define fn : R → R by
x2n
.
n→∞ 1 + x2n
fn (x) := lim
∞
Prove that (fn |[b,∞) )∞
n=1 converges uniformly, for any (fixed) b > 1, and that (fn |[−c,c] )n=1 converges
uniformly, for any (fixed) c ∈ (0, 1).
Exercise 8.2.2. [TBB08, Exercise 9.3.3] Let X be a finite set and suppose that (fn : X → Rm )∞
n=1 is
m
a sequence of functions which converge pointwise to f : X → R . Prove that f = u− lim fn .
n→∞
Exercise 8.2.3. For a function f : X → R, define
kf k∞ := sup{|f (x)| : x ∈ X} ∈ [0, ∞].
Prove that a sequence (fn : X → R)∞
n=1 converges uniformly to f : X → R if and only if
lim kfn − f k∞ = 0.
n→∞
∞
Exercise 8.2.4. Let X be a set and let (fn : X → R)∞
n=1 and (gn : X → R)n=1 be sequences which
converge uniformly.
95
(a) [TBB08, Exercise 9.3.9] Prove that (fn + gn )∞
n=1 converges uniformly.
(b) [TBB08, Exercise 9.3.10] Must the sequence (fn gn )∞
n=1 converge uniformly? Give a proof or
counterexample.
Exercise 8.2.5. [TBB08, Exercise 9.3.4] Let X = X1 ∪ X2 , let (fn : X → Rd )∞
n=1 be a sequence of
d
functions, and let f : X → R . Prove that if f |X1 = u− lim fn |X1 and f |X2 = u− lim fn |X1 then
n→∞
n→∞
f = u− lim fn .
n→∞
Exercise 8.2.6. Show that the series
∞
X
sin(nx)
n=1
2n
converges uniformly on R.
Exercise 8.2.7. Let (fn : X → Rm )∞
n=1 be a sequence of functions. Prove that the sequence converges
uniformly if and only if it is “uniformly Cauchy”, which means: for every > 0 there exists n0 such
that for all n, n0 ≥ n0 ,
kfn (x) − fm (x)k2 < .
[Hint. Use the Cauchy Convergence Criterion for Rm to first prove that g = pw−lim fn exists. Then
n→∞
prove that g = u− lim fn by a similar proof to the Cauchy Convergence Criterion.]
n→∞
Exercise 8.2.8. (cf. [TBB08, Exercise 9.3.31]) Let X be a set and (fn : X → Rm )∞
n=1 is a sequence of
∞
functions. Suppose that for every sequence (xn )n=1 in X, we have
lim fn (xn ) = 0.
n→∞
Prove that
u− lim fn = 0 (the zero function).
n→∞
Exercise 8.2.9. Let f : R → R be a function and define a sequence of functions (fn : R → R)∞
n=1 by
fn (x) := f (x +
1
).
n
(a) Prove that if f is continuous then f = pw− lim fn .
n→∞
(b) Prove that if f is uniformly continuous then f = u− lim fn .
n→∞
(c) Give an example of a function f such that (fn )∞
n=1 doesn’t even converge pointwise to f .
8.3
Properties of uniform convergence
Theorem 8.3.1. Let X ⊆ Rd be a set and a ∈ X. Let (fn : X → Rm )∞
n=1 a sequence of functions
m
which converges uniformly to f : X → R . If each fn is continuous at a then so is f .
Hence, if each fn is continuous on X then so is f .
96
Proof. Let > 0. Since f = u− lim fn , there exists n0 such that for all n ≥ n0 and all x ∈ X,
n→∞
kfn (x) − f (x)k2 < .
3
Fix any n ≥ n0 Next, since fn is continuous at a, there exists δ > 0 such that for all x ∈ X, if
kx − ak2 < δ then
kfn (x) − fn (a)k2 < .
3
Now let x ∈ X be a point satisfying kx − ak2 < δ. Then
kf (x) − f (a)k2 = kf (x) − fn (x) + fn (x) − fn (a) + fn (a) − f (a)k2
≤ kf (x) − fn (x)k2 + kfn (x) − fn (a)k2 + kfn (a) − f (a)k2
< + + = ,
3 3 3
as required.
Applying the above theorem to a sequence of partial sums, we get the following.
Corollary 8.3.2. Let X ⊆ Rd and suppose that (fn : X → Rm )∞
n=1 is a sequence of continuous
∞
∞
P
P
functions. If
fn converges uniformly then the function
fn is continuous.
n=1
n=1
Example 8.3.3. Consider the series
∞
X
xn
n=0
If we fix some interval [−b, b] and set Mn :=
bn
,
n!
n!
.
then for every x ∈ [−b, b], we have
xn
≤ Mn .
n!
Since
Mn+1
Mn
→ 0, it follows by the Ratio Test that
∞
P
Mn converges, so by Weierstrass M -test,
n=1
∞
X
xn
n=0
n!
converges uniformly on [−b, b]. It follows that this defines a continuous function on [−b, b]. Since b
∞ n
P
x
is arbitrary, it follows that
is continuous on R.
n!
n=0
Theorem 8.3.4. Let (fn : [a, b] → R)∞
n=1 be a sequence of continuous functions which converges
uniformly to f : [a, b] → R. Then
Z b
Z b
f (t) dt = lim
fn (t) dt.
a
n→∞
a
Remark 8.3.5. In this theorem, we know that each fn and therefore also f is continuous, and hence
they are all Riemann integrable. In Exercise 8.3.4, this theorem is generalized to the case that each fn
is just Riemann integrable (where the main challenge is to show that f is also Riemann integrable).
97
Proof. Define Mn := sup{|fn (t) − f (t)| : t ∈ [a, b]}. This is called kfn − f k∞ in Exercise 8.2.3, and
by that same exercise, we know that
Mn → 0 as n → ∞.
For each n we have
b
Z
0≤
Z
b
f (t) dt −
a
fn (t) dt
a
b
Z
f (t) − fn (t) dt
=
Z
a
b
|f (t) − fn (t)| dt
≤
(Exercise 7.2.7)
a
≤ Mn (a − b).
(Corollary 7.2.4)
By the Squeeze Theorem, it follows that
Z b
Z b
lim
f (t) dt −
fn (t) dt = 0,
n→∞
a
a
as required.
Applying the above theorem to a sequence of partial sums, we get the following.
Corollary 8.3.6. Let (fn : [a, b] → R)∞
n=1 be a sequence of continuous functions. If the series
∞
P
fn
n=1
converges uniformly then
∞ Z
X
n=1
b
fn (t) dt =
Z bX
∞
fn (t) dt.
a n=1
a
Example 8.3.7. Let x ∈ [0, 1). By the Weierstrass M -test (with Mn = xn ), the series
∞
P
tn converges
n=0
uniformly on [0, x]. At each value of t, this is a geometric series, so that we know that
∞
X
tn =
n=0
It follows that
1
.
1−t
∞
∞
X
X
xn+1
xn
t dt =
=
.
n
+
1
n
0
0
n=0
n=1
Rx 1
We also know, from the Fundamental Theorem of Calculus, that 0 1−t = ln(1 − x), so this gives us
an interesting formula:
∞
X
xn
, x ∈ [0, 1).
ln(1 − x) =
n
n=1
Z
x
∞
X
1
dt =
1−t
n=1
Z
x
n
Essentially the same argument also proves this formula for x ∈ (−1, 0].
98
For derivatives, the requirement for uniform convergence is slightly more subtle. In Example 8.1.3,
0 ∞
we had a sequence (fn : [0, 1] → R)∞
n=1 converging pointwise to the zero function, although (fn )n=1
converges to a nonzero function. Looking closely at this example, we see that in fact 0 = u− lim fn ,
n→∞
which tells us that uniform convergence of the fn is not enough to improve the behaviour with respect
to differentiation. Instead, what we need is that the sequence (fn0 )∞
n=1 converges uniformly. We also
ask that the sequence consists of continuously differentiable functions; this isn’t really necessary (see
[TBB08, Theorem 9.37]).
0
Theorem 8.3.8. Let (fn : [a, b] → R)∞
n=1 be a sequence of differentiable functions such that fn
is continuous for each n. Suppose that the sequence (fn0 )∞
n=1 converges uniformly to some function
converges
pointwise
to
f
:
[a, b] → R. Then f is differentiable and
g : [a, b] → R and that (fn )∞
n=1
0
f = g.
Proof. For x ∈ [a, b], we have
f (x) = lim fn (x)
n→∞
Z
x
fn0 (t) dt
= lim fn (a) +
n→∞
Z ax
= f (a) + lim
fn0 (t) dt
n→∞ a
Z x
g(t) dt
= f (a) +
(Theorem 7.3.2)
(Theorem 8.3.4).
a
Hence by the Fundamental Theorem of Calculus (Theorem 7.3.1), f 0 = g.
Using partial sums we get the following.
0
Corollary 8.3.9. Let (fn : [a, b] → R)∞
n=1 be a sequence of differentiable functions such that fn
∞
∞
P
P 0
is continuous for each n, and let f =
fn (converging pointwise). If the series
fn converges
n=1
n=1
uniformly then
0
f =
∞
X
fn0 .
n=1
Example 8.3.10. We already know that
both sides: on the left we get
1
.
(1−x)2
1
1−x
=
∞
P
xn for x ∈ (−1, 1). We would like to differentiate
n=0
On the right, differentiating term-by-term would give
∞
P
nxn−1
n=0
– but we don’t know yet whether (or where) this converges and whether it agrees with the derivative
on the left.
∞
P
Fix some b ∈ (0, 1). We’ll show that
nxn−1 converges uniformly on [−b, b]. Indeed, setting
n=0
Mn := nbn−1 , then for all x ∈ [−b, b] we have
|nxn−1 | ≤ Mn .
Moreover,
Mn+1
Mn
→ b < 1, so by the Ratio Test,
∞
P
Mn converges. Hence by the Weierstrass M -test,
n=0
∞
X
nxn−1
n=0
99
converges uniformly.
We may now apply the above corollary to conclude that
∞
X
1
=
nxn−1 ,
(1 − x)2
n=0
for x ∈ [−b, b].
Since b is arbitrary, it follows that
∞
X
1
=
nxn−1 ,
(1 − x)2
n=0
for x ∈ (−1, 1).
Exercise 8.3.1. [TBB08, Exercise 9.4.1] Does there exist a sequence of discontinuous functions (fn :
[a, b] → R)∞
n=1 which converges uniformly to a continuous function f : [a, b] → R?
Exercise 8.3.2. Let X ⊆ Rd and let (fn : X → Rd )∞
n=1 be a sequence of uniformly continuous functions
which converges uniformly to f : X → Rd . Prove that f is uniformly continuous.
Exercise 8.3.3. [TBB08, Exercise 9.5.2] Prove that
Z
0
∞
∞
πX
X
2
sin(nt)
dt
=
.
2
n
(2n − 1)3
n=1
n=1
Exercise 8.3.4. Let (fn : [a, b] → R)∞
n=1 be a sequence of functions which converges uniformly to
f : [a, b] → R.
(a) Let P be a partition of [a, b], fix n ∈ N≥1 , and let > 0 be such that
> kfn − f k∞ = sup{|fn (x) − f (x)| : x ∈ [a, b]}.
Prove that
mi (P, f ) > mi (P, fn ) − ,
Mi (P, f ) < Mi (P, fn ) + ,
for all i and
L(P, f ) > L(P, fn ) − ,
U (P, f ) < U (P, fn ) + .
(b) Prove that if each fn is Riemann integrable then so is f , and
Z
b
Z
f (t) dt = lim
n→∞
a
b
fn (t) dt.
a
[Exercise 8.2.3 may be handy.]
Exercise 8.3.5. [TBB08, Exercise 9.5.3] Suppose (fn : [a, b] → R)∞
n=1 is a sequence of continuous
functions which converges uniformly to f : [a, b] → R. For each n, define Fn : [a, b] → R by
Z x
Fn (x) :=
fn (t) dt,
a
and define F : [a, b] → R by
Z
x
F (x) :=
f (t) dt.
a
Prove that (Fn )∞
n=1 converges uniformly to F .
100
Exercise 8.3.6. Using the previous exercise, prove that if (fn : [a, b] → R)∞
n=1 is a sequence satisfying
∞
the hypotheses of Theorem 8.3.8, then (fn )n=1 converges uniformly.
Exercise 8.3.7. Define f : R → R by
f (x) := 1 +
x2 x4 x6
+
+
+ ··· .
1!
2!
3!
(a) Verify that the series defining f converges.
(b) [TBB08, Exercise 9.6.2] Prove that
f 0 (x) = 2xf (x),
x ∈ R.
(c) Find a function (defined using a series) g : R → R which satisfies
g 0 (x) = 3x2 g(x),
101
x ∈ R.
Chapter 9
Power series
In this chapter we look at a particularly fruitful way of writing a function as a series. The series we
consider can be considered, in a sense, as polynomials of (possibly) infinite degree. We will make use
of the theory developed in previous chapters to help us do calculus with these series.
9.1
Convergence of a power series
Definition 9.1.1. A power series is a series of the form
∞
X
an (x − c)n ,
n=0
(an )∞
n=1
where
is a sequence of real numbers, c ∈ R, and x is a variable. The numbers a0 , a1 , . . . are
the coefficients of the power series, and c is called the centre of the power series.
If we substitute a real number for x, then either the series converges or it doesn’t; if I is a set
of real numbers at which the series does converge, then we can view the power series as a function
∞
P
I → R. We will, additionally, often view a power series as a series of functions (i.e., as
fn where
n=0
fn : R → R is given by fn (x) = an xn ).
∞
P
Definition 9.1.2. Let
an (x − c)n be a power series. The interval of convergence of this power
n=0
series is the set
{b ∈ R :
∞
X
an (b − c)n converges}.
n=0
It is clear that the interval of convergence always contains the centre, c.
Example 9.1.3. If we take c = 0 and all coefficients equal to 1, we get the power series
∞
X
xn .
n=0
When we substitute a real number for x, we get the Geometric Series (Example 3.1.8). We analyzed
earlier exactly what real numbers this series converges for, and from that analysis we may conclude
that the interval of convergence is (−1, 1). For x in this interval of convergence, we have
∞
X
n=0
xn =
1
.
1−x
102
Example 9.1.4. Consider the power series
∞
P
nn xn . For an x 6= 0, we use the ratio test:
n=0
(n + 1)n+1 xn+1
= (n + 1)|x|
n n xn
n+1
n
n
≥ (n + 1)|x|,
and therefore,
lim
n→∞
(n + 1)n+1 xn+1
= ∞.
n n xn
Consequently, the series does not converge. The interval of convergence of this power series is therefore
{0}.
Theorem 9.1.5. Let
∞
P
an (x − c)n be a power series, and define
n=0
R :=
1
lim sup
p
n
|an |
n→∞
(interpreted as 0 if the lim sup is ∞, and as ∞ if the lim sup is 0). Then for b ∈ R,
(i) if |b − c| < R then
∞
P
an (b − c)n converges, while
n=0
(ii) if |b − c| > R then
∞
P
an (b − c)n diverges.
n=0
Remark 9.1.6. The number R in the above proposition is called the radius of convergence of the
∞
P
power series
an (x − c)n .
n=0
We can rephrase the conclusion of the above proposition in terms of the interval of convergence
of the power series. If R ∈ (0, ∞), it tells us that the interval of convergence is one of
(c − R, c + R),
(c − R, c + R],
[c − R, c + R),
or [c − R, c + R].
If R = 0 then the interval of convergence is {c}, whereas if R = ∞ then the interval of convergence
is R. In particular, in all cases, the interval of convergence is an interval.
Proof. Let b ∈ R, b 6= c (since of course the series converges if b = c). We will use the Root Test
(Proposition 3.2.7). We compute
lim sup
n→∞
p
p
|b − c|
n
|an (b − c)n | = lim sup |b − c| n |an | =
.
R
n→∞
Hence by the Root Test, the series
∞
P
an (b − c)n converges if
n=0
|b−c|
R
< 1, i.e, if |b − c| < R, and it
diverges if |b−c|
> 1, i.e., if |b − c| > R. (Note that these arguments are valid even in the cases R = 0
R
or R = ∞, if we allow |b−c|
to mean ∞ and |b−c|
to mean 0.)
0
∞
As noted in the remark before the proof, if the radius of convergence of
∞
P
an (x−c)n is R ∈ (0, ∞),
n=0
then we can conclude that the interval of convergence is one of (c−R, c+R), (c−R, c+R], [c−R, c+R),
or [c − R, c + R]. We already saw in Example 9.1.3 a situation where we get (c − R, c + R). In the
following examples, we demonstrate that the other cases are possible.
103
Example 9.1.7. Consider the power series
∞
P
n=1
R=
xn
.
n
The radius of convergence is
1
1
p
= = 1.
n
1
lim sup 1/n
n→∞
For the endpoint b = −1, we have that
∞
X
(−1)n
n=1
n
converges by the Alternating Series Test. For the endpoint b − 1, we have that
∞
X
1
n
n=1
is the Harmonic Series, and therefore it diverges. Hence the interval of convergence is [−1, 1).
∞
P
xn
. The radius of convergence is
Example 9.1.8. Consider the power series
n(n+1)
n=1
R=
1
1
p
= 1.
=
1
lim sup n 1/n(n + 1)
n→∞
For the endpoint b = −1, once again we can use the Alternating Series Test to conclude that
∞
X
(−1)n
n(n + 1)
n=1
converges. For the endpoint b = 1, we have that
∞
X
n=1
1
n(n + 1)
converges by Example 3.1.5.
We have seen one example where the radius of convergence is 0 (Example 9.1.4. As a simple
example of the opposite case, any polynomial can be viewed as a power series with infinite radius of
convergence (it is a power series in which only finitely many coefficients are nonzero). There are also
examples with infinitely many nonzero coefficients, such as the following.
∞ n
P
x
. The radius of convergence is
Example 9.1.9. Consider the power series
nn
n=1
R=
1
1
1
p
= = ∞.
=
n
lim sup 1/n
0
lim sup 1/nn
n→∞
n→∞
Hence the interval of convergence is R.
As we saw in Chapter 8, we cannot conclude much about a function just from knowing that it is
a pointwise limit. To advance the theory of power series, we will want to have uniform convergence;
this is the purpose of the next result.
104
∞
P
Proposition 9.1.10. Let
an (x − c)n be a power series and let R be its radius of convergence. Let
n=0
[a, b] be any closed bounded interval contained in (c − R, c + R) (which is R in the case R = ∞).
Then the series converges uniformly on [a, b].
Proof. We use the Weierstrass M -test (Theorem 8.2.6). Set M := max{|a − c|, |b − c|}, so that
M < R. Let Mn := |an M n |. Then using the Root Test as in the previous proof, we know that
∞
P
Mn converges.
n=0
For every x ∈ [a, b], we have |an xn | ≤ Mn , and therefore it follows by the Weierstrass M -test that
the series
∞
X
an x n
n=0
converges uniformly on [a, b].
Interesting fact 9.1.11. The above proposition can be strengthened, to say that for any closed bounded
∞
P
interval [a, b] contained in the interval of convergence, the power series
an (x − c)n converges unin=0
formly on [a, b] ([TBB08, Theorem 10.10]). This makes a difference when the interval of convergence
includes endpoints; for example, in Example 9.1.8, we saw that the power series
∞
X
n=1
xn
n(n + 1)
has interval of convergence [−1, 1]. The series therefore converges uniformly on [−1, 1], although
Proposition 9.1.10 is not sufficient to tell us this.
∞ n
P
x
converges uniformly on [−1, b) for any b ∈ (−1, 1).
Likewise, the series
n
n=1
Exercise 9.1.1. Determine the interval of convergence of the following power series.
(a)
∞
P
n=0
(b)
∞
P
n=0
(c)
∞
P
n=0
(d)
∞
P
n=2
(e)
∞
P
n=1
(f)
∞
P
n2
(x
2n
− 3)n .
2n
(x
n2
+ 3)n .
xn
.
n!
(x−2)n
n ln(n)
(you might use the Integral Test to help sort out convergence at the endpoints).
xn
.
n!
an xn where
n=0
(
1,
an =
−1,
if n = 4k or n = 4k + 1 for some k ∈ N≥0 ;
otherwise.
105
Exercise 9.1.2. Give an example of a power series whose interval of convergence is (0, π].
Exercise 9.1.3. [TBB08, Exercise 10.2.2] Prove that if the limit lim
n→∞
∞
P
n
the radius of convergence of the power series
an (x − c) .
an
an+1
exists, then it is equal to
n=0
[Warning: this limit doesn’t always exist, whereas the lim sup used to define the radius of convergence
does always exist.]
∞
∞
P
P
Exercise 9.1.4. Let
an xn and
bn xn have radii of convergence Ra and Rb respectively.
n=0
n=0
(a) Show that the radius of convergence of
∞
P
(an + bn )xn is at least min{Ra , Rb }.
n=0
(b) Determine the radius of convergence of
∞
P
an xn+1 .
n=0
(c) Determine the radius of convergence of
∞
P
an xkn where k ∈ N≥1 .
n=0
(d) [TBB08, Exercise 10.2.11] If |an | ≤ |bn | for all n, determine what the relationship is between Ra
and Rb .
Exercise 9.1.5. [TBB08, Exercise 10.2.12] Let
R. Determine the radius of convergence of
∞
P
∞
P
an xn be a power series with radius of convergence
n=0
an x2n .
n=0
Exercise 9.1.6. [TBB08, Exercise 10.2.13] Let
∞
P
an xn be a power series with radius of convergence
n=0
R ∈ (0, ∞). Prove that the radius of convergence of
∞
P
2
an xn is 1.
n=0
9.2
Continuity, integration, and differentiation
Theorem 9.2.1. Let
f : I → R by
∞
P
an (x − c)n be a power series with interval of convergence I, and define
n=0
f (x) :=
∞
X
an (x − c)n .
n=0
Then:
(i) f is continuous on I, and
(ii) For any a, b ∈ I,
Z
b
f (t) dt =
a
∞
X
an
(b − c)n+1 − (a − c)n+1 .
n+1
n=0
Proof. We define fn : I → R by fn (x) := an (x − c)n .
106
(i): Let x ∈ I, and first suppose that x − c is less than the radius of convergence. Then we can
find r > 0 such that x ∈ (c − r, c + r). By Proposition 9.1.10,
f |[c−r,c+r]
∞
X
= u−
fn |[c−r,c+r] .
n=0
Therefore, by Theorem 8.3.1, f |[c−r,c+r] is continuous, and so (by Exercise 5.2.2) f is continuous at
x.
If x−c is not less than the radius of convergence, then x is an endpoint of I, and we use Interesting
Fact 9.1.11 to see that the series converges uniformly on a closed interval containing x, and so f is
continuous at x.
(ii): Similarly, using either Proposition 9.1.10 or (if one or both of a, b are endpoints of I)
Interesting Fact 9.1.11, we get that
f |[a,b]
∞
X
= u−
fn |[a,b] ,
n=0
Therefore, by Theorem 8.3.4,
Z
∞ Z
X
b
f (t) dt =
=
n=0
∞
X
=
n=0
∞
P
fn (t) dt
n=0 a
∞ Z b
X
a
Theorem 9.2.2. Let
b
an (t − c)n dt
a
an
(b − c)n+1 − (a − c)n+1 .
n+1
an (x − c)n be a power series with radius of convergence R > 0, and define
n=0
f : (c − R, c + R) → R by
f (x) :=
∞
X
an (x − c)n .
n=0
Then the power series
∞
P
nan (x − c)n−1 also has radius of convergence R, and for x ∈ (c − R, c + R),
n=1
0
f (x) =
∞
X
nan (x − c)n−1 .
n=0
Proof. Let R0 be the radius of convergence of the power series
∞
P
n=1
107
nan (x − c)n−1 .
Then since
lim
n→∞
√
n−1
n = 11
R0 =
1
√
lim sup n−1 nan
n→∞
=
lim sup
1
√
n−1
√
n n−1 an
n→∞
=
1
√
lim sup n−1 an
n→∞
= R.
Now, let x ∈ (c − R, c + R). We may find a closed interval [a, b] contained in (c − R, c + R) such that
x ∈ (a, b).
∞
P
Let fn (x) := an (x − c)n . Using the radius of convergence we just computed, we know that
fn0
converges uniformly on [a, b]. Hence it follows from Theorem 8.3.8 that f 0 =
∞
P
n=0
fn0 on [a, b], and in
n=0
particular, at x.
Corollary 9.2.3. Let
∞
P
an (x − c)n be a power series with radius of convergence R > 0, and define
n=0
f : (c − R, c + R) → R by
f (x) :=
∞
X
an (x − c)n .
n=0
Then for each n,
an =
f (n) (c)
.
n!
Proof. By induction on n. For n = 0, we have
f (0) (c)
f (c)
=
= a0 .
0!
1
Now assume that it is true for n and let us prove it for n + 1. Set g := f 0 , so that
∞
X
g(x) =
(n + 1)an+1 xn .
n=0
Since the radius of convergence of this power series is the same as R > 0, we may apply the inductive
hypothesis to g, which tells us
g (n) (c)
.
(n + 1)an+1 =
n!
Noting that f (n+1) = g (n) , and dividing both sides by n + 1, we get
an+1 =
f (n+1) (c)
f (n+1) (c)
=
.
(n + 1)n!
(n + 1)!
√
n
1
This follows from the fact that rn → ∞ for any r > 1: for if n−1 n > 1 then there would exist r > 1 such that
√
n−1
n > r for all n sufficiently large, and from this, we get rn−1 /n < 1, a contradiction.
108
Exercise 9.2.1. [TBB08, Exercise 10.4.2] Find power series expansions for
x
1 + x2
x
(1 + x2 )2
and
(centred at 0).
∞
P
Exercise 9.2.2. (cf. [TBB08, Exercise 10.4.6]) Let
an xn be a power series with radius of convergence
n=0
R > 0, and define f : (−R, R) → R by
f (x) :=
∞
X
an x n .
n=0
(a) Prove that if f is even, meaning that f (x) = f (−x) for all x, then an = 0 for all n odd.
(b) Prove that if f is odd , meaning that f (x) = −f (x) for all x, then an = 0 for all n even.
Exercise 9.2.3. In this question, we want to prove that there is a function f : (−1, 1) → R which
satisfies
f (0) = 1, f 0 (x) = f (x2 ).
(a) Suppose that there were such a function, and that we could write it as a power series
f (x) =
∞
X
an x n .
n=0
Find recurrence equations for the coefficients (an )∞
n=0 (i.e., an equation expressing an in terms of
a0 , . . . , an−1 ).
(b) With (an )∞
n=0 as in (a), prove that an ≤ 1 for all n.
(c) With (an )∞
n=0 as in (a), prove that the series
∞
X
an x n
n=0
converges on (−1, 1), and that the function f it defines does satisfy the conditions we began
with.
(d) (Difficult) Compute exactly the radius of convergence of this power series.
9.3
Taylor series
The final corollary of the previous section tells us that if a function f can be “represented” as a
power series centred at c (here by “represented” we mean, that the power series converges to f at
all points in some open ball around c), then we can recover the coefficients of the power series by
taking repeated derivatives of f and evaluating them at c. This suggests the question: when can a
function be represented as a power series (centred at a point c)? Here we develop tools to answer
this question.
109
Definition 9.3.1. Let I ⊆ R be an open interval and f : I → R be a function which is infinitely
differentiable (meaning that f (n) exists for all n). For c ∈ I, the Taylor series of f (centred at c) is
the power series
∞
X
f (n) (c)
(x − c)n .
n!
n=0
For N ∈ N≥0 , the N th Taylor polynomial of f (centred at c) is
PN (x) :=
N
X
f (n) (c)
n=0
n!
(x − c)n .
In other words, the N th Taylor polynomial is the unique polynomial of degree at most N such
that
(i)
PN (c) = f (i) (c) for i = 0, . . . , N.
Theorem 9.3.2 (Lagrange Remainder Theorem). Let f : (a, b) → R be a function for which
f 0 , f (2) , . . . , f (N +1) all exist on (a, b), let c ∈ (a, b), and let PN (x) be the N th Taylor polynomial
of f centred at c. Then for x ∈ (a, b), there exists z between c and x such that
f (x) − PN (x) =
f (N +1) (z)
(x − c)N +1 .
(N + 1)!
Proof. Fix x and assume (without loss of generality) that x > c. Define M :=
f (x)−PN (x)
,
(x−c)N +1
so that
f (x) − PN (x) = M (x − c)N +1 .
We need to prove that M =
f (N +1) (z)
(N +1)!
for some z ∈ (x, c). Define g : (a, b) → R by
g(t) := f (t) − PN (t) − M (t − c)N +1 .
(i)
For i = 0, . . . , N , we have f (i) (c) = PN (c), while the ith derivative of M (t − c)N +1 , evaluated at c, is
0. Hence,
g (i) (c) = 0.
We also have g(x) = 0 (by definition of M ) so that we may apply Rolle’s Theorem to g on [c, x], to
obtain z1 such that
g 0 (z1 ) = 0.
Now, since g 0 (c) = g 0 (z1 ) = 0, we may apply Rolle’s Theorem to the function g 0 on [c, z1 ] to obtain
z2 such that
g 00 (z2 ) = 0.
We continue in this way, obtaining z1 , . . . , zN +1 with zi ∈ [c, zi−1 ] ⊆ [c, x] such that
g (i) (zi ) = 0 for i = 1, . . . , N + 1.
Now, since PN is a polynomial of degree at most N , its (N + 1)th derivative is zero, whereas the
(N + 1)th derivative of (t − c)N +1 is (N + 1)!. Hence,
0 = g (N +1) (zN +1 )
= f (N +1) (zN +1 ) − M (N + 1)!,
and thus M =
f (N +1) (zN +1 )
,
(N +1)!
as required.
110
Example 9.3.3. Let f : R → R be the function f (x) := sin(x). Then we have for k ∈ N≥0
f (2k) (x) = (−1)k sin(x),
f (2k+1) (x) = (−1)k cos(x),
and from this we see that the Taylor series for f centred at 0 is
∞
X (−1)k x2k+1
x3 x5 x7
x−
+
−
+ ··· =
.
3!
5!
7!
(2k
+
1)!
k=0
Let us use the Lagrange Remainder Theorem to prove that the Taylor series converges to f
everywhere. If we let PN (x) be the N th Taylor polynomial, our task is to prove that lim PN (x) =
N →∞
f (x) for all x ∈ R.
So, fix x ∈ R. For each N , by the Lagrange Remainder Theorem, there exists z between 0 and x
such that
f (N +1) (z) N +1
f (x) − PN (x) =
x
.
(N + 1)!
Since f (N +1) (z) is either ± sin(z) or ± cos(z), we see that |f (N +1) (z)| ≤ 1, and so
|f (x) − PN (x)| ≤
Now, for any x, we have
f (x), as required.
|x|N +1
(N +1)!
|x|N +1
.
(N + 1)!
→ 0 as N → ∞, and therefore by the Squeeze Theorem, PN (x) →
Exercise 9.3.1. Compute the Taylor series for the function f (x) := ex centred at 0, and use the
Lagrange Remainder Theorem to prove that the Taylor series converges to f (x) for all x ∈ R.
Exercise 9.3.2. Fix α ∈ R. For n ∈ N≥1 , define
α(α − 1) · · · (α − n − 1)
α
.
:=
n
n!
Consider the function f : [0, ∞) → R given by f (x) := xα .
(a) Prove that for all n ∈ N≥1 and all x ∈ (0, ∞),
f
(n)
(x) = n!
α
n
1
x 2 −n .
(You may use the “Power Rule” for differentiation.)
(b) Determine the Taylor polynomial Pn (x) and the Taylor series for f centred at 1.
(c) Assuming α > 0, prove that
α
n+1
(x − 1)n+1 xα ,
n+1
1−x
α
|f (x) − Pn (x)| ≤
,
n+1
x
|f (x) − Pn (x)| ≤
111
for x ≥ 1,
for x ≤ 1.
(d) Prove that
α
n
<1
for all α ∈ (0, 1) and all n ∈ N≥1 .
(e) Prove that if α ∈ (0, 1) and x ∈ ( 12 , 1) then the Taylor series for f (centred at 1) is equal to f at
x.
Exercise 9.3.3. Define
(
2
e−1/x ,
x 6= 0;
f (x) :=
0, x = 0.
(a) Prove that for each n, there is a polynomial qn (x) such that
1
(n)
f (x) = qn
f (x),
x
for x 6= 0.
(b) Prove that f is infinitely differentiable and that
f (n) (0) = 0
f (x)
k
x→0 x
for all n. You may use (without proof) that lim
= 0, for any k ∈ N≥1 .
(c) Determine the Taylor series for f centred at 0. What is the radius of convergence of this power
series? For what values of x is f equal to its Taylor series?
112
Review
Items marked ∗ : you definitely won’t be asked to prove these on the final exam.
Chapter 1 – The real numbers
• Should be comfortable with working with R (field structure, order structure, | · |), but don’t need
to know the axioms per se, apart from completeness.
• Important definition: supremum, infimum.
Chapter 2 – Sequences
• Definitions: boundedness, convergence/divergence, increasing/decreasing/monotone, subsequence,
Cauchy sequence, lim sup/lim inf.
• Should be well-versed at -n0 proofs (proving that a sequence converges, proving that a sequence
does not converge, proving some other property assuming that a sequence converges).
• Uniqueness of limits
• If a sequence converges then it is bounded.
• Algebra of limits∗ .
• If an ≤ bn for all n then lim an ≤ lim bn (assuming the limits exist).
n→∞
n→∞
• Squeeze Theorem∗ .
• Every real number is a limit of rationals.
• Monotone convergence criterion.
• If lim an = L then lim ank = L.
n→∞
k→∞
• Every sequence has a monotone subsequence∗ .
• Bolzano–Weierstrass Theorem.
• Cauchy Convergence Criterion∗ .
• lim inf an ≤ lim sup an .
n→∞
n→∞
• lim inf an = lim inf{an , an+1 , an+2 , . . . } (and likewise for lim sup)∗ .
n→∞
n→∞
• lim an = L if and only if lim sup an = lim inf an = L (Theorem 2.7.6).
n→∞
n→∞
n→∞
113
• Some good practice exercises: Exercise 2.4.1, 2.6.1, 2.7.6.
Chapter 3 – Series
• Definition: convergence.
• Examples: harmonic series, geometric series.
• Linearity.
• Order-preserving: if an ≤ bn then
∞
P
an ≤
n=1
•
∞
P
an converges iff
n=1
∞
P
∞
P
bn (assuming both converge).
n=1
an converges.
n=m
• Divergence Test.
• Boundedness Test.
• Comparison Test∗ .
• Absolute Convergence Test.
• Ratio Test.
• Root Test∗ .
• Alternating Series Test.
• Cauchy Convergence Criterion for Series.
• Some good practice exercises: Exercise 3.2.3, 3.2.6.
Chapter 4 – Topology of Rd
• Definition: norms, k · k2 , convergence in Rd , boundedness, open/closed sets, A◦ , A, ∂A, limit
points, (sequential) compactness.
• Cauchy–Schwarz and ∆-inequalities∗ .
(1)
(d)
(i)
• lim (an , . . . , an ) = (L1 , . . . , Ld ) iff lim an = Li for i = 1, . . . , d.
n→∞
n→∞
• Cauchy Convergence Criterion for Rd .
• Bolzano–Weierstrass Theorem for Rd .∗
• B(a; r) is open.
• Permanence properties of open sets, closed sets.
• F is closed if and only if F = F (Proposition 4.3.9).
• R, (0, 1] are not compact. Every finite set is compact. (You should be able to prove these
without using the Heine–Borel Theorem.)
114
• Heine–Borel Theorem∗ .
• Permanence properties of compact sets.
• Some good practice exercises: Exercise 4.3.1, 4.4.6.
Chapter 5 – Continuous functions
• Most of the things in this chapter are done at the general level of functions f : X → Rm , where
X ⊆ Rd . However, the most important thing is to understand these concepts even for functions
f : A → R where A ⊆ R. If you still aren’t comfortable with topology of Rd , then don’t allow
yourself to get bogged down on that aspect; just focus on this case where we work in R.
• Definitions: limit of a function, one-sided limits, continuity, (strictly/weakly) increasing/decreasing,
uniform continuity, infinite limits and limits at ∞.
• Uniqueness of the limit.
• Sequential Characterization of Limits∗ .
• Algebra of Limits, Squeeze Theorem.
• Composition of continuous functions is continuous.
• Algebra of continuous functions (Proposition 5.2.6).
• If K is compact and f is continuous then f (K) is compact.
• Extreme Value Theorem.
• Intermediate Value Theorem.
• If I is an interval and f : I → R is continuous and injective, then it is strictly increasing or
strictly decreasing, and its inverse is continuous∗ .
• If K is compact and f : K → Rd is continuous then it is uniformly continuous.
• Some good practice exercises: Exercise 5.1.7, 5.3.3, 5.5.5.
Chapter 6 – Differentiation
• Definitions: the derivative/differentiability, local minima/maxima.
• If f is differentiable at a then f is continuous at a.
• Linearity, Product Rule, Chain Rule∗ , Inverse Rule.
• If f is differentiable at a and has a local min/max at a then f 0 (a) = 0.
• Rolle’s Theorem.
• Mean Value Theorem (you don’t need to know Cauchy’s Mean Value Theorem)∗ .
• Some good practice exercises: Exercise 6.1.5, 6.4.3.
115
Chapter 7 – Integration
• Definitions: partition, mi (P, f ), Mi (P, f ), upper/lower Darboux sums, integrable/integral, improper integrals.
• Examples: a function that is constant except at one point, a function that is not Riemann
integrable.
• L(P, f ) ≤ U (P 0 , f ) for any partitions, P, P 0 .∗
• f is integrable iff ∀ > 0 ∃ a partition P such that U (P, f ) − L(P, f ) < .
• If f is continuous then f is integrable.
• Additive Property.
• Linearity.
• If f (x) ≤ g(x) for all x then
Rb
a
f≤
Rb
a
g (assuming both exist).
• Fundamental Theorem of Calculus and its consequence (Theorem 7.3.2).
• Some good practice exercises: Exercise 7.1.6. 7.2.1, 7.2.7
Chapter 8 – Sequences and series of functions
• Definitions: pointwise convergence, uniform convergence.
• Examples: sequences (fn )∞
n=1 that converge pointwise to f such that:
– Each fn is continuous but f is not.
– f 0 6= lim fn0 .
n→∞
Rb
Rb
– a f 6= lim a fn .
n→∞
• The Weierstrass M -test.
• If each fn is continuous then so is u− lim fn .
n→∞
• If each fn is integrable then
Rb
a
u− lim fn = lim
n→∞
Rb
n→∞ a
fn .
• If each fn is differentiable and (fn0 ) converges uniformly then ( lim fn )0 = u− lim fn0 .∗
n→∞
n→∞
• Some good practice exercises: Exercise 8.2.9, 8.3.3, 8.3.7.
Chapter 9 – Power series
• Definitions: power series, interval of convergence, radius of convergence, Taylor series/polynomial.
• A power series converges on (c − R, c + R) and diverges on R \ [c − R, c + R] (where c is the
centre and R is the radius of convergence).
• A power series converges uniformly on [a, b] for any [a, b] ⊆ (c − R, c + R).
• Continuity, integration, and differentiation of power series.
• Lagrange Remainder Theorem∗ .
• Some good practice exercises: Exercise 9.1.2, 9.2.2, 9.3.1, 9.3.3.
116
Index
convergent sequence, 9
convergent series, 25
converges uniformly, 93
M -test, 94
Absolute Convergence Test, 31
absolute value, 6
accumulation point, 46
addition (field), 1
Algebra of Limits, 53
Alternating Series Test, 35
Archimedean Property, 5
Darboux sum, 77
decreasing function, 60
decreasing sequence, 15
derivative, 67
differentiable function, 67
Dirichlet function, 57
distance between real numbers, 6
Divergence Test, 28
divergence to ±∞ (sequence), 10
divergent sequence, 9
divergent series, 25
dot product, 39
bijective function, 60
Bolzano–Weierstrass Theorem, 18
Bolzano–Weierstrass Theorem for Rd , 43
boundary (of a set), 46
boundary point, 46
bounded sequence, 8
bounded set, 3
bounded sets, sequences in Rd , 42
Boundedness Test, 29
Euclidean norm, 39
even function, 109
eventual upper/lower bound, 20
Extreme Value Theorem, 58
Cauchy, 42
Cauchy Convergence Criterion, 19
Cauchy Convergence Criterion (for series), 38
Cauchy Convergence Criterion for Rd , 42
Cauchy sequence, 19
Cauchy’s Mean Value Theorem, 74
Cauchy–Schwarz Inequality, 40
centre (of a power series), 102
Chain Rule, 70
Change of Variables (integration), 89
closed set, 44
closure (of a set), 46
compact set, 48, 49
Comparison Test, 30
complete (ordered field), 4
component functions, 53
continuity (at a point), 55
continuous function (globally), 57
convergence in Rd , 41
field, 1
fixed point, 59
Fundamental Theorem of Calculus, 87
geometric sequence, 14
Geometric Series, 102
geometric series, 28
greatest lower bound, 4
harmonic series, 27
Heine–Borel Theorem, 49
improper integral, 90
increasing function, 60
increasing sequence, 15
infimum, 4
injective function, 60
integral, 80
117
Integral Test, 36, 90
Integration by Parts, 89
interior (of a set), 46
interior point, 46
Intermediate Value Theorem, 58
interval of convergence, 102
inverse function, 60
Inverse Rule, 71
isolated point, 46
partition, 77
periodic function, 59
pointwise convergence, 91
power series, 102
preimage, 57
Product Rule, 70
Quotient Rule, 71
radius of convergence, 103
Ratio Test, 32
Riemann integrable function, 80
Riemann sum, 85
Rolle’s Theorem, 74
Root Test, 34
L’Hôpital’s Rule, 76
Lagrange Remainder Theorem, 110
least upper bound, 3
liminf, 20
limit in Rd , 41
limit inferior, 20
limit of a function, 51
limit of a sequence, 9
limit point, 46
limit superior, 20
limsup, 20
Lipschitz function, 64
local maximum, 72
local minimum, 72
lower bound, 3
lower Darboux sum, 77
sequence, 8
sequence in Rd , 41
Sequential Characterization of Limits, 52
sequentially compact set, 48
series, 25
Squeeze Theorem (for a function), 53
Squeeze Theorem (for sequences), 13
subsequence, 17
supremum, 3
surjective function, 60
maximum, 3
Mean Value Theorem, 75
metric space, 50
minimum, 3
Monotone Convergence Criterion, 16
monotone sequence, 16
multiplication (field), 1
Taylor polynomial, 110
Taylor series, 110
telescoping series, 27
topological space, 50
total order, 2
triangle inequality (for R), 6, 7
triangle inequality (for a norm), 39
norm, 39
uniformly continuous, 62
upper bound, 3
upper Darboux sum, 77
odd function, 109
one-to-one, 60
onto, 60
open ball, 43
open set, 44
ordered field, 2
vector space, 39
Weierstrass M -test, 94
118
Bibliography
[Leb16] Jiřı̀ Lebl. Basic analysis: introduction to real analysis (version 4.0). 2016. Available online
at http://www.jirka.org/ra/.
[Sav17]
Alistair Savage. Elementary real analysis. MAT2125, Winter 2017 course notes. 2017.
http://alistairsavage.ca/mat2125/notes/MAT2125-Elementary Real Analysis.pdf.
[TBB08] Brian S. Thomson, Judith B. Bruckner, and Andrew M. Bruckner. Elementary real analysis,
second edition. 2008. Available online at http://classicalrealanalysis.info/Elementary-RealAnalysis.php.
119