Lecture Unit 1.1: The vector space Rp , inner product
and Euclidean norm
Charles Maepa
Department of Mathematics and Applied Mathematics
University of Pretoria
In dealing with mathematical problems, specialization plays, as I
believe, a still more important part than generalization. Perhaps in
most cases where we seek in vain the answer to a question, the cause
of the failure lies in the fact that problems simpler and easier than the
one in hand have been either not at all or incompletely solved. All
depends, then, on finding out these easier problems, and on solving
them by means of devices as perfect as possible and of concepts
capable of generalization. - David Hilbert
In dealing with mathematical problems, spec
Charles Maepa (University of Pretoria)
Lecture Unit 1.1: The vector space Rp , inner product
1 / 21 and Euclidean norm
Outline
1
A vector space
Definitions and remarks
2
An inner product space
Definitions and remarks
3
An Euclidean space
Definitions and remarks
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
2 / 21 and Euclidean norm
Definitions
Definition (Vector, equality)
Let p ∈ N = Z+ .
1. An ordered set ⟨x1 , . . . , xp ⟩ of p real numbers is called a
p-dimensional vector or a vector with p components. Denote by Rp
the set of all p-dimensional vectors. That is,
Rp = {⟨x1 , . . . , xp ⟩ : x1 , . . . , xp ∈ R}.
Notation: Denote vectors by barred lower case letters, such as
x̄ = ⟨x1 , x2 , . . . , xp ⟩, and call the real numbers x1 , . . . , xp the
components of the vector x̄.
2. The vectors x̄ = ⟨x1 , . . . , xp ⟩ and ȳ = ⟨y1 , . . . , yp ⟩ are said to be
equal if xi = yi for all i = 1, 2, . . . , p.
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
3 / 21 and Euclidean norm
Definitions
Definition (Sum, scalar multiplication)
3. Given vectors x̄, ȳ ∈ Rp and a real number α, the sum of x̄ and ȳ is
defined by
x̄ + ȳ
= ⟨x1 , x2 , . . . , xp ⟩ + ⟨y1 , . . . , yp ⟩
= ⟨x1 + y1 , x2 + y2 , . . . , xp + yp ⟩
and the scalar multiple of x̄ with α is defined by
αx̄ = ⟨αx1 , αx2 , . . . , αxp ⟩.
4. Let x̄ and ȳ be vectors in Rp .
(1) The zero vector in Rp is the vector 0̄ = ⟨0, . . . , 0⟩ with all p
components equal to 0.
(2) The negative of x̄ is the vector −x̄ = ⟨−x1 , . . . , −xp ⟩.
(3) The difference of x̄ and ȳ is the vector x̄ − ȳ =
x̄ + (−ȳ ).
In dealing with mathematical problems, spec
Charles Maepa (University of Pretoria)
Lecture Unit 1.1: The vector space Rp , inner product
4 / 21 and Euclidean norm
• Vector addition and scalar multiplication satisfy the following properties:
Theorem 1 (Properties of vector addition and scalar multiplication)
Given x̄, ȳ , z̄ ∈ Rp and α, β ∈ R, we have
(1) x̄ + ȳ = ȳ + x̄.
(2) x̄ + (ȳ + z̄) = (x̄ + ȳ ) + z̄.
(3) α(β x̄) = (αβ)x̄.
(4) α(x̄ + ȳ ) = αx̄ + αȳ .
(5) (α + β)x̄ = αx̄ + β x̄.
(6) x̄ + 0̄ = x̄.
(7) x̄ + (−x̄) = 0̄.
(8) 0x̄ = 0̄.
(9) 1x̄ = x̄.
Proof.
Exercise 1.1 − 1 of the Notes.
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
5 / 21 and Euclidean norm
• Given sets S, T and V , a binary operation is a rule
∗ : S × T −→ V
∗(s, t) 7→ s ∗ t.
In particular, if S = T = V , then ∗ is a binary operation on V .
5. Definition (Abstract definition of a vector space)
A vector space is a set V (whose elements are called vectors) equipped
with two binary operations, called vector addition and scalar
multiplication such that
(A1) x̄ + ȳ = ȳ + x̄, ∀x̄, ȳ ∈ V ;
(A2) (x̄ + ȳ ) + z̄ = x̄ + (ȳ + z̄) ∀x̄, ȳ , z̄ ∈ V ;
(A3) There exists an element 0̄ in V such that 0̄ + x̄ = x̄ and x̄ + 0̄ = x̄ for
all x̄ in V ;
(A4) Given x̄ in V there is an element −x̄ in V such that x̄ + (−x̄) = 0̄
and (−x̄) + x̄ = 0̄;
(M1) 1x̄ = x̄ ∀x̄ ∈ V .
(M2) a(bx̄) = (ab)x̄ ∀a, b ∈ R and x̄ ∈ V ;
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
6 / 21 and Euclidean norm
5. Definition (Continued)
(D) a(x̄ + ȳ ) = (ax̄ + aȳ ) and (a + b)x̄ = ax̄ + bx̄ ∀a, b ∈ R and
∀x̄, ȳ ∈ V .
• Therefore Rp is a vector space by the above Theorem 1.
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
7 / 21 and Euclidean norm
Homework 1
NB: Students must convince themselves that they can answer the
following questions satisfactorily. Feedback is given to attempted
solutions only and not to the “I don’t know statements”. The
questions are examinable material.
1. Let S be any set and let RS denote the collection of all functions u
with domain S and range in R. That is,
RS = {u|u : S −→ R}.
Define u + v and au by
(u + v )(s) = u(s) + v (s)
(au)(s) = au(s)
for all s ∈ S. Show that RS is a vector space under these operations.
[Define 0, −u ∈ RS by 0(s) = 0, ∀s ∈ S, and
(−u)(s) = −u(s), ∀u ∈ RS and ∀s ∈ S.]
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
8 / 21 and Euclidean norm
Homework 1
2. Let S = {1, 2, 3, . . . , p} for some p ∈ N. Show that the vector space
RS is “essentially the same ” as the space Rp .
[Hint: Since RS and Rp are vector spaces (by (1) and Theorem 1
above), show that the function
Φ : RS −→ Rp
u 7→ ⟨u(1), u(2), . . . , u(p)⟩ = ⟨u1 , u2 , . . . , up ⟩
is
(i) a bijection, and
(ii) linear; that is, for all u, v ∈ RS and α, β ∈ R, we have
Φ(αu + βv ) = αΦ(u) + βΦ(v ).
Then RS and Rp are (linearly) isomorphic vector spaces - this is
what is meant by “essentially the same”.]
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
9 / 21 and Euclidean norm
6. Definition
Let x̄ and ȳ be vectors in Rp . The inner product of x̄ and ȳ is
x̄ • ȳ =
p
X
xi yi .
i=1
Notation: While we’ll consistently use the given notation x̄ • ȳ , different
notations are used by some texts such as (x̄, ȳ ) = x̄ • ȳ or ⟨x̄, ȳ ⟩ = x̄ • ȳ .
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
10 / 21 and Euclidean norm
The inner product satisfies the following properties:
Theorem 2 (Properties of the inner product
Let x̄, ȳ and z̄ be vectors in Rp and α and β be real numbers. Then
(1) x̄ • x̄ ≥ 0 and x̄ • x̄ = 0 if and only if x̄ = 0̄.
(2) x̄ • ȳ = ȳ • x̄.
(3) (αx̄ + β ȳ ) • z̄ = α(x̄ • z̄) + β(ȳ • z̄).
• These properties can be listed using the alternative notations cited
above. • The concept of an inner product can be abstracted by
seeing x̄ • ȳ as the function value •(⟨x̄, ȳ ⟩) = x̄ • ȳ , where
• : V × V −→ R is a function as defined. By Theorem 2, ∗ := •
defines an inner product on V = Rp .
• We may say, in general:
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
11 / 21 and Euclidean norm
7. Definition (Abstract definition of an inner product space)
An inner product (or dot product or scalar product) in a vector space V
is a function
∗ : V × V −→ R
(x̄, ȳ ) 7→ x̄ ∗ ȳ
(here, ∗ is meant to suggest that an inner product need not be
defined by a finite sum!) satisfying the following properties:
(i) x̄ ∗ x̄ ≥ 0 ∀x̄ ∈ V ;
(ii) x̄ ∗ x̄ = 0 ⇐⇒ x̄ = 0̄ ∀x̄ ∈ V ;
(iii) x̄ ∗ ȳ = ȳ ∗ x̄
∀x̄, ȳ ∈ V ;
(iv) x̄ ∗ (ȳ + z̄) = x̄ ∗ ȳ + x̄ ∗ z̄ and (x̄ + ȳ ) ∗ z̄ = x̄ ∗ z̄ + ȳ ∗ z̄ ∀x̄, ȳ , z̄ ∈ V ;
(v) (ax̄) ∗ ȳ = a(x̄ ∗ ȳ ) = x̄ ∗ (aȳ ) ∀a ∈ R ∀x̄, ȳ ∈ V .
• A vector space in which an inner product has been defined is called
an inner product space.
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
12 / 21 and Euclidean norm
Example 1
If w1 and w2 are strictly positive, show that the definition
_______
⟨x1 , x2 ⟩ • ⟨y1 , y2 ⟩ = x1 y1 w1 + x2 y2 w2 ,
yields an inner product on R2 . Generalize this to Rp .
Solution. Complete it yourself.
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
13 / 21 and Euclidean norm
Theorem 3 (Cauchy-Schwarz Inequality)
Given x̄, ȳ ∈ Rp we have
|x̄ • ȳ | ≤
√
√
x̄ • x̄ ȳ • ȳ .
Proof.
Exercise 1.2 − 2. of the Notes.
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
14 / 21 and Euclidean norm
• The inner product can always be used to define a norm in a very natural
way.
6. Definition
Let x̄ be√a vector in Rp . The Euclidean norm of a vector x̄ ∈ Rp is
∥x̄∥2 = x̄ • x̄.
• Note: The Euclidean norm defines a function from Rp into positive real
numbers R+ . In fact,
∥ · ∥2 : Rp −→ R
x̄ 7→ ∥x̄∥2 .
• In terms of the Euclidean norm ∥ · ∥2 , the Cauchy-Schwarz Inequality
takes the familiar form
|x̄ • ȳ | ≤ ∥x̄∥2 ∥ȳ ∥2 .
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
15 / 21 and Euclidean norm
Theorem 3 (Properties of the Euclidean norm)
Given x̄, ȳ ∈ Rp , and α in R, we have
(1) ∥x̄∥2 ≥ 0 and ∥x̄∥2 = 0 if and only if x̄ = 0̄.
(2) ∥αx̄∥2 = |α|∥x̄∥2 .
(3) ∥x̄ + ȳ ∥2 ≤ ∥x̄∥2 + ∥ȳ ∥2 .
Proof.
Exercise 1.2 − 3. of the Notes.
A vector space on which the Euclidean norm has been defined is said to be
the Euclidean normed space.
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
16 / 21 and Euclidean norm
Example 2
Show that, if ȳ ̸= 0̄, then the equality in the Cauchy-Schwarz
Inequality can hold if and only if there is a real number c such that
x̄ = c ȳ .
Solution. ⇒: Assume that ȳ ̸= 0̄ and that √
the equality holds in the
√
Cauchy-Schwarz Inequality. Then |x̄ • ȳ | = x̄ • x̄ ȳ • ȳ . Hence
√
√
|x̄ • ȳ |2 = ( x̄ • x̄ ȳ • ȳ )2
⇔ (x̄ • ȳ )(x̄ • ȳ ) = (x̄ • x̄)(ȳ • ȳ )
(x̄ • ȳ)(x̄ • ȳ)
= x̄ • x̄ (since ȳ ̸= 0̄.)
⇔
(ȳ • ȳ)
Put c =
x̄•ȳ
ȳ •ȳ ,
and consider
∥x̄ − c ȳ ∥22 = (x̄ − c ȳ ) • (x̄ − c ȳ )
= x̄ • x̄ − c x̄ • ȳ − c ȳ • x̄ + c 2 ȳ • ȳ
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
17 / 21 and Euclidean norm
Example 2
Solution (Continued).
n x̄ • ȳ o
n x̄ • ȳ o2
n x̄ • ȳ o
ȳ • ȳ
(x̄ • ȳ ) −
(ȳ • x̄) +
ȳ • ȳ
ȳ • ȳ
ȳ • ȳ
(and using x̄ • ȳ = ȳ • x̄)
= x̄ • x̄ −
= 0
Therefore
∥x̄ − c ȳ ∥2 = 0 ⇔ x̄ − c ȳ = 0̄
x̄ = c ȳ .
⇐: Conversely, assume that x̄ = c ȳ for some c ∈ R. Then
∥x̄∥2 = |c|∥ȳ ∥2 . Hence
|x̄ • ȳ | = |(c ȳ ) • ȳ |
= |c|∥ȳ ∥22
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
18 / 21 and Euclidean norm
Example 2
Solution (Continued).
n ∥x̄∥ o
2
∥ȳ ∥22
∥ȳ ∥2
= ∥x̄∥2 ∥ȳ ∥2 ,
=
(since ȳ ̸= 0̄)
and so equality holds in the Cauchy-Schwarz Inequality.
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
19 / 21 and Euclidean norm
Homework 2
1. Deduce from the Cauchy-Schwarz Inequality that
x̄ • ȳ ≤ ∥x̄∥2 ∥ȳ ∥2 . . . . . . (∗)
or show directly that this is so.
2. Then, if x̄ and ȳ are non-zero, show that the equality holds in (1)(∗)
if and only if there is some strictly positive real number c such that
x̄ = c ȳ .
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
20 / 21 and Euclidean norm
For Further Reading I
WTW 310 Notes,
Department of Mathematics and Applied Mathematics, University of
Pretoria, 2015
Real Analysis,
Departement Wiskunde en Toegepaste Wiskunde, Universiteit van
Pretoria, 2016
R.G. Bartle
The elements of real analysis.
Wiley & Sons, New York, 1976.
Charles Maepa (University of Pretoria)
In dealing with mathematical problems, spec
Lecture Unit 1.1: The vector space Rp , inner product
21 / 21 and Euclidean norm