1.
Mathematical Preliminaries
1.1. Sets
. A set is a well-defined collection of elements.
Example: A = {a, b, c} is a set with three elements.
a  A means that the element a is in the set A
d  A means that the element d is not in the set A.
.  denotes the null set or empty set, i.e., the set with no element.
. set A is a subset of set B if a  B for all a  A. This is denoted by A  B, or B  A.
. union of sets: A  B = {x: x  A or x  B}.
. intersection of sets: A  B = {x: x  A and x  B}.
. A and B are disjoint sets if A  B = .
. the complement of set A in set B is the set B\A = {x: x  B and x  A}.
. Cartesian product: Let x  X and y  Y, and let (x, y) be an ordered pair. Then (x, y)  XY,
where XY is the Cartesian product of X and Y.
1.2. Logic
Consider two propositions P and Q.
If P implies Q, then P is a sufficient condition for Q, and Q is a necessary condition for P. This
is denoted by P  Q.
If P implies Q and Q implies P, then “P holds if and only if Q holds”, P and Q are equivalent, and
P is a necessary and sufficient condition for Q. This is denoted by P  Q.
Let {not P} denote the statement that P is not true.
Contrapositive: If P implies Q, then {not Q} implies {not P}.
Example 1: Let x be a real number, P = {x > 0} and Q = {x2 > 0}. Then, P  Q and {not Q}
{not P} hold, but {Q  P} is false.
Example 2: Let P = {x > 0} and Q = {x3 > 0}. Then P  Q, and {not P}  {not Q} hold.
1.3. Numbers
. natural numbers: N = {1, 2, 3, ...}
. integers: Z = {..., -3, -2, -1, 0, 1, 2, 3, ...}
. rational numbers: Q = {a/b: a  Z, b  Z, b  0}
. irrational numbers: 21/2, 31/2, e, , ...
. real numbers: R = {x: x is rational or irrational}
R+ = {x: x  R, x  0}
R++ = {x: x  R, x > 0}
. complex numbers: C = {a + b i: a  R, b  R, i = (-1)1/2}, where a is the real part, and b is the
imaginary part
. the n-fold Cartesian product of R: Rn = R...R = {(x1, x2, ..., xn): xi  R, i = 1, ..., n}, where xi
is the i-th coordinate of x = (x1, x2, ..., xn)
1.4. Functions
Let X and Y be sets.
Definition: f: X  Y is a mapping associating each element of X with an element of Y.
X is the domain of f
f(x) is the image of x under f
2
f(X) = {f(x): x  X} is the image of X under f
. a function: if only one point in Y is associated with each point in X
. a correspondence: if more than one point in Y can be associated with each point in X
. inverse function: x = f-1(y) if and only if y = f(x)
. a function f: X  Y is onto if f(X) = Y.
It means that the equation f(x) = y has at least one solution for each y.
. if f(x) and f-1(y) are both single-valued, then f is one-to-one.
It means that the equation f(x) = y has at most one solution for each y.
. composite function: h = g(f(x)) = g  f is the composition of f with g satisfying f: A  B  C,
g: C  D, g  f: A  D.
1.5. Bounds
Let S  R.
. S is bounded from above (from below) if there exists a  R (b  R) such that x  a (x  b) for
all x  S. Then, a is an upper bound of S, and b is a lower bound of S.
. The least upper bound (lub) or supremum (sup) of S is the upperbound of S such that there does
not exist a smaller upper bound. It is denoted by sup(S).
. The supremum of S is called a maximum (max) of S if sup(S)  S. It is denoted by max(S).
. The greatest lower bound (glb) or infimum (inf) of S is the lower bound of S such that there
does not exist a larger lower bound. It is denoted by inf(S).
. The infimum of S is called a minimum (min) of S if inf(S)  S. It is denoted by min(S).
. Property:
If S  R and S has an upperbound, then S has a supremum.
If S  R and S has a lowerbound, then S has an infimum.
1.6. Vector Space
Consider a set V.
L1- associative law: x + (y + z) = (x + y) + z, for all x, y, z,  V
L2- identity: there exists 0  V such that x + 0 = x for all x  V
L3- inverse: there exists (-x)  V such that x + (-x) = 0 for all x  V
L4- commutative law: x + y = y + x for all x, y  V
L5- associative law: (x) = () x for all ,   R, and for all x  V
L6- identity: there exists 1  V such that 1x = x for all x  V
L7- distributive law: (x + y) = x + y for all   R, and for all x, y  V
L8- distributive law: ( + )x = x + x for all ,   R, and for all x  V
L9- closure: x  V and y  V implies that (x + y)  V
L10- closure: x  V and   R implies that (x)  V.
Definition: A set V is vector space (or linear space) if it satisfies L1-L10. Then x  V is called
a vector.
Examples: Rn, or Cn is each a vector space.
1.7. Norms and distances
Consider a function d(x, y) satisfying:
M1: d(x, y) = 0 if and only if x = y
M2: d(x, y) + d(y, z)  d(z, x)
M3: d(x, y)  0 for all x, y
3
M4: d(x, y) = d(y, x).
Definition: For a given set X, if a function d: XX  R satisfies M1-M4, then:
X is a metric space, denoted by (X, d)
d is a metric
d(x, y) is the distance between points x and y.
Examples:
d1(x, y) = [i (xi - yi)2]1/2 = Euclidian distance, denoted by ||x - y||
d2(x, y) = maxi |xi - yi|
d3(x, y) = i |xi - yi|
Note: Topology consists in studying the properties of sets that are independent of the distance
measure chosen.
Definition: Let V be a vector space. A real value function N: V  R is called a norm on V if:
N(x)  0 for all x  V
N(x) = 0 if and only if x = 0
N(r x) = |r| N(x) for all r  R and x  V, and
N(x + y)  N(x) + N(y) for all x, y  V.
Example: N(x) = d1(x, 0) = [i (xi)2]1/2 = ||x|| is the Euclidian norm of x in R.
Rn, with Euclidian norm and Euclidian metric, is a normed vector space.
Every normed vector space is a metric space with respect to the induced metric defined by d1(x,
y) = ||x - y||.
1.8. Convex Sets
Let X be a vector space (e.g., X = Rn).
Definition: A set S  X is convex if any x, y  S implies that
( x + (1-) y)  S, for all   R, 0    1.
Note: ( x + (1-) y) is called a linear combination of x and y.
Properties:
. Any intersection of convex sets is convex.
. Let Si, i = 1, ..., m, be convex sets in vector space X. Then:
. (iI i Si) = {x: x = i=1,…,m i xi, xiSi, iR, i = 1, …, m} is a convex set.
. (S1S2...Sm) = i=1,…,m (Si) is a convex set.
1.9. Compact Sets
Let S  Rn.
Definition: An open ball about x0  Rn with radius r  R, r > 0, is defined as:
Br(x0) = {x: x  S, d(x, x0) < r},
where d(x, x0) is the Euclidian distance between points x and x0.
4
Definition: An open set S  Rn is a set S such that, for each x  S, there exists an open ball Br(x)
completely contained in S.
. The union of open sets is open.
. A finite intersection of open sets is open.
Definition: The interior of a set S, denoted by int(S), is the union of all open sets contained in S.
. A set S is open if and only if S = int(S).
Definition: A set S  Rn is closed if the set (Rn\S) is open.
. The intersection of closed sets is closed.
. A finite union of closed sets is closed.
Definition: The closure of a set S, denoted by cl(S), is the intersection of all closed sets
containing S.
. A set S is closed if and only if S = cl(S).
Definition: The boundary of a set S  Rn is the set cl(S)cl(Rn/S).
Definition: A set S is bounded if there exists an open ball with a finite radius which contains S.
Definition: A collection of open sets (S)A in a metric space X is said to be an open cover of a
given set S  Rn if S  A S.
The open cover (S)A of S is said to admit a finite subcover if there exists a finite
subcollection (S)F such that S  F S.
Definition 1: A set S  Rn is compact if and only if it is closed and bounded.
Definition 2: A subset S of a metric space X is compact if and only if every open cover of S has a
finite subcover.
Note: The definition 2 of compactness applies to sets in any metric space, while definition 1
applies only to sets in Rn.
1.10.
Sequences
Let (X, d) be a metric space (e.g., X = Rn), and let S  X.
Definition: A sequence {xj: j = 1, ..., } in S converges to y if, for any  > 0, there exists a
positive integer j’ such that j  j’ implies d(y, xj) < .
This is denoted by y = limj {xj}, where y is the limit of {xj}.
Note: It does not follow that y = limj {xj}  S.
Examples of convergent series:
e = limn (1 + 1/n)n, where n  {1, 2, 3, …}
 2.71828…
More generally, for x  R,
5
ex = limn {(1 + x/n)n, where n  {1, 2, 3, …}
This defines the exponential function, ex: R  R++. It satisfies
e0 = 1
ex+y = ex  ey
(ex)y = exy
for x, y  R.
And for y > 0, y = ex  ln(y) = x. This defines the logarithmic function, ln(y): R++  R,
as the inverse function of ex. It satisfies
ln(1) = 0
ln(e) = 1
ln(x  y) = ln(x) + ln(y), and ln(x/y) = ln(x) - ln(y), for x, y  R++
ln(xy) = y  ln(x), for x  R++ , y  R.
Definition: A sequence {xj: j = 1, ..., } in S is a Cauchy sequence if for any  > 0, there exists a
positive integer j’ such that, for any i, j  j’, d(xi, xj) < .
Definition: If every Cauchy sequence in a metric space is also a convergent sequence, then the
metric space is said to be complete.
. A sequence {xj: j = 1, …,} in Rn is a Cauchy sequence if and only if it is a convergent
sequence, i.e. if and only if there is y  Rn such that limj {xj}  y.
By the above definition, this implies that Rn is complete (although not all metric spaces are
complete).
Definition: Let m(j) be an increasing function: m: {1, 2, 3, ...}  {1, 2, 3, ...}, such that m(k+1)
> m(k). Given a sequence {xj: j = 1, 2, ..., }, {xm(j): m = 1, ..., } is a subsequence of
{xj: j = 1, 2, ..., }.
. A set S  Rn is closed if and only if every convergent sequence of points in S converges to a
point in S.
. A set S  Rn is compact if and only if every sequence in S has a convergent subsequence whose
limit is in S.
. A sequence {xj: j = 1, 2, ..., } in Rn converges to y if and only if every subsequence of {xj: j =
1, ..., } converges to y.
. Every bounded sequence contains a convergent subsequence.
Definition: A sequence {xj: j = 1, 2, ..., } is (strictly) increasing if, for all m > n, xm  (>) xn for
all n.
A sequence {xj: j = 1, 2, ..., } is (strictly) decreasing if, for all m > n, xm  (<) xn for all
n
. Let X  R, X  . If X is bounded from above (below), there exists an increasing (decreasing)
sequence in X converging to sup(X) (inf(X)).
Definition: Assume that  are allowed as limits of a sequence.
6
The lim sup of the sequence {xj: j = 1, 2,…} in R is defined as limj {aj: j = 1, 2 …},
where aj = sup{xj, xj+1, xj+2, …}. It is denoted by limj supkj xk, or simply by lim
supj xj.
The lim inf of the sequence {xj: j = 1, 2,…} in R is defined as limj {bj: j = 1, 2 …},
where bj = inf{xj, xj+1, xj+2, …}. It is denoted by limj infkj xk, or simply by lim inf j
xj.
. A sequence xj in R converges to a limit y  R if and only if y = lim supj xj = lim infj xj.