Lecture 2: Convex sets
August 28, 2008
Lecture 2
Outline
• Review basic topology in Rn
• Open Set and Interior
• Closed Set and Closure
• Dual Cone
• Convex set
• Cones
• Affine sets
• Half-Spaces, Hyperplanes, Polyhedra
• Ellipsoids and Norm Cones
• Convex, Conical, and Affine Hulls
• Simplex
• Verifying Convexity
Convex Optimization
1
Lecture 2
Topology Review
Let {xk } be a sequence of vectors in Rn
Def. The sequence {xk } ⊆ Rn converges to a vector x̂ ∈ Rn when
kxk − x̂k tends to 0 as k → ∞
• Notation: When {xk } converges to a vector x̂, we write xk → x̂
• The sequence {xk } converges x̂ ∈ Rn if and only if for each component
i: the i-th components of xk converge to the i-th component of x̂
|xik − x̂i| tends to 0 as k → ∞
Convex Optimization
2
Lecture 2
Open Set and Interior
Let X ⊆ Rn be a nonempty set
Def. The set X is open if for every x ∈ X there is an open ball B(x, r) that
entirely lies in the set X , i.e.,
for each x ∈ X there is r > 0 s.th. for all z with kz − xk < r, we have z ∈ X
Def. A vector x0 is an interior point of the set X , if there is a ball B(x0, r)
contained entirely in the set X
Def. The interior of the set X is the set of all interior points of X , denoted
R
by (X)
R
• How is (X) related to X ?
• Example X = {x ∈ R2 | x1 ≥ 0, x2 > 0}
R
(X) = {x ∈ R2 | x1 > 0, x2 > 0}
R
(S) of a probability simplex S = {x ∈ Rn | x 0, e0x = 1}
R
Th. For a convex set X , the interior (X) is also convex
Convex Optimization
3
Lecture 2
Closed Set
Def. The complement of a given set X ⊆ Rn is the set of all vectors that do
not belong to X :
the complement of X = {x ∈ Rn | x ∈
/ X} = Rn \ X
Def. The set X is closed if its complement Rn \ X is open
• Examples: Rn and ∅ (both are open and closed)
{x ∈ R2 | x1 ≥ 0, x2 > 0} is open or closed?
hyperplane, half-space, simplex, polyhedral sets?
• The intersection of any family of closed set is closed
• The union of a finite family of closed set is closed
• The sum of two closed sets is not necessarily closed
• Example: C1 = {(x1, x2) | x1 = 0, x2 ∈ R}
C2 = {(x1, x2) | x1x2 ≥ 1, x1 ≥ 0}
C1 + C2 is not closed!
• Fact: The sum of a compact set and a closed set is closed
Convex Optimization
4
Lecture 2
Closure
Let X ⊆ Rn be a nonempty set
Def. A vector x̂ is a closure point of a set X if there exists a sequence
{xk } ⊆ X such that xk → x̂
Closure points of X = {(−1)n/n | n = 1, 2, . . .}, X̂ = {1 − x | x ∈ X}?
• Notation: The set of closure points of X is denoted by cl(X)
• What is relation between X and cl(X)?
Th. A set is closed if and only if it contains its closure points, i.e.,
X is closed iff cl(X) ⊂ X
Th. For a convex set, the closure cl(X) is convex
Convex Optimization
5
Lecture 2
Boundary
Let X ⊆ Rn be a nonempty set
Def. The boundary of the set X is the set of closure points for both the set
X and its complement Rn \ X , i.e.,
bd(X) = cl(X) ∩ cl(Rn \ X)
• Example X = {x ∈ Rn | g1(x) ≤ 0, . . . , gm(x) ≤ 0}. Is X closed?
What constitutes the boundary of X ?
Convex Optimization
6
Lecture 2
Dual Cone
Let K be a nonempty cone in Rn
Def. The dual cone of K is the set K ∗ defined by
K ∗ = {z | z 0x ≥ 0 for all x ∈ K}
• The dual cone K ∗ is a closed convex cone even when K is neither closed
nor convex
• Let S be a subspace. Then, S ∗ = S ⊥.
• Let C be a closed convex cone. Then, (C ∗)∗ = C .
• For an arbitrary cone K , we have (K ∗)∗ = cl(conv(K)).
Convex Optimization
7
Lecture 2
Convex set
• A line segment defined by vectors x and y is
the set of points of the form αx + (1 − α)y
for α ∈ [0, 1]
• A set C ⊂ Rn is convex when, with any two vectors x and y that belong
to the set C , the line segment connecting x and y also belongs to C
Convex Optimization
8
Lecture 2
Examples
Which of the following sets are convex?
• The space Rn
• A line through two given vectors x and y
l(x, y) = {z | z = x + t(y − x), t ∈ R}
• A ray defined by a vector x
{z | z = λx, λ ≥ 0}
• The positive orthant {x ∈ Rn | x 0} ( componentwise inequality)
• The set {x ∈ R2 | x1 > 0, x2 ≥ 0}
• The set {x ∈ R2 | x1x2 = 0}
Convex Optimization
9
Lecture 2
Cone
A set C ⊂ Rn is a cone when, with every vector x ∈ C , the ray {λx | λ ≥ 0}
belongs to the set C
• A cone may or may not be convex
• Examples:
{x ∈ Rn | x 0} {x ∈ R2 | x1x2 ≥ 0}
For a two sets C and S , the sum C + S is defined by
C + S = {z | z = x + y, x ∈ C, y ∈ S} (the order does nor matter)
Convex Cone Lemma: A cone C is convex if and only if C + C ⊆ C
Proof: Pick any x and y in C , and any α ∈ [0, 1]. Then, αx and (1 − α)y
belong to C because... . Using C + C ⊆ C , it follows that ... Reverse: Let
C be convex cone, and pick any x, y ∈ C . Consider 1/2(x + y)...
Convex Optimization
10
Lecture 2
Affine Set
A set C ⊂ Rn is a affine when, with every two distinct vectors x, y ∈ C ,
the line {x + t(y − x) | t ∈ R} belongs to the set C
• An affine set is always convex
• A subspace is an affine set
A set C is affine if and only if C is a translated subspace, i.e.,
C = S + x0 for some subspace S and some x0 ∈ C
Dimension of an affine set C is the dimension of the subspace S
Convex Optimization
11
Lecture 2
Hyperplanes and Half-spaces
Hyperplane is a set of the form {x | a0x = b} for a nonzero vector a
Half-space is a set of the form {x | a0x ≤ b} with a nonzero vector a
The vector a is referred to as the normal vector
• A hyperplane in Rn divides the space into two half-spaces
{x | a0x ≤ b} and {x | a0x ≥ b}
• Half-spaces are convex
• Hyperplanes are convex and affine
Convex Optimization
12
Lecture 2
Polyhedral Sets
A polyhedral set is given by finitely many linear inequalities
C = {x | Ax b} where A is an m × n matrix
• Every polyhedral set is convex
• Linear Problem
minimize
c0 x
subject to Bx ≤ b, Dx = d
The constraint set {x | Bx ≤ b, Dx = d} is polyhedral.
Convex Optimization
13
Lecture 2
Ellipsoids
Let A be a square (n × n) matrix.
• A is positive semidefinite when x0Ax ≥ 0 for all x ∈ Rn
• A is positive definite when x0Ax > 0 for all x ∈ Rn, x 6= 0
An ellipsoid is a set of the form
E = {x | (x − x0)0P −1(x − x0) ≤ 1}
where P is symmetric and positive definite
• x0 is the center of the ellipsoid E
• A ball {x | kx − x0k ≤ r} is a special case of the ellipsoid (P = r2I )
• Ellipsoids are convex
Convex Optimization
14
Lecture 2
Norm Cones
A norm cone is the set of the form
C = {(x, t) ∈ Rn × R | kxk ≤ t}
• The norm k · k can be any norm in Rn
• The norm cone for Euclidean norm is also known as ice-cream cone
• Any norm cone is convex
Convex Optimization
15
Lecture 2
Convex and Conical Hulls
A convex combination of vectors x1, . . . , xm is a vector of the form
α 1 x1 + . . . + α m xm
αi ≥ 0 for all i and
Pm
i=1 αi
=1
The convex hull of a set X is the set of all convex combinations of the
vectors in X , denoted conv(X)
A conical combination of vectors x1, . . . , xm is a vector of the form
λ1x1 + . . . + λmxm
with λi ≥ 0 for all i
The conical hull of a set X is the set of all conical combinations of the
vectors in X , denoted by cone(X)
Convex Optimization
16
Lecture 2
Affine Hull
An affine combination of vectors x1, . . . , xm is a vector of the form
t1x1 + . . . + tmxm
with
Pm
i=1 ti
= 1, ti ∈ R for all i
The affine hull of a set X is the set of all affine combinations of the vectors
in X , denoted aff (X)
The dimension of a set X is the dimension of the affine hull of X
dim(X) = dim(aff (X))
Convex Optimization
17
Lecture 2
Simplex
A simplex is a set given as a convex combination of a finite collection of
vectors v0, v1, . . . , vm:
C = conv{v0, v1 . . . , vm}
The dimension of the simplex C is equal to the maximum number of linearly
independent vectors among v1 − v0, . . . , vm − v0.
Examples
• Unit simplex {x ∈ Rn | x 0, e0x ≤ 1}, e = (1, . . . , 1), dim -?
• Probability simplex {x ∈ Rn | x 0, e0x = 1}, dim -?
Convex Optimization
18
Lecture 2
Practical Methods for Establishing Convexity of a Set
Establish the convexity of a given set X
• The set is one of the “recognizable” (simple) convex sets such as
polyhedral, simplex, norm cone, etc
• Prove the convexity by directly applying the definition
For every x, y ∈ X and α ∈ (0, 1), show that αx + (1 − α)y is also in X
• Show that the set is obtained from one of the simple convex sets through
an operation that preserves convexity
Convex Optimization
19
Lecture 2
Operations Preserving Convexity
Let C ⊆ Rn, C1 ⊆ Rn, C2 ⊆ Rn, and K ⊆ Rm be convex sets. Then, the
following sets are also convex:
• The intersection C1 ∩ C2 = {x | x ∈ C1 and x ∈ C2}
• The sum C1 + C2 of two convex sets
• The translated set C + a
• The scaled set tC = {tx | x ∈ C} for any t ∈ R
• The Cartesian product C1 × C2 = {(x1, x2) | x1 ∈ C1, x2 ∈ C2}
• The coordinate projection {x1 | (x1, x2) ∈ C for some x2}
• The image AC under a linear transformation A : Rn 7→ Rm:
AC = {y ∈ Rm | y = Ax for some x ∈ C}
• The inverse image A−1K under a linear transformation A : Rn 7→ Rm:
A−1K = {x ∈ Rn | Ax ∈ K}
Convex Optimization
20