IE 8534
1
Lecture 4. The Dual Cone and Dual Problem
Shuzhong Zhang
IE 8534
2
For a convex cone K, its dual cone is defined as
K∗ = {y | ⟨x, y⟩ ≥ 0, ∀ x ∈ K}.
The inner-product can be replaced by ‘xT y’ if the coordinates of the
vectors are given. Clearly, K∗ is always a closed set, regardless if K is
closed or not. In fact, K∗ is also always convex regardless if K is
convex or not.
Shuzhong Zhang
IE 8534
3
In light of the dual cone, the problem of finding a lower bound on the
optimal value for the standard conic optimization problem can be
done as follows. Introduce the Lagrangian multiplier y for the equality
constraint b − Ax = 0, and s for the conic constraint x ∈ K. The
objective becomes
cT x + y T (b − Ax) − sT x = y T b + (c − AT y − s)T x.
This will be a lower bound on the optimal value whenever
c − AT y − s = 0 and s ∈ K∗ . The quest to find the best lower bound
leads to the dual problem for the standard conic optimization problem:
maximize
bT y
subject to
AT y + s = c
(1)
s ∈ K∗ .
Shuzhong Zhang
IE 8534
4
As an example, for the standard SDP problem (primal),
minimize
C •X
subject to
Ai • X = bi , i = 1, ..., m
X ≽ 0,
its dual problem is in the form of
maximize
subject to
∑m
i=1 bi yi
∑m
i=1 yi Ai
+S =C
S ≽ 0.
Shuzhong Zhang
IE 8534
5
The so-called weak duality relationship is immediate, by the
construction of the dual problem. This is shown in the next theorem.
Theorem 1 Let x be feasible for the primal problem, and (y, s) be
feasible to its dual. Then,
bT y ≤ cT x.
Moreover, if bT y = cT x then xT s = 0, and they are both optimal to
their own respective problems.
If a pair of primal-dual feasible solutions x and (y, s) both exist and
they satisfy xT s = 0 then we call them a pair of complementary
optimal solutions.
Shuzhong Zhang
IE 8534
6
A very useful theorem is the following.
Theorem 2 Let x be feasible for the primal problem, and (y, s) be
feasible to its dual, and x ∈ int K and s ∈ int K∗ . Then, the sets of
optimal solutions for both problems are nonempty and compact, and
any pair of primal-dual optimal solutions will be complementary to
each other.
Shuzhong Zhang
IE 8534
7
Let us spend some time now to study the structures of the dual cones.
Let us start with the topological structures. If rel (K) is denoted as the
relative interior of K, then (rel (K))∗ = K∗ . Let the Minkowski
summation of two sets A and B be
A + B = {z | z = x + y with x ∈ A, y ∈ B}.
It is clear that the sum of two convex sets remains convex. However,
the sum of two closed convex sets may not be closed any more.
Shuzhong Zhang
IE 8534
8

−1




Example 1 The set R+  0 
 + SOC(3) is convex but not closed.
1
For instance, the point (0, 1, 0)T is not in the set, but it is in the
closure of the set, since for any ϵ > 0, we may write
 
 


1
ϵ
−1
ϵ +ϵ
  1
 

 1  =  0  +  1 ,
 
 

ϵ
0
1
− 1ϵ
where the second term is in SOC(3).
Shuzhong Zhang
IE 8534
9
However, if A and B are both polyhedra, then A + B remains a
polyhedron, hence closed. It is interesting to note that the
set-summation and the set-intersection operations are a duality pair:
Proposition 1 Suppose that K1 and K2 are convex cones. It holds
that
(K1 + K2 )∗ = K1∗ ∩ K2∗ .
Proof. For y ∈ (K1 + K2 )∗ , it follows that 0 ≤ (x1 + x2 )T y for all
x1 ∈ K1 and x2 ∈ K2 . Since 0 ∈ K2 , it follows that 0 ≤ (x1 )T y for all
x1 ∈ K1 , thus y ∈ K1∗ , and similarly y ∈ K2∗ . Hence, y ∈ K1∗ ∩ K2∗ .
Now, take any y ∈ K1∗ ∩ K2∗ , and any x1 ∈ K1 and x2 ∈ K2 . We have
(x1 + x2 )T y = (x1 )T y + (x2 )T y ≥ 0, and so y ∈ (K1 + K2 )∗ . This
proves (K1 + K2 )∗ = K1∗ ∩ K2∗ as desired.
2
Shuzhong Zhang
IE 8534
10
The following bipolar theorem can be considered as a cornerstone for
convex analysis.
Theorem 3 Suppose that K is a convex cone. Then (K∗ )∗ = cl K.
Proof. For any x ∈ K, by the definition of the dual cone, xT y ≥ 0
for all y ∈ K∗ , and so K ⊆ (K∗ )∗ . Therefore cl K ⊆ (K∗ )∗ .
The other containing relationship (K∗ )∗ ⊆ cl K can be shown by
contradiction. Suppose such is not true then there exists
z ∈ (K∗ )∗ \ cl K. By the separation theorem for convex cones, there is
vector c such that
cT z < 0, and cT x ≥ 0 ∀x ∈ cl K.
The last relation suggests that c ∈ K∗ , which contradicts with the first
relation because, as z ∈ (K∗ )∗ , it should lead to cT z ≥ 0, but not the
opposite. This contradiction completes the proof.
2
Shuzhong Zhang
IE 8534
11
This proof suggests that the bipolar theorem for convex cone is
equivalent to the separation theorem itself. Suppose that K is a closed
convex cone, and v ̸∈ K. Then, the bipolar theorem asserts that
v ̸∈ (K∗ )∗ , and this implies that there is y ∈ K∗ such that v T y < 0,
and xT y ≥ 0 for any x ∈ K because y ∈ K∗ , hence a separation exists.
Shuzhong Zhang
IE 8534
12
In case K is closed, the bipolar theorem is what in folklore understood
as ‘the dual of the dual is the primal itself again’. Combining
Theorem 3 and Proposition 1, it follows that
cl (K1 + K2 ) = (K1 ∩ K2 )∗ .
It is also useful to compute the effect of linear transformation on a
convex cone in the context of dual operation. Let A be an invertible
matrix. Then, we have
(AT K)∗ = A−1 K∗
by observing that y ∈ (AT K)∗ ⇐⇒ (AT x)T y ≥ 0 ∀x ∈ K ⇐⇒ Ay ∈ K∗
⇐⇒ y ∈ A−1 K∗ .
Shuzhong Zhang
IE 8534
13
A convex cone is called solid if it is full dimensional. Mathematically,
K ⊆ Rn is solid ⇐⇒
span K = Rn , or K + (−K) = Rn .
A convex cone is called pointed if it does not contain any nontrivial
subspace; i.e. K ∩ (−K) = {0}. The following result proves to be useful:
Proposition 2 If K is solid then K∗ is pointed, and if K is pointed
then K∗ is solid.
Shuzhong Zhang
IE 8534
14
Proof. Suppose that K is solid, i.e. there is a ball B(x0 ; δ) ⊆ K with
radius δ > 0 and centered at x0 , and at the same time suppose that
there is a line Rc ∈ K∗ , then
0 ≤ s(x0 + t)T c, ∀∥t∥ ≤ δ and s ∈ R,
which is clearly impossible. This contradiction shows that if K is solid
then K∗ must be pointed. By the bipolar theorem, the second part
follows by symmetry.
2
Shuzhong Zhang
IE 8534
15
It is easy to compute the following dual cones
• (Rn+ )∗ = Rn+ .
• SOC(n)∗ = SOC(n).
n×n
n×n ∗
n×n
n×n ∗
.
) = H+
, and (H+
) = S+
• (S+
The above standard cones are so regular that their corresponding dual
cones coincide with the original cones themselves (the primal cones).
In general, they are not the same. As an example, consider the
following Lp cone, where p ≥ 1,



 t

n
 t ∈ R, x ∈ R , t ≥ ∥x∥p .
Lp (n + 1) = 
 x

Shuzhong Zhang
IE 8534
16
One can show that (Lp (n + 1))∗ = Lq (n + 1) where 1/p + 1/q = 1.
Now, the self-duality of SOC(n + 1) is due to the fact that
SOC(n + 1) = L2 (n + 1).
In convex analysis, the dual object for a convex function f (x) is known
as the conjugate of f (x), defined as
f ∗ (s) = sup{(−s)T x − f (x) | x ∈ dom f },
where ‘dom f ’ stands for the domain of the function f . The conjugate
function is also known as the Legendre-Fenchel transformation.
Shuzhong Zhang
IE 8534
17
Examples of some popular conjugate functions are:
1. For f (x) = 12 xT Qx + bT x where Q ≻ 0, the conjugate function is
f ∗ (s) = 12 (s + b)T Q−1 (s + b).
∑ n xi
2. For f (x) = i=1 e , the conjugate function is
∑n
∑n
∗
f (s) = − i=1 si log(−si ) + i=1 si .
∑n
3. For f (x) = i=1 xi log xi , the conjugate function is
∑n 1+si
∗
f (s) = i=1 e
.
4. For f (x) = cT x, the conjugate function is f ∗ (s) = 0 for s = −c
and f ∗ (s) = +∞ when s ̸= −c.
∑n
5. For f (x) = − i=1 log xi , the conjugate function is
∑n
∗
f (s) = −n − i=1 log si .
6. For f (x) = max1≤i≤n xi , the conjugate function is f ∗ (s) = 0 if
∑n
∗
s
≤
−1,
and
f
(s) = +∞ otherwise.
i
i=1
Shuzhong Zhang
IE 8534
18
If f (x) is strictly convex and differentiable, then f ∗ (s) is also strictly
convex and differentiable. The famous bi-conjugate theorem states
that f ∗∗ = cl f . There is certainly a relationship between the two dual
objects: the dual cone and the conjugate function. Let us now set out
to find the exact relationship. First of all, suppose that f is a convex
function, mapping from its domain in Rn to R. Its epigraphy is a set
in Rn+1 defined as




 t
 t ≥ f (x), t ∈ R, x ∈ dom (f ) .
epi (f ) := 

 x
Shuzhong Zhang
IE 8534
19
It is well known that f is a convex function if and only if epi (f ) is a
convex set. It turns out that H (epi (f ∗ )) is the dual of H (epi (f )),
after taking the closure and reorienting itself. To be precise, we have:
Theorem 4 Suppose that f is a convex function, and








  p   q  p > 0, q − pf (x/p) ≥ 0 .
K := cl








x
Then,







  u  
 v  v > 0, u − vf ∗ (s/v) ≥ 0 .
K∗ = cl








s
Shuzhong Zhang
IE 8534
20


Let L2 := 
0, 1
 and L := L2 ⊕ In = [L2 , 02,n ; 0n,2 , In ]. The effect
1, 0
of applying L to x is to swap the position of x1 and x2 . Equivalently,
Theorem 4 can be stated as
∗
(H (epi (f ))) = cl (L (H (epi (f ∗ )))) .
Shuzhong Zhang
IE 8534
21
Another related concept is the so-called polar set. Suppose S ⊂ Rn .
Then its polar set is defined as
S ◦ = {y | (−y)T x ≤ 1, ∀x ∈ S}.
Clearly, S ◦ is always closed and convex, and it always contains the
origin.
If S is a cone, then S ◦ = S ∗ .
One can verify that

 


 1
 ∈ H (S)∗ .
S ◦ = y 

 y
Shuzhong Zhang
IE 8534
22
Some basic properties regarding the polar set immediately follow:
• If A ⊆ B then A◦ ⊇ B ◦ ;
• (∪i∈I Si )◦ = ∩i∈I Si◦ ;
• If S = conv(v1 , · · · , vm ) then
S ◦ = {x | −viT x ≤ 1, i = 1, 2, ..., m}.
In other words, the polar of a polytope is a polyhedron.
• It always holds S ⊆ S ◦◦ .
Shuzhong Zhang
IE 8534
23
By Theorem 3, we also have
Theorem 5 If S is a convex set, then
S ◦◦ = cl S.
The above is evidently equivalent to Theorem 3. They can both be
referred to as the bipolar theorem.
Shuzhong Zhang
IE 8534
24
Let us return to primal-dual conic optimization models.
In linear programming, it is known that whenever the primal and the
dual problems are both feasible then complementary solutions must
exist. This important property, however, is lost, for a general conic
optimization model, as the following example shows.
Consider
minimize
x11
subject to
x + x21 = 1,
12

x , x12
 11
 ≽ 0.
x21 , x22
This problem is clearly feasible. However, there is no attainable
optimal solution.
Shuzhong Zhang
IE 8534
25
Its dual problem is
maximize
y
subject to
y
0, 1
+
s11 , s12

1, 0

s21 , s22

s11 , s12


s21 , s22



=

1, 0
,
0, 0
 ≽ 0,
or, more explicitly,
maximize
y

subject to

1,
−y
−y,
0

 ≽ 0.
The dual problem is feasible and has an optimal solution; however, a
pair of primal-dual complementary optimal solutions fails to exist,
although both primal and dual problems are feasible.
Shuzhong Zhang
IE 8534
26
One may guess that the unfortunate situation was caused by the
non-attainable optimal solution. Next example shows that even if both
primal and dual problems are nicely feasible with attainable optimal
solutions, there might still be a positive duality gap, meaning that the
complementarity condition does not hold.
minimize
−x12 − x21 + 2x33
subject to
x12 + x21 + x33 = 1,
x22 = 0,

x11 , x12 , x13

 x21 , x22 , x23

x31 , x32 , x33


 ≽ 0.

Shuzhong Zhang
IE 8534
27
The dual of this problem is
maximize
subject to
y1

0,

 −1 − y1 ,

0,
−1 − y1 ,
0
−y2 ,
0
0,
2 − y1


 ≽ 0.

The primal problem has an optimal solution, e.g.


1, 0, 0


∗

X =  0, 0, 0 
, with the optimal value equal to 2. The dual
0, 0, 1
problem also is feasible, and has an optimal solution e.g.
(y1∗ , y2∗ ) = (−1, −1), with optimal value equal to −1. Both problems
are feasible and have attainable optimal solutions; however, their
optimal solutions are not complementary to each other, and there is a
positive duality gap: cT x∗ − bT y ∗ = (x∗ )T s∗ > 0.
Shuzhong Zhang
IE 8534
28
Key References:
• Yu. Nesterov and A. Nemirovsky, Interior Point Polynomial
Methods in Convex Programming, Studies in Applied Mathematics
13, SIAM, 1994.
• J.F. Sturm, Theory and Algorithms for Semidefinite Programming.
In High Performance Optimization, eds. H. Frenk, K. Roos, T.
Terlaky, and S. Zhang, Kluwer Academic Publishers, 1999.
• S. Zhang, A New Self-Dual Embedding Method for Convex
Programming, Journal of Global Optimization, 29, 479 – 496, 2004.
Shuzhong Zhang