MIT LIBRARIES
III
III
111
l|ll llljl
DUPL
II
Mil
IHII l||l|
3 9080 02246 1237
)28
[DEWEy
\^^
1414
•i
MIT
Sloan School of Management
Sloan Working Paper 4176-01
July 2001
ON THE PRIMAL-DUAL GEOMETRY
OF LEVEL SETS IN LINEAR
AND CONIC OPTIMIZATION
Robert M. Freund
The paper can be downloaded without charge from the
Social Science Research Network Electronic Paper Collection:
http://papers.ssm.com/paper.taf?abstract_id=283690
ON THE PRIMAL-DUAL GEOMETRY OF LEVEL
SETS IN LINEAR AND CONIC OPTIMIZATION
^
Robert M. Freund"^
M.I.T.
July,
2001
Abstract
For a conic optimization problem
P
:
minimize^
c^x
Ax —
s.t.
b
xeC
and
its
dual:
D
:
supremum^
s.t.
3
b^y
A^y +
s
=
c
seC\
we present
a geometric relationship between the
level sets of the
maximum norms
of the
primal and the inscribed sizes of the level sets of the
dual (or the other way around).
AMS
Subject Classification: 90C, 90C05, 90C60
Keywords: Convex Optimization, Conic Optimization,
Duality, Level Sets
^This research has been partially supported through the MIT-Singapore Alliance.
02142-1347,
Sloan School of Management, 50 Memorial Drive, Cambridge,
^MIT
USA.
email: rfreund@mit.edu
MA
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
1
1
Primal-Dual Geometry of Level Sets for Linear Optimization
Consider the following dual pair of linear optimization problems:
LP
c^x
minimize
:
Ax — b
X > 0,
s.t.
and:
LD
:
maximize
b'^y
s.t.
A'^y
s
+
>
s
=
c
,
whose common optimal value is z* For e >
and <5 >
(5-level sets for the primal and dual problems as follows::
.
P, := [x
\
Ax = b,x>0, Jx <
2*
+
0,
define the
e-
and
e}
and
Di :=
Is
A'^y
I
+
s
=
>
c, s
for
some y
satisfying b^y
>
z*
—
S\
.
Define:
i?e
:=
max
||2:||i
Ax = b
Jx < z* +
x >
s.t.
e
and
rs
:=
max
s.t.
minj{sj}
AJ'y
h^y
s
The quantity R^
level set P^,
measured
is
equivalently as the
s
(2)
>
simply the size of the largest vector x
in the
as the positivity of the
=c
>z* -6
+
Li-norm.
The quantity
most positive vector
maximum
s in
rs
in the
primal
can be interpreted
the dual level set Ds, or
distance to the boundary of the nonnegative
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
orthant over
Theorem
=
If
min {e,6} < R^
rs
—
if r^
R^
positive
is
<
+
e
+oo, and R^
6
and
finite,
then
.
= +oo
if
and only
ifr^
—
0.
theorem follows as a special case of a more general
convex conic optimization, namely Lemma 4.1 in Section 4.
The proof
result for
Theorem
itivit}'
is finite.
and only
if
following theorem presents a reciprocal
rf.
1.1 Suppose that z*
Otherwise, R^
The
points s in Ds.
all
relationship between R^ and
2
of the
of this
1.1
bounds the
and the posD^ from above and below, and shows
size of the largest vector in P^
most positive vector
in
that these quantities are almost exactly inversely proportional. In fact, taking
6
=
e,
the result states that R^
Corollary 1.1
If
<
R^
>
all e
f.
< ^.
•
r,
<
f
^1
R.
(3)
2e
the result follows trivially, since then R^'
<
and
i?^
e
<
< +oo.
R^>
r,
Let 6 :=
<
+
e
e,
e.
that R^' grows at most linearly in
for all
<
course, there
(5
<
(5,
which
and so
R^'
However,
(3)
is
for all e
at a rate
< ('-^\
R,.
1.1
>
0.
we have
\
as e increases,
and shows
no greater than ^.
r^'
,
namely:
true as an elementary consequence of the convexity
LD
(independent of the results contained herein).
does not seem to lend
By exchanging
and
i?^,'
a version of (3) for rs and
is
of the feasible region of
e'
=
Then from Theorem
Corollary 1.1 bounds the rate of growth of
Of
26].
|
Proof: If i?e =
So suppose that
e
[e,
oo, then
R, <
for
the interval
r^ lies in
itself to
an independent elementary proof.
the roles of the primal and dual problems, we obviously
can construct analogous results for the most positive vector x
for the size of the largest vector s in D^.
in P^ as well as
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
Norm on X
Conic Optimization with a
2
We now
3
consider the generalization of linear optimization to convex optimiza-
tion in conic linear form:
P
z* :=
:
minimum^
c^x
Ax =
s.t.
b
X G C,
and
dual;
its
D
:— supremumy^^
v'
:
b^y
A^y + s —
SEC*,
s.t.
where
CC
space
X, and
for
A'
is
a closed convex cone in the
the
b lies in
(finite)
in
>
and
and dual problems as
For
e
least to Duffin
as well as in Ben-Israel et
[2]
>
6
0,
we
7i-dimensional linear vector
(finite)
m-dimensional vector space Y. This format
convex optimization dates back at
can be found
c
define the
e-
al.
[2].
Strong duality results
[1].
and
(5-level sets for
the primal
follows:
P, := Ix
\
Ax =
b,x e C,
Jx <
2*
-I-
e|
and
Ds
—
+
A'^y
Is
—
s
c,s €
C*
some
ior
I
We make
>
v*
- 5\
.
the following assumption:
Assumption A:
so
1'* satisfying b'^y
y G
C* has an
and
z*
v* are
both
finite.
C
The cone
contains no line (and
interior).
X
Suppose that
with the dual norm
r centered at u
G
•
||
A
and so A^* is endowed
endowed with a norm
||,. Let B{v,r) and B*{w,r) denote the balls of radius
and w G A*, respectively, defined for the appropriate
is
•
||
||
norms.
We
denote the
maximum norm
R, :=
max^
s.t.
We
denote by
rs
of P^ by Re, defined as:
||x||
xeP,.
the inscribed size of D^, defined as:
,
.
^^'
—
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
s
s.t.
e Ds
(6)
CC*
B*{s,r)
As
optimization, rs measures the distance of the most
in tlie case of linear
interior point of the dual level set
Ds
to the
boundary of the cone C*.
Before presenting the version of Theorem
tion,
A
we
regular
is
K
if
Definition 2.1
K
We
-—
<
jeintK
nonempty
[3].
interior,
use the following definition of the
normed
linear vector space
X,
defined as:
is
max
;=
We
line).
// A' is a regular cone in the
the min-width of
norm
convex conic optimiza-
a closed convex cone, has a
is
and is pointed (i.e., contains no
min-width of K:
tk
1.1 for
review the concept of the min- width of a regular cone, see
first
cone A'
.
remark that
>
X
I
r^-
— max
x€intA'
I
<
I
7^
-
maximum
measures the
B[x,r)
C A
\\x\\
ratio of the radius to the
and so larger values of r^- correspond to an intuitive notion of greater minimum width of K. Note that
is a regular cone, since
has a nonempty interior and K is
tk G (0, 1] if
pointed, and r^^- is attained for some x^ G intA" satisfying \\x^\\ = 1, as well as
along the ray ax^ for all a > 0. Let r^-- be defined similarly for the dual cone
of the center of an inscribed ball in A',
K
K
AT'.
The
Theorem
then z*
=
following
is
analogous to Theorem
A
2.1 Suppose that Assumption
v'
1.1 for conic
holds. If
R^
<
5
is
problems:
positive
and
finite,
and
Tc-
Otherwise, R^
—
if
•
min {e,6} < R^-
and only
ifrg
=
-|-oo,
r^
and R^
e-\-
—
.
-|-oo if
and only
ifrs
—
0.
Here we have had to introduce the min-width tq- into the left inequality,
somewhat weakening the result. In the next section we show how to define a
family of cone-based norms for which tc = 1; we also show that for norms
induced by a i3-normal barrier function on C that re* > l/\fd. Theorem 2.1
is
proved
in Section 4.
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
Corollary
—
TC-
]R.
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
:=
||s||,
miiio
Q
s.t.
s
+
as° e C*
-s + as° €
whose dual norm
is
easilv seen to be:
:=
\\x\\
{s){x+x)
min^i.i.2
x^eC
Under
||,,
||
it
is
easily
In the case
Loo-norm as
4
||s||»
C
shown that
when
for s
\\s°\\t
=
1
.
and
= 5R",
= 0* = 5R" and
=
X K" and the Li-norm as
A'
£
—
tq-
,
1.
=
s°
||x||
e,
we
recover the
for
x £
X
—
K".
A Technical Lemma, and Proofs of Main Results
The
technical result that
follows. Let s"
is
the basis for the other results in this paper
as
is
6 intC* be given. Let
R, :=
max^
(s°)'^x
s.t.
X e P,
,
^
,
'
and
rs
:=
maxj^r
^
s.t.
s
e Ds
(8)
s-rs° eC*
Lemma
then z'
—
4.1 Suppose that Asssumption
v'
A
holds.
.
If
R^
is
positive
and
finite,
and
min {e,6} < R,-fs <
Otfierwise, R^
—
if
and only
if f^
=
e
+00, and R^
+
6
(9)
.
= +00
if
and only
if f^
=
0.
.
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
Lemma
Before proving
we
4.1,
P
prove strong duality for
first
and
D
when Rf < +00.
Proposition 4.1
< +00,
R^
If
then z*
=
v*
Proof: Note that if ^^ < 00 for some e > 0, then all of the level sets P^ of
P are bounded, and so P attains its minimum. The boundedness of P^ also
implies that
=
{0}
and
e C,
i;
Jv <
0}
>
z'
v*.
Suppose that
>
z*
(10)
.
v*
,
let e
=
""'
^
,
let
S=
{(1^,0)13^ 6 Y* s £ C* satisfying
,
Then 5
a
is
exists (x, 9)
x'^
0,
I
elementary to show that
It is
Av =
[v € .Y
(A^y +
nonempty convex
^
and
x
+ 9a >
for all {w, a)
+
(-b^y
—
s
set in A'*
satisfying x'^w
s-c)+9
w = A^y +
+e+
u*
/;)
5R,
>
b^y
c,
>
v*
+e—
a}.
^ S, whereby there
(0, 0)
G 5. Therefore:
Vy € Y\\/s G C*,
V?/
>
.
(11)
This implies that
Case
^
1:
Therefore x
which
is
Case
2:
c^x <
In
0,
is
>
b9, 6
Without
0.
feasible for P,
0,
We now
and x E C.
(11) also implies that z*
9
=
^
In this case x
0.
Lemma
If fj
0.
< c^x <
v*
-\-e
—
<
1.
z*,
0,
Ax =
x e C,
0,
and
(11) implies that
contradicting (10).
both cases we have a contradiction, and so
=
we can assume that 9
loss of generality
and
have two cases:
a contradiction.
Proof of
fs
>
Ax =
=
it is
=
v*
.
|
examine the extreme cases fs = +00 and
a straightforward exercise in convex analysis to
4.1: Let us
+00, then
z*
first
show under Assumption A that z* = v* = 0,b = 0, the feasible region of P is
the singleton {0}, and there exists [y, s) satisfying A'^y+s = 0, s G intC*. This
then implies that R^
=
0.
If fs
=
0,
then
it is
convex analysis to show under Assumption
Ax =
0,
X G C, c^x
We now
there exists s £
<
0.
+ 6>s'^x =
R^^
=
{v
7^
satisfying
+00.
< fs < +00. For any a G (0,f^),
— as° G C* For every x G P^ we have
when
satisfying v := s
e
that there exists i
This then implies that
consider the case
Ds
A
also a straightforward exercise in
.
+ as°fx >
ais'^fx,
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
—
wherebv ^t <
and so R,f^ <
,
+
e
8
This proves the second inequality of
5.
(9).
To prove the first inequality of (9), we proceed
program together with its dual:
We
as follows.
write (7)
as the following
P
maxj
R, :=
:
R^
=
Lemma
4.2 below,
under Assumption
V,
v :=
:
A
s
e>o.
we show strong duality for P and D, namely
and in the case when R,, < +oo.
P
is
bounded,
let
P
will attain its
optimum.
{y,s,6) be a feasible solution
satisfying
-6^y +
and define y :=
s
-
s"
G C*.
ReVs
and
e)e
+ = 9c
s-s° eC*
t
For a 6 (0,min{e, J}) and sufficiently small,
D
+
{z*
A'^y
s.t.
Because the feasible region of
of
+
-b'^y
infy,,,^
ec
X
In
D
{s°fx
Ax = b
Jx <z* +
s.t.
letting
a
^
We
>
(z*
will
+
<R, + a
(12)
show below that
^^
establishes the
e)9
(min{e, 5
first
- a})
(13)
inequality of (9).
We
prove (13) by
considering three cases.
Case
1:
^
=
0.
In this case A'^y
any feasible solution of
D
+
s
—
satisfying b'^y
(y,s)
={y,s)+
>
and -b'^y < R^ + a. Let
z* - a, and define
-
Re
Then
(y, s) is feasible for
b'^y
=
b^y
+
-
-^
b'^y
+a
2:
4^s° = Pt-v + seC*,
>
and
iy,s).
>z*-a-5 + a =
whereby
implies (13).
Case
be
D, and
Rf
Also, s
+a
(y, s)
^^ -€<5.
Define
s
G Ds and
z*-5.
fs
>
4^.
This th en
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
wlierebv
(y, s) satisfies s
C,
£
whence
>
r^
£ D^- Furthermore,
However,
r
,
and so
Case
> yf^, whereby
^
>
3:
and
^^^ -
e
^
>
>
5.
+
and
+
^-
e
= ^ + ^,
s
I
()
+
Re
.
>
r^
c,
^
^
wliich sliows that s
i.
=
s
Rt+Oi
1,T
y=-/y>
,T-
h'
+
A'^y
I
>
v
'-'
.-*
-
<5,
""'-' so
^'J s-^s'^ G C*
G
^ and
^ C*,
'^
I
'
ce
-j^^, which then implies (13).
Let
(y, s)
be any feasible solution of
D
satisfying
b'y>z'-a
(14)
and define
= A(%^)+(l-A)(y,s-)
(y,5)
where
6
— a
^^-€-a
Then A 6
(y, s)
[0, 1]
and so
>
that b'^y
for
a e
satisfies .4-^y
z'
-
+
whereby
5,
A
^5
and so
(0, ^),
>
7T
s
s
=
9
from which
then R^
is
true,
e C*
=
It
.
Finally, s
5
also follows
-
—a
T^
R, + a - a9 - ee
-.
7,
and
letting
a —
from
(12)
^^
and
and
(14)
|s° G C*, and so
>
-
— a
5
Re
proves the
)•
4.2 Suppose that Assumption
+a
A
holds.
If
first
R^
inequality of (9).
is
positive
and
x^(s -
for
s°)
=
x and {y,s,9) feasible
9c'^x
-
y'^Ax
for
- {sYx <
P
and D,
9{z*
+
e)
|
finite,
V.
Proof: Note that
<
c, s
G D^.
a convex combination of
(13) follows.
Therefore (13)
Lemma
=
(y, s) is
respectively,
- y^6 - [s^x,
.
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
and so
let f
< -y^h +
{s°)'^x
^^
=
5 =
and
q)
(«',
{
+
{z*
I
3y e
Y% seX*,e>0
x'^
s,
nonempty convex
a
is
^
exists {x,^)
+
A'^y
+
Vy e
s°
+
1:
>
(3
Therefore x
which
is
Case
2:
j5
also X
/
0,
is
0.
Without
feasible for
/3a
+
x
=
0,
and
5R,
P
and
(0,0) ^ 5,
£
for all [w, a)
+ ^ [-b^y +
6/?
<v - e + a,s
e)e
ye >
and
/3
+
[z*
>
0.
-
e)9
v??
0,
loss of generality
=
Q.
>
-
v,
s°
e C*}.
whereby there
5. Therefore
v
+e+
r])
>
o
(i5)
We now
have two cases:
we can assume that
>
(15) also implies that {s°)'^x
In this case (15) implies that
which contradicts
we obtain a
Proof of Theorem
=
<
7?^
v
=
/3
-
e
>
1.
R^,
a contradiction.
In both cases
sO
+
>
vu G c*,
This implies that x G C, .4x -
Case
Suppose that
v.
satisfying
[z*
set in ,Y*
u)
y\
+
-b'^y
satisfying x'^w
[-Be
<
Tlierefore R^
let
w = -ec + A^y +
Then 5
e)0.
10
Ax =
0,
=
v.
0,
and
(10).
contradiction, and so R^
Follows from
1.1:
x G C, c^x <
Lemma
4.1
|
by setting
C =
5R"
and
e:=(l,...,l)^.,
Before proving Theorem
2.1,
we
state the following
two propositions.
The proof of
the Proposition 4.2 follows immediately from Proposition 3 of [4]
and Proposition 2.1 of [3]. Proposition 4.3 is a special case of the Hahn-Banach
Theorem; for a short proof of this proposition based on the subdifferential
operator, see Proposition 2 of
Proposition 4.2 Suppose K*
IS
attained at
some point
s°
[4].
is
a regular convex cone whose min-width
€ intK* satisfying
TK-M <{SYX
for
all
X G A'
||5°||,
<\\X\
=
1.
Then
r^--
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
Proposition 4.3 For every x €
||x||,
=
1
and
||.r||
—
x^x.
Proof of Theorem
of
C*
is
also Tc-
X
,
<
<
the property that
|
2.1: Let s°
(s°)-^x
E X* with
there exists x
G C* be a point
attained, normalized so that
||x||
11
||x||
for all
||s°||t
j G
C
=
1-
in
C*
at
Then
which the min-width
B*{s'^,tc-)
(from Proposition
4.2).
C C* and
From (7) and
s - rs° G C*
< R^ < Rt- Also, since the constraint
implies that
C C* and the constraint B*{s,r) C
s - rs'^ € C, it follows from (8) and (6) that Tcfg < rs < f^. Therefore
if and only if .R^ =
if and only if fs — +oo if and only if rs — +oo,
i?f =
and i?f = +00 if and only if R^ — +oo if and only if f^ =
if and only if
—
0, from Lemma 4.1.
rs
(5) it follows
that tc-R^
C
implies that B*{s,rTc-)
positive
If i?t is
Tc- m'in{e,5},
from
theorem. Also, this
=
and
Lemma
finite, it
follows then that R^
rs
>
tc- Re
fs
^
which proves one of the inequalities of the
shows that rs > 0. For any x G P^, there exists x E X* for
=
4.1,
And
any a G {0,rs),
there exists {y,s) feasible for D such that s E Ds and B*{s,a) C C*. Then
e + 6 > c^x — b^y = s^x = {s — ax + ax)'^x > ax^x — a||a;||, where the second
inequality follows from the fact that (s - ax) G C* since B*{s,a) C C*.
Therefore ||x|| <
for any a G {0,rs) and x G P^, whereby Rs rs < e + S,
completing the proof |
which
||x||,
1
and x'^x
^
\\x\\,
see Proposition 4.3.
for
PRIMAL-DUAL GEOMETRY OF LEVEL SETS
12
References
[1]
A. Ben-Israel, A. Charnes, and K.O. Kortanek.
solvability over cones. Bulletin of the
[2]
R.J. Duffin. Infinite programs. In
Amials of Mathematical Studies
AMS,
Duality and asymptotic
75:318-324, 1969.
H.W. Kuhn and A.W. Tucker,
38,
editors,
pages 157-170, Princeton, N.J., 1956.
Princeton University Press.
[3]
R.M. Freund and J.R. Vera. Condition-based complexity of convex
mization in conic linear form via the ellipsoid algorithm.
SIAM
opti-
Journal
on Optimization, 10(1):155-176, 1999.
[4]
[5]
R.M. Freund and J.R. Vera. Some characterizations and properties of the
"distance to ill-posedness" and the condition measure of a conic linear
system. Mathematical Programming, 86:225-260, 1999.
Y. Nesterov and A. Nemirovskii.
in
Convex Programming. Society
(SIAM), Philadelphia, 1994.
Interior- Point
for Industrial
Polynomial Algorithms
and Applied Mathematics
^232
f^OV
2002
Date Due
IIIIIHM
I
I
I
ll'lllllllllll
3 9080 02246 1237