I
h-
MIT
Sloan School of Management
Sloan Working Paper 4179-01
September 2001
COMPLEXITY OF CONVEX OPTIMIZATION
USING GEOMETRY-BASED MEASURES
AND A REFERENCE POINT
Robert M. Freund
This paper can be downloaded without charge from the
Social Science Research
Network Electronic Paper Collection:
http://papers.ssm. com/abstract_id=288 1 34
DEWEY
MASSACh USEI IS INSTITUTE
OF TECHMOLOGY
Jt'ri
3 2002
LIBl^RIES
COMPLEXITY OF CONVEX OPTIMIZATION
USING GEOMETRY-BASED MEASURES AND A
REFERENCE POINT
^
Robert
\l.
Freund^
M.I.T.
September, 2001
Abstract
Our concern
lies in
solving the following convex optimization prob-
lem:
Gp
:
minimize^
s.t.
c^x
Ax =
b
X eP,
where
X.
of
P
is
a closed convex subset of the n-dimensional vector space
We bound the complexity of computing an almost-optimal solution
Gp in terms of natural geometry-based measures of the feasible re-
gion and the level-set of almost-optimal solutions, relative to a given
reference point x^ that might be close to the feasible region and/or the
almost-optimal level
set.
This contrasts with other complexity bounds
convex optimization that rely on data-based condition numbers or
algebraic measures, and that do not take into account any a priori reffor
erence point information.
AMS
Subject Classification: DOC, 90C05, 90C60
Keywords: Convex Optimization, Complexity, Interior-Point Method, Barrier Method
'This research has been partially supported through the MIT-Singapore Alhance.
'^MIT Sloan School of Management, 50 Memorial Drive, Cambridge,
02142-1347,
USA. email; rfreund@mit.edu
MA
GEOMETRY-BASED COMPLEXITY
1
Main Result
Introduction, Motivation, and
1
Consider the following convex optimization problem:
Gp
:
:=
z*
minimum^
c^x
Ax —
s.t.
b
X e P.
where P is a closed convex set in the (finite) 7i-dimensional linear vector space
X, and b lies in the (finite) m-dimensional vector space Y. We call this prob-
lem linear optimization with a ground-set, and we
practical applications,
/
< X <
the ground-set. In
C
b.x
or perhaps the solution to network flow con-
>
However, for ease of presentation, we will
0.
the following assumption:
Assumption A: P
For
>
e
0.
Gp
solution of
is
P
could be the solution of box constraints of the form
Ni =
form
straints of the
make
P
a convex cone
u,
call
has an interior, and {x
we
call
Let
Ax =
6}
z* + e. The
complexity bound
<
intP
fl
x an e-optimal solution of
that satisfies c^x
an algorithm and associated
solution of
|
Gp
/
if
x
0.
is
a feasible
chief concern in this paper
for
computing an e-optimal
Gp.
be any norm on
•
II
II
A',
and
let
B{x.
denote the ball of radius
r)
r
centered at x:
B{x,r) :=
The norm
•
||
||
{u-
G
A'
||u,'
—
<
x||
r}
might be a problem-appropriate norm
context at hand. However, in Section
on A' that arise "naturally"'
.
|
5.1.
we
in association
will
for the actual
examine
in detail
problem
two norms
with the ground-set P.
The computational engine that we will use to solve Gp is the barrier
method based on the theory of self-concordant barriers, and we presume that
the reader has a general familiarity with this topic as developed in
[7],
for
example.
rier Fp[-) for
F|| ||(-)
P.
[5]
and/or
We therefore assume that we have a 7?p-self-concordant barWe also assume that we have a i^y ||-self-concordant barrier
for the unit ball:
5(0,1)
The work
here
is
=
{x
I
|lx||
<
1}
.
motivated by a desire to generalize and improve sev-
eral aspects of the general
complexity theory for conic convex optimization
[7], key elements of which we now attempt to summarize in a brief and somewhat simplified manner. In [7] the general convex
optimization problem Gp is assumed to be conic, that is, P is assumed to
developed by Renegar in
,
be closed convex cone
C
.
and the data
for the
problem
is
given by the array
GEOMETRY-BASED COMPLEXITY
d
=
(A,b,c).
One complexity
assuming that Gp has a
on interior-point methods that
feasible solution, there
will
^
Newton's method
is
[7]
dist(x,
(see
Theorem
mm{s,
aC)
3.1
is
as fol-
an algorithm based
compute an e-optimal solution
€
\
iterations of
can be gleaned from
result that
lows:
O
2
of
in
yy
||c(||}
and Corollary
Gp
7.3 of
[7]),
where we use the notation dc to denote the complexity value of the barrier for
the cone C. Here x G C is a given interior point of the cone C that is specified
as part of the input to the algorithm, dist(x, dC) is the distance from x to the
boundary of C, \\d\\ is the (suitably defined) norm of the data ci, and s is a
positive scalar that must be specified as input to the algorithm. The quantity
C{d) is the condition number of the data d, defined as:
=
^^
Cid)
"-^^^^
mm{pp(d),po(d)}
(2)
^'>
'
where pp{d), poid) are the primal and dual distances to ill-posedness, see [7]
and motivating discussion. {C{d) naturally extends the concept
of condition number of a system of equations to the far broader problem of
conic convex optimization.) The complexity result (1) is remarkable for its
breadth and generality, as well as for its reliance on natural data-dependent
concepts imbedded in condition-number theory. In order to keep the presenfor details
in (1)
we have shown a
and slightly weaker complexity result
than the verbatim complexity bound in [7]. Furthermore, [7] has many
tation brief,
simplified
other complexity results related to conic convex optimization in finite as well
as infinite-dimensional settings.
While the significance of
(1)
and the many related
results in
[7]
cannot
be overstated, there are certain issues with this type of complexity bound that
are not very satisfactory. One issue has to do with undue data dependence.
Given a data instance d = {A, 6, c) for Gp and a nonsingular matrix B and
a vector n of multipliers, we can create an equivalent representation of the
problem Gp using the different data d — {A,b,c) :— {B~^A,B~^b,c — A'^n).
The two data
instances d and d will generally give rise to different complexity
bounds using
(1) since in general
represent the
C{d)
^
C{d), etc., yet both data instances
same underlying optimization problem.
Another issue with the condition number approach is that the problem must be in conic form. While any convex optimization problem can be
transformed to conic form, such a transformation might not be natural (such
as converting a quadratic objective to a linear objective using a second-order
cone constraint,
different
tion
etc.)
or unique (and so might further introduce arbitrarily
data for the same original problem). Yet a third issue with the condi-
number approach has
to
do with the
fact that the theory
assumes that the
GEOMETRY-BASED COMPLEXITY
data
arising only in the linear equation system
is
and that the cone
C
fixed independent of
data
fourth issue has to do with the role of the starting point x.
The
used to defined the cone
A
bound
and the objective function,
any data. In
this format,
is
(1) is
is
not accounted for in the theory.
not dependent on or sensitive to the extent to which x might be
nearly feasible and/or nearly optimal.
bound that accounted
for the
It
would be nice to have a complexity
proximity of x to the feasible and/or optimal
solution set.
The algorithm and
analysis presented in this paper represent an at-
tempt to overcome the above-mentioned issues. In Sections 3 and 4, we develop
and analyze interior-point algorithms FEAS and OPT for finding a feasible solution x of Gp and an e-optimal solution x of Gp, respectively. This pair of
algorithms and their complexity analysis depend on certain geometry-based
measures for analyzing convex optimization problems and the concept of a
reference point x^ which we now discuss.
,
Reference Point and Interior Point
1.1
The
phase-I algorithm
FEAS
requires that the user specify two points as part
and an
of the input of the algorithm, the reference point x^,
The
x° G intF.
reference point
x""
might be chosen to be an
interior point
guess of
initial
a feasible and/or optimal solution, the solution to a previous version of the
problem (such as
X,
etc.
If
P
is
in
of the space
warm-start methodologies), or the origin
the box defined by the constraints
be chosen as a given corner of the box such as
< x <
I
=
x'"
/
if
;
then
u,
P
is
x''
might
convex cone
=
0, or a known point on the
boundary or the interior of C, etc. Certain properties of x'" will enter into the
complexity bounds derived herein, particularly related to the distance from
x"" to the feasible region, to the set of e-optimal solutions, and to the set of
nicely-interior feasible solutions. There is no assumption concerning whether
C, x^ might be chosen to be the origin
or not
x"" is
in the
Algorithm
ground-set
FEAS
point will be used in
are.
(By analogy,
P
x""
or satisfies the linear equations
also requires an initial point x°
many ways
to
measure how
in linear optimization e := (1,
€ intP. This
,
.
1)-^ is
.
be desirable for x° to be nicely interior to P.
We
r :=r(x°) :=min{dist(x°,aP),l}
Although the quantity dist(x°,c>P)
will enter into
b.
interior
interior other points in
.
a
the positivity of other vectors v by computing the largest
It will
Ax =
P
used to measure
for
which v > ae.)
define:
.
our complexity bounds
(3)
for
GEOMETRY-BASED COMPLEXITY
solving
Gp
4
through the quantity t{x°), we do not require that we Icnow the
value of dist(x°, dP). However, for the two important classes of norms that
we
will consider in Section 5.1,
—
that t{x^)
this will
1;
Phase
1.2
The complexity
I
will
show that d'ist{x°,dP) >
1
we
which implies
be very desirable from a complexity point of view.
Geometry Measure
FEAS
of the phase-I algorithm
ing geometric measure which
g
we denote by g
be bounded by the follow-
will
:
max{||x
—
x^W, 1}
min{r, 1}
Ax =
s.t.
B{x,r)
If
we ignore
for the
the ratio defining
moment
g, (4)
(4)
b
C P
.
the "T's in the numerator and denominator of
could be re-written
as:
X — X
g :— mimmunij.
dist(x,(9P)
Ax =
s.t.
xeP
and so g (or g) measures the extent to which
solution X that is itself not too close to the
smaller to the extent that
x'"
is
close to
^^^
b
.
close to an interior feasible
boundary of P, and g (or g) is
feasible solutions x G intP that are
x'" is
themselves far from the boundary of P.
The
ratios defining g
and g
arise naturally in the
complexity of the
elUpsoid algorithm applied to the problem of finding a feasible solution of
Gp.
one were to initiate the ellipsoid algorithm at the ball centered at the
reference point x^ with a radius given by ||x'' — x\\ + f (where {x,f) are an
optimal solution of (5)), then it is easy to see that a suitably designed version
If
of the ellipsoid
iterations,
readers to
method would compute a
feasible solution of
under the presumption that the norm
[4]
for
•
||
||
is
Gp
in
O (n^ ln(5))
ellipsoidal.
(We
refer
an excellent treatment of the ellipsoid algorithm.) In the more
typical context in continuous optimization
bound on the distance from the
where we do not have an
a priori
feasible region to the reference point, there
GEOMETRY-BASED COMPLEXITY
is
5
a natural projective transformation of
tlie
algorithm will compute a feasible solution of
Lemma
4.1 of
that g
Therefore 5
[2].
Phase-I problem
problem
Gp
ellipsoid
iterations, see
a very relevant geometric measure for the
is
the context of the ellipsoid algorithm. Herein,
in
also relevant for the complexity of the Phase-I
is
which the
for
O (71^ ln(5))
in
problem
we
for
will see
a suitably
constructed interior-point algorithm.
Incidentally, the constants "1" appearing in the
numerator and denomi-
(4) appear for the convenience of the complexity
and could be replaced by any other positive absolute constants 71, 72.
nator of the ratio defining g in
analysis,
Phase
1.3
Our complexity
imum
Geometry Measure D^
II
analysis of the phase-II algorithm
distance from the reference point
D^ := max
At
first
glance
it
|||x
-
x""!!
may seem odd
|
x'"
Ax =
to
OPT
b,x e P, c^x
optimal solutions
optimization.
is
unbounded, which can
Then the dual
not expect to have an
text, the
more
z'
arise, for
feasible region has
efficient
<
z'
+
el
(6)
.
maximize rather than minimize
However, consider the ill-posed case when
ing D^.
on the max-
will rely
to the set of e-optimal solutions:
no
is
the
finite
in defin-
but the set of
example, in semidefinite
and so we would
solving Gp. In this con-
interior,
complexity bound for
relevant complexity measure
is
maximum
distance to the
e-optimal solution set (which would be infinite in this case) rather than the
minimum
distance (which would be finite in this case). Also, in
of conic optimization with
=
.r''
0,
it
is
[1]
in the case
shown that D^ defined using
(6) is
inversely proportional to the size of the largest ball contained in the level sets
of the dual problem,
and so D^
is
very relevant in studying the behavior of
primal-dual and/or dual interior-point algorithms for conic problems.
1.4
Main Result
Theorems
II
3.1
algorithms
and
4.1 contain
FEAS
complexity bounds on the phase-I and phase-
and OPT,
respectively.
Taken as a
pair, the
combined
complexity bound for the algorithms to compute an e-optimal solution of
using the reference point
x''
and the interior-point
x'' is:
Gp
GEOMETRY-BASED COMPLEXITY
/''^/-
O
\/^P
+
+
m,n{dist/x°.aP).l}
+
ll^°--^'ll
\\
(7)
II
v
A + max{f,l}
+P +
^
iterations of
+
^llll
In
'^\\
Newton's method, where the
s :=
m&xlc^w
U)
I
||u;||
<
I,
Aw =
0>
I
<
J
||c||,
M
II
.
Note that (7) depends logarithmically on the phase-I and phase-II geometry
measures g and D^, the inverse of the distance from x° to the boundary of P,
as well as the distance from x^ to the reference point x".
we present two choices of norms
on
bound (7) simplifies to:
In Section 5.1,
||
A' that arise
||
naturally and for which the complexity
[/J^ln
iterations of
(g
+ D, + dp + max
Newton's method.
number based complexity bound
and
rems
3.1
2
Summary
We
|-, l|
In Section 5.2,
(1)
+
||x°
-
x'
we show how the condition-
can be derived as a special case of Theo-
4.1.
of Interior-Point
Methodology
employ the basic theoretical machinery of interior-point methods
in our
analysis using the theory of self-concordant barrier functions as articulated
Renegar [7] and [8], based on the theory of self-concordant functions of
Nesterov and Xemirovskii [5]. The barrier method is essentially designed to
approximately solve a problem of the form
in
OP
where 5
:
z
=
min{c^u;
it)
|
G 5},
a compact convex subset of the 7i-dimensional space
and
€ A^*.
requires the existence of a self-concordant barrier function F{w)
the relative interior of the set 5, see [7] and [5] for details, and proceeds
is
A',
c
The method
for
by approximately solving a sequence of problems of the form
OP^
for a
:
minjc-^ii;
-f-
^iF{w)
\
w
6 relint5},
decreasing sequence of values of the barrier parameter
fj,.
We
base our
complexity analysis on the general convergence results for the barrier method
GEOMETRY-BASED COMPLEXITY
presented in Renegar
[7],
7
which are similar to (but are more accessible
our purposes than) related results found in
w^ €
at a given point
the
method
starts
ending when
OP^
for
stage
II,
some
barrier
The method performs two
relint5.
method
for
starts
In stage
stages.
I,
from w° and computes iterates based on Newton's method,
w
has computed a point
it
The
[5].
barrier parameter
that
fi
that
is
is
generated internally in stage
method computes a sequence
the barrier
an approximate solution of
I.
In
of approximate solutions w''
of OP^i^, again using Newton's method, for a decreasing sequence of barrier
parameters
c^w <
+
The goal of the barrier method
OP, which is a feasible solution w of OP
converging to zero.
fi^
an e-optimal solution of
One
description of the complexity of the barrier
Assume
that
S
barrier
method
z
e.
is
to find
for
which
method
is
as
follows:
•
and
a bounded set,
is
V
V
iterations of Newton's
method
In the above expression,
2'
=
w^ G
relintS
is,
R=
m\n{(Fw
u;
I
z"'
—
R
to
is
2'
1^ + ^))
eJJ
Tlie
\
(8)
compute an e-optimal solution of OP.
the range of the objective function
c^w
where
and
G 5}
sym(u', S) := max{^
in the
given.
sym(u;^5)
2"
=
maxjc'^u'
|
and sym(u;, S) is a measure of the symmetry of the point
the set S, and is defined as
This term
is
requires
o(>/d\n(i) +
over the set S, that
that
y ^
S
^ w - t{y — w)
w
w
G 5},
with respect to
£ S].
complexity of the barrier method arises since the closer the
to the boundary, the larger
starting point
is
at this point,
and so more
effort
is
is
the value of the barrier function
generally required to proceed from such a
point.
The
barrier
method can
also be used in equation-solving
mode, to solve
the system:
weS
c^w
for
some given value
for equation-solving
of
6.
=
6
(9)
A description of the complexity of the barrier method
mode
is
as follows:
,
GEOMETRY-BASED COMPLEXITY
•
Assume
S
tfiat
the barrier
is
a bounded set and that w° G relintS.
method
w
more,
compute a point
^^
„
,
w
T
it
is
(11) based
w G
on
[5]
and
Let
line.
S, let
Newton
E
S,
c^w
=
S.
Further-
(11)
3.^|^_^^
||
Iw
:=
\
w
w—
E S,c
5\
.
5
under the assumption that
[7],
w'^
has an interior and
denote the analytic center of T, namely
:— argmiUj^ {F{w)
w
\
E T}
.
denote the norm induced by the Hessian
||^,
barrier function F(-) at w,
the
w
(10)
tt]]
a prominent role in our analysis, we present a derivation
w'^
For
f.:',2"j,
the level set:
will play
contains no
G
will also satisfy:
T
Because
^^
+^—r
that satisfies
.ymKr)>
where
If 6
requires
0{^\n\i) +
iterations to
8
namely
:—
\\v\\^
namely
that all u; €
direction for F{-) at w,
from Proposition 2.3.2 of
[5]
H{w)
of the
Jv^H{w)v, and let i]{w) denote
ri{w) := —H{w)~^VF{w). Recall
T
satisfy \\w
—
w'^W^c
<
3i3
+
1.
There is a fixed constant 7 < 1, the value of 7 being dependent on the specific
implementation of the barrier method, such that the final iterate u' G T of
— Ac||^ < 7 for some multiplier A. By
taking a fixed extra number of Newton steps if necessary, we can assume that
7 := ^. Then from Theorem 2.2.5 of [8] we have \\w - w'^\\u, < 7 + n^hr < f
and so for all w G T we have
the barrier
method
will satisfy
\w
— wWw
||7?(u')
—
<
\\w
<
(itt)
<
<
1(3^1) +
3.5t?
w'^Wyij
\\u'
+
-
w'\\^,c
of
[5]),
c^w =
5
and
and together with
\\w
-
u'||u,
<
+
i
(12)
+
1,
w\\
!
1.25.
(The second inequality above follows from Theorem
if u; satisfies
—
Ww"^
then
w
€
2.1.1 of
T
(12) this then implies that
(also
[5].)
Furthermore,
from Theorem
sym(u',r)
>
2.1.1
3-5^5^:1-25
)
GEOMETRY-BASED COMPLEXITY
Remark
2.1 Note
9
(11) implies that for any objective function vector
tliat
seX':
maxs^w - s^w <
~
weT
(3.5t?
^
+
1.25)
'
(
s^w -
\
mins-^ui
weT
(13)
^
J
'
and
max s^w — s^w >
~
w€T
I
Vs. 5(9
+
)
s^w —
(
1.25/ V
mills'^ u;
weT
)
.
(14)
J
Complexity of Computing a Feasible Solu-
3
tion of
Gp
we present and analyze algorithm FEAS for computing a feasible
Gp using the barrier method in equation-solving mode. The output
of algorithm FEAS will be a point x that will satisfy .4x = 6, x G P, as well as
In this section
solution of
several other important properties that will be described in this section.
computed point x
will also
in Section 4, that will start
of
be used to initiate algorithm
from x and
will
OPT,
The
to be presented
then compute an e-optimal solution
Gp.
Algorithm
mode
FEAS
employ the barrier method in equation-solving
problem denoted by Pi:
will
to solve the following optimization
Pi
t'
:
:=
maximum^
(
t
Az = {b- Ax^)e
z + dx' - tx° e{e- t)P
< 1
e > t
t > -2
s.t.
\\4
where
x""
<
[lb)
1
and x" are the pre-specified reference point and
and where we use the notation aP as follows:
interior-point, re-
spectively,
{x G
aP
:=
{
recP
/)
X
X
I
= aw
for
some
iv
E P}
if
if
if
q >
q =
Q <
(16)
GEOMETRY-BASED COMPLEXITY
10
and where recP denotes the recession cone of P.
Note that Pi
2,
with
w —
mode
equation-solving
value
will
c^w =
is
an instance of the optimization problem
=
{z,t,9), c
J :=
(0, 1,0), etc.
We
will
to solve Pi for a feasible solution (I,
a feasible solution
0, i.e.. for
OP of Section
employ the barrier method
(i,
6) of
i,
then convert this solution to a feasible solution of
t,
9)
in
with objective
Pi for which i
—
We
0.
G p via the elementary
transformation:
+
x^
(17)
(where the algorithm will ensure that 9
>
Q
>
0,
then
that
if (i, 0,
and
9) is feasible for Pi
Ax —
that X from (17) satisfies
and so
(17) will
be
legal).
Note
straightforward to verify
it is
b,x € P.
In order to solve Pi for a solution (zJ,9)
=
{z,0,9),
we
first
must
construct a suitable barrier function for Pi. Let Si denote the feasible region
of Pi. namely:
Si := {{:J,9)
.4.'
=
[b
-
Ax'')B, z
I
+
9x'
-
tx^
G {9-t)P,9 <
l,9>t,t>
-2.
||;||
(18)
and consider the barrier function:
z
F{zJ.B) := -ln(?+2)-ln(l-0)+f|| \\{z)+400 Fp
+
9x'
-
tx°
2dp\n{e-t)
(19)
Define:
d:=2 +
Then from the
we have:
barrier calculus,
Proposition 3.1 F{z,t,9)
Note that
We
1^
=
Ci)p
+
d^
is
and
a
-d
II
+ SOOdp
in particular
.
from Proposition
-self- concordant
barrier for S^.
5.1.4 of
[5],
|
A.
will initiate the barrier
Thus our algorithm
i?||
method
at the point {z,t,9)° := (0, -1.0).
for finding a feasible point of
Algorithm FEAS: Construct problem
ing the starting point {z.t,9)^ :=
Gp
is
as follows:
Pi and the barrier function (19). Us(0.-1,0). apply the barrier method, in
<
l}
GEOMETRY-BASED COMPLEXITY
11
equation-solving mode, to compute a feasible solution
isfies
f
=
=
5
0.
If
such a solution
is
(i,
t,
0) of P\ that sat-
computed, then compute x using
We now examine the complexity of algorithm FEAS. To do
bound the symmetry of 5i at the point {z, t, ^)^:
Proposition 3.2
{z,t,6)^ := (0, -1,0)
The proof
We
of Pi
of Proposition 3.2
will also
and g
deferred to the end of the section.
is
need the following relationship between the optimal value
denote the optimal value of Pi. Then
t'
(min{dist(x°,6>P),l})
We
and
:
Proposition 3.3 Let
The proof
first
mm{dist(xO,aP),l}
^x
,,0
(,
we
so,
a feasible solution of Pi,
is
(17).
of Proposition 3.3
•
5
<
-^
<
+
g [g
also deferred to the
is
+
I
-
\\x^
x'
end of the
section.
next examine the range of the objective function value of Pi. Be-
cause
= max{t
2"
{z,t,e)
I
(from the constraints
<
i
z'
^
<
1
of Fi)
= mm{t
I
=r <
e 5i}
1
and
> -2
{z,t,9) e Si}
,
we have
P:=max{^
|
Finally, observe that z'
{zj,0) G Si} -
< -I
mm{t
{zj.,9) E Si}
\
<
3
(from Proposition 3.2), and so with
(20)
.
(5
:=
we
have:
min{." -
(5,
S-z'}> mm{t\
1}
=T >
,
g{g
where the last inequality is from Proposition
and 3.1 as well as (20) and (21), and using
Appendix, we obtain the following:
3.3.
(10)
^
+
,
l
^
+
|,
p
\\x°
-
^
>
(21)
x'^W)
Combining Propositions 3.2
and Proposition A.l of the
GEOMETRY-BASED COMPLEXITY
Theorem
12
Under Assumption A. algorithm
3.1
and by transformation a
solution {z.t,9) of Pi
FEAS
will
compute a
feasible solution
feasible
x ofGp, in
at
most:
Udp +
O
In
J||
II
Lp +
^ii
+
II
iterations of Newton's method.
1
\x'-x^\\+g
min{dist(xO,c)F),l}
|
Given the output {zj.,9) and the transformed point x given
from algorithm FEAS, define the following
52 := Ix e
The
Lemma
(i)
3.1 Suppose that Assumption
(3.57?
+
2.25) 5
||x-x''||
<
(3.51^
(ii) i
(iti)
M
1
fv)
J^/ 1
<
-
^
>
ll^ll
11-11
>
^
+
A
is satisfied,
>
9)
and x that
will
be
and
let (i, t, 9)
and x
be
2.25)5
U
'^^^'^'\
(3.5t?
3.1:
It
+
3.25)
\
c
Then
PnB{x\i
-5/
follows from Proposition 3.3 that
f >
0,
and so from
method the point {z,t,9) = {z,0,9) will satisfy Az = {b — Ax'')9,
<l,9 >0, and ||i|| < 1. Then ^ > validates (17), and
b, X E P, and ||x - x^\\ = i||z|| < i
whereby we see that x € 52-
9x' e mt{9P), 9
Ax =
t,
(22)
•
g^^^^^a
Let [x, f) be an optimal solution of (4).
the barrier
(i,
< i}
^-
Proof of Lemma
also
x'W
3.5t3+2.25
V
+
-
3.5^+2.25
B
z
\\x
FEAS. Then
X e S2, and sym(I-, 52)
(vi)
Ax ^b,x eP,
I
in Section 4:
output of algorithm
I
set:
following characterizes important properties of
used in the analvsis
tfie
X
in (17)
This proves the
first
assertion of
(i).
'
GEOMETRY-BASED COMPLEXITY
n
Let Ti := Sx
{{z,t,e)\
region of Pi, see (18), and
(11).
(2.iJ)
=
{z,Oj)
-
t
13
0} where recall that Si
n
T2 := Si
let
{{z,t,9)\
+
3.5?^
Furthermore, since T2
Also, the affine transformation
{z,i,9) to x,
it
+
and since symmetry
completing the proof of
Then from
1.25
is
1.25
maps
x'')
T2 onto 52 (see (22))
preserved under afRne transfor-
——--j—+
max
9
=
=
|.4x
x''||
(2,f,0)eri
and 5 >
1
x € P}, and note that
6,
1
1
—
>
r
max{d', 1}
T-.
max{5,l}
Noting as well that
1.
(23)
,
1.25
(i).
-
Let S := min{||x
6
the feasible
follows that
6.0V
>
+
0) h-> (|
t,
(2,
sym(x,52) >
since g
is
6}.
also follows that
it
3.b{)
mations,
=
the intersection of Ti with an afRne space passing
is
sym{{zJJ),T2)>
and maps
0,9
will satisfy
sym{{z,iJ),Ti)>
through {z,i,9), then
=
t
g
m.m(^^t,6)eTi ^
=
0, it
follows from
(13) that
(3.5t9
and rearranging
since
min(^
||j
- x'H =
(gjgT-j
^
=
+
1.25)^
yields i
^<
0, it
i
<
=
(3.5'i}
>
max(,_t,0)eTi
>
—
'--9
+
1.25)(^-min(,,,,o)gri^)
9-9
,
'
9
+ 2.'2b)g.
< (3.5i9 + 2.25)5.
(3.5i^
This proves
Noting that
(iii)
then follows
maX(,,t,(,)6Ti ^
follows from (14) that
l-9>m&X(^^tfl)eTi9-9
>
(
3,5^+1.25 )(^
3.5i9+1.25
and rearranging
(ii).
yields
1
-
^
>
„
,
"
^^t^{z ,t ,0)€T, 0)
'
which proves
(iv).
<
1
and
GEOMETRY-BASED COMPLEXITY
and
We now
= ||i||,
prove
Given
(v).
-
pll
>
max(^,t,o)6T,
^
(
(
3.5^+2.25
proving
)'
A'* satisfying
\\z\\,
=
1
z^z - z^z
~
from
is
E
Appendix. Then
)(^^^
3.5^+1.25
where the second inequality above
<
there exists z
z,
see Proposition A. 3 of the
z^ z
1
IMI
14
min(,,,.tf)gri z'^z)
and rearranging
(14),
yields 1
-
(v).
In order to prove (vi),
we
will use the following claim:
Ax =
there exist (f f) satisfying
,
B{x,
b,
r)
C
P,
\\x
—
+
x^\\
< -
f
,
(24)
9
and
min{r, 1}
'-5(3.01^ +
^^''
3.25)
'
where (i,r) is an optimal solution of (4). We assume for the
without loss of generality that r < 1 and so minjr", 1} = f-
rest of the proof
Before proving (24) and (25), we use them to prove part (vi) of the
Lemma. From (24) and (25) we have x E S2 and so from (23) x^ := x —
^
K-^
3 5tf+2 25
-
€
x)
52.
Rearranging
_^
^
and so
.f
(26) that
is
B
3.57^
3.5?^
+
+
this,
2.25
3.
we obtain
1
1
^
25^
3.5i5
+
3.25^
,
.
^"
'
It then follows from
which combined with (25) proves
a convex combination of x' e S2 and x G 52.
C \P n B
(x, 3-5^^:325)
i)],
fx'",
(vi).
It
1:
{g
+
x||
+
r
Case
X'
and so
Case
remains to prove (24) and
(24)
2: {g
7")^
=
and
<
||x
-
1.
Let x
=
+
<
x'"||
(25).
x and f
We
=
<
i.
-.
+
r.
g
+
r
(1
-
/?)x
(3.bi)
+
2.2o){e{g
r
consider two cases:
Then Ax =
5,
Furthermore,
i ciiuiiv^xiiiwiv., f
,
5(x,
=
r
f)
>
_
C
P, and
g(3.5i3+3.25)
(25) are proved.
+ f)9>l.
Let x
=
/3x,
where
—^
P =
l
+
,
+ r)-l)
(27)
GEOMETRY-BASED COMPLEXITY
and
let f
=
Then
(if.
G
[0, 1],
+f =
||f_x'"||
where the
j3
and so
\\(l
15
Ax =
- P){x -
p{x -
+
x"-)
C
and B{x,f)
5,
P. Also,
+ pf
x')\\
<
{I- p)\\x-x'\\+P\\x-x'\\+
<
[l
<
(l-^)(J)(|f^)+^5 +
_
1
I3f
- 13)^ + Pg + Pf
last inequality follows
from part
/3r
lemma, and the
(v) of the
last
equality follows directly from (27). Also,
f
^
=
I3f
<
since ^
1,
r
<
1,
+
and g >
I-
Proof of Proposition
for Pj.
It suffices
then {0
-
l3d,
3
-.
(3.5(^
+
1
to
-1 -
3.2:
2.25){9{g
-
f)
-
1)
>
"
5(3.5i3
+
This then proves (24) and (25)
Note
show that
a/3,
+
if
first
+
(0
=
+
that {z,t,ef
-1 +
d,
a,
(36) is feasible for Pi,
3.25)
in this case.|
(0,-1,0)
(5)
where
/?
is
=
+
1
- a)P,
6
<l,S>-l + Q,
-2 <
5
<
1
feasible
feasible for Pi,
3+2||J"p-x'-||
'
^^^
< ^ < 5. Since (0 + d, -1 + a, +
is
=
Ad {b Ax^)6, d + 6x^ + (1 Q')x° G
\\d\\ < 1, -1 + a > -2, and it follows that
where r is given by (3). Note that ^
by presumption feasible for Pi, then
((5
is
(5)
-
and
1
< a <
2
(28)
.
-aP,~P6). Then Az = {b - Ax')9, and ||z|| < 1
since /3 < 1. Also 9 = -pS < 2,5 < 1 from (28) and /? < |. Next, notice that
from (28) and /? < i.
e - t = -P5 + 1 + ap ^ 1 - P{S - a) > I - ^{1 + 1) >
Also, t = —I — aP > -2 since a < 2 and /3 < |. It remains to prove that
z + Ox'' - tx° e {9 - t)P. To see this, note first that
Let {z,t,e) := (-f3d,-l
\\-
pd-8p{x' -x'')\\-aP + 5p <
<
=
Then
p{\\d\\
pI?,
r
.
+
+
2\\x'
2\\x'
-x^) +
-x°\\)
2p
(from(28))
"
GEOMETRY-BASED COMPLEXITY
which
IG
same as 2 + 9x^ - tx^ £ {6 — t)P. This shows that
Pi, and so sym((c, f,0)°, Si) > /3 as desired.|
the
is
feasible for
Proof of Proposition
fruiu (4) that
{z,t,0)
and note
3.3: Let (i, f) be an optimal solution of (4),
we can assume that
r
<
1.
From Assumption A,
>
r
is
Define
0.
the following:
3^-^'
-
max{||i
x'"||,
3f
^
^
g_
max{||f -
1}'
x''||,
1}
1
,29)
max{||x -
.r''||,
1}
where
^ =
3+l +
Then /? < 1, since in particular g >\, and from
e <l, -2 <t <9, and \\z\\ < 1. If (zJj) also
z
then {z,t,9)
is
+
9x'
feasible for Pi,
we have Az — {b-
(29)
satisfies
(31)
whereby
> ^ =
- 5
max{||x-x'-||,l}
^
+
g{g
^
I
proving the second inequality of the proposition.
\\x'
-
5
+
1
+
-
||x^
x°||
>
-x'\\+r +
||.r
\\x'
-
(39)
'
x^\\)
Therefore, to prove the
second inequality of the proposition, we must show (31). Xote
^ =
Ax'')6,
-tx° e{0-t)P,
^'
f>t=
(30)
||:,r_^0||-
x°||
>
r
+
that
first
\\x
-
x°||,
(33)
and
^l|x -A\-
Y^Wi - A\ = y-Wi -A\<
where the last inequality above follows from
f < 1), and so
z
+ 9x'-tx°
=^^ +
9-t
t
and 9 >
(33),
0,
r-
t
r,
(34)
(since
<
/?
and
1
r^
rrt(--AeP,
we have ^||f-x°|| < r and B{x,f) C P. Therefore z + 9x^ —
- t)P, and we have proven the second inequality of the proposition.
since from (34)
tx"
G
(^
To prove the
first
inequality of the proposition, let {z*,t',9'') be any
optimal solution of Pi, and note from (32) that
define
x
where r
have x°
is
+
defined in
Td e
P
(3).
from
-+x
=
Then
(3).
.4x
=
Also, z'
,
b,
+
r
=
and
9*x''
>
t*
—
for
0.
Therefore 9*
[9*
-t'\
Z*
+9'X' -t'x°
(3d)
any d satisfying
- rx° € (r - r)P.
fr\,o
and
0,
,
||d||
If
then
^
>
n
r,
<
^
1
we
> t%
GEOMETRY-BASED COMPLEXITY
If
6**
=
r, then
In either case,
x
+
+
rci
?-
<
<
-
O^x'
X
9
since
+
z'
rd
t'x° £ recP,
and so
=
h (x"
G P, and so 5(x,
maxjll.c'"
^"
-
1}
xll,
,''
J
mui{r,
C
r)
e P.
Tci)
P. Therefore
—
f lla:'"
= max
-^
+
1)
1
xll
^
^,
r
r
y
-
Now
1.
_
I
e*
t*r
r
and 11^—^
17
= 1^ <
<
p7, so 5
1
"~
t*T
proving the
p;:,
left
inequality of the
proposition-!
Certain ideas and constructs used in the results of this section arose
from or were inspired directly from Section 3 of Renegar
of solving phase-I by using the barrier
method
[7],
including the idea
in equation-solving
mode, trans-
forming to the original problem via an elementary projective transformation,
and establishing key properties of the output of the algorithm (upper bounds
on norms and lower bounds on distances from constraints) using symmetry
properties of the output of the barrier method.
Complexity of Computing an e-optimal So-
4
Gp
lution of
In this section
of
Gp
we present algorithm
OPT
for
computing an e-optimal solution
method
initiated at the point i, using the barrier
where x
is
the output of algorithm
FEAS.
Using X as a starting point, we
set constraint of the
form "c^x
<
will
c^x
modify
+
s" to
Gp slightly by adding a levelGp for some suitably chosen
positive scalar offset s which will then render the point
<
half-space generated by the constraint c^x
as to
how
to choose the offset
proportional to the
norm
of c
||c|L
s.
optimization-mode,
in
c^x-\-s.
One would
x
in the interior of the
The question then
arises
think that s should be chosen
:
;= m.s.x\c^w
I
||u'||
However, because the objective function c^x of
from the modified objective function
(c
—
<
l[
Gp
.
differs
(36)
only by a constant
A^tx)'-^x over the feasible region of
GEOMETRY-BASED COMPLEXITY
Gp
any given value of
for
From
this perspective,
s
and note
subspace
tt,
we must be mindful
of the equations
Ax =
b.
natural to choose s proportional to:
it is
max
:=
18
|c^ a'
I
<
||u'||
l,-4u'
=
o|
(371
,
maximum objective value over the unit ball, "reduced" by the
constraints Aw — 0, and so s is the norm of the linear functional c^x
s
is
the
over the vector subspace of solutions to
Aw =
L^ norm
computationally tractable norms such as the
not trivial; in fact
computation
However, even
0.
in JR", the
for otherwise
computation
a linear program for the Zoo norm.
of s
is
We
therefore will instead use the information inherent in the barrier function
F\\
its
for the unit ball as a
ii(-)
proxy
for the
H{Q) denote the Hessian matrix of
max [c^w
52 :=
and note that
S2
It
=
$2
\
Aw =
in constructing the offset
•
||
F\\ ||(-)
||
at
0,
x
=
0,
and
w'^H{{))w
<
s]c'^H{{))-'c
for
s.
Let
define:
l}
(38)
,
admits a closed form solution when rank(.4)
be convenient
will
is
=
m
:
- c^//(0)-^4^ iAH{0)-'A'^)-' AH{0)-
our purposes to determine
s
proportional to S2 as
follows:
and we consider the following amended version of Gp:
Ps
:
z*
:= minimum-c
c^x
Ax =
s.t.
b
(40)
xeP
(Fx
< c^x +
s
Note that since x is feasible for Gp, then x is also feasible for Pj,
and Pg and Gp have the same optimal objective function value and the same
set of optimal solutions. (The idea of solving phase-H by adding a level set
constraint of the objective function was used by Renegar [7], but without an
explicit construction for computing the offset s.)
In order to apply the barrier
method
(in
optimization mode) to compute
an e-optimal solution of Fj, we need to specify the barrier function to be used.
The obvious
choice
is:
GEOMETRY-BASED COMPLEXITY
F{x) := Fp{x)
whose complexity value
It is
>
is
0,
(Jx + s-
In
i3p
+
c^x)
(41)
,
I.
and that
>
s
except
when the
objective
constant over the entire feasible region of Gp, in which case x
then an optimal solution of Gp. In light of this observation, the algorithm
for
computing an e-optimal solution of
Algorithm OPT: Compute
tion (41), using (38)
X
most
at
easily seen that s
function c^x
is
is
-
19
is
and
as follows:
is
and construct problem Pj and the barrier func-
(39). If s
the output of algorithm
mode,
s
Gp
>
0,
then using the starting point x (where
FEAS), apply the
method,
barrier
in
optimization
compute an e-optimal solution x of Pg. Otherwise, s = 0. and x
Gp and no further computation is required.
to
is
an optimal solution of
The
bound
rest of this section
Theorem
4.1
by algorithm
is
devoted to proving the following complexity
OPT:
algorithm
for
Under Assumption A, and starting from the point x computed
FEAS, algorithm
OPT
will
compute an e-optimal solution oj
Gp
in at most:
oi^fh'\n {g^- D, + dp +
iterations of Newton's method.
Remark
Theorem
d\\
y
We
<
max
| -, 1
1 J J
\
4.1 Note that we could replace s by
4-1- since s
-h
\\c\\,
in the iteration
bound of
||c||,.
begin the analysis underlying the proof of Theorem 4.1 by relating
the two quantities s and
Proposition 4.1
s:
s
I
<
s
<
s
^
Proof: Since Fy ||(x) is a -i^n y-self-concordant barrier for B{0, 1) = {x ||x|| <
1}, then F{x) := f|| ||(x) - F\\ \\{-x) is a 2d\\ ||-self-concordant barrier for
B(0, 1), whose analytic center is x'^ = 0, and note that the Hessian of F{x)
|
|
GEOMETRY-BASED COMPLEXITY
at
=
X
H{x)
2//(0) where
is
Proposition 2.3.2 of
^x^{2H{0))x < l} C 5(0,
{x
I
From
this
C [x
1)
then follows that 4=52
it
the Hessian of Fy
is
||(-)
at x.
Then from
follows that
it
[5]
20
<
—^|—
<
s
yx^(2/f(0))x
<
3(2,9|| y)
\
S2,
+
and therefore the
l}
.
m
result
follows from (39).
We
will
make use
the level sets of
Lemma
lemma which bounds
of the following
Gp.
4.1 Suppose that x
a feasible solution of
is
Gp
for some given level a, and further suppose that B{x,f)
Suppose that
Then for
Q
all t
Q>
max
>
and for
[\\x
-x\\\ Ax
all
C P
=
b,x e P,c'^x
x satisfying
Ax =
< a]
X e P, and c^x
>
w
Also X
that
< a+
the
t,
r
s
< a + t. Define s'
< Q < Q{1 + |4),
:=
c'^x
—
a. If s^
<
0,
Ax = 6,
<
c^x
a, and
satisfies
then
proving the result. Suppose instead that
x\\
and define
0,
where
(43)
.
b,x £ P,c^x
Proof: Given the hypotheses of the lemma, suppose that x
s^
0.
2t
- x\\<Q\l +
V
—
>
for some r
fiolds:
\x
||x
< a
satisfying (Fx
satisfies:
following inequality
therefore
the growth of
s^
:=
c
TT
c^
\
r
s^
,.
i^
,,,
a — c^x + rs
r^
\a — c^ x + rs +
/
-fc)+\
J
—
X,
s'-
(44)
J
G argmax„{c^u Av = 0, ||u|| < 1}. Then ||c|| < 1, and c^c
fc e P, A(x - fc) = b, Ax = b, X e P, and so it follows from
— b, w E. P, and c^ w = a, since (Fc = s. Therefore
|
-
Aw
\a —
^X + rs +
Q
^ >
—
\
—
^"^
||u.
II
II
II
XT
-f =
11
II
_
7
11
II
f
\a-c' "-^i'-^
x+TS + s^i
a-c^i+fs \
^11/
7"-.Al
i;
\ a — c' x + r5 + s'
II
'
_
t/^^
)c+(
"-/"f
\a-c'
x + rs +
J
fs^
7'---,!
a — c' x-trs+s^
s'1 )
J
(j
^
- f )||
'
''
=
s.
(44)
GEOMETRY-BASED COMPLEXITY
21
Therefore
^z^) + ^z^,
Q{i +
<
\\x-x\\
(45)
P
However, as noted above, x - re G
and A{x -
re)
=
6,
and c^{x -
re)
<
e^x < a, and so
Q^
—
\\x
{x
—
=
fc)\\
f.
Therefore from (45) we have
< Q(l +
||x-x||
Q(l +
<
—
since x^
Lemma
Ax =
b
c^x — a <
B(x,
C P
r)
ll)^
f.|
4.2 Suppose that D^
,
+ 4(f)
fl)
c^f
,
is finite.
<
Then
and f
2:*+e,
there exists (x, f) satisfying
> min {r,
1}
min
<
1,
c^x —
z
(46)
where (x,r)
is
an optimal solution of
Proof: Without
that c^x
B{x,
r)
<
C
that c^x
z'
P, and
>
z*
-\-
we can assume that r < 1. Suppose
x and f = r, we have c^x < 2' + e, Ax
loss of generality
Then
+e.
f
e,
—
f
and
X
setting
=
x
=
min{l,
f
(4)-
^,,^^_.. },
first
—
b,
proving the result. Suppose instead
let
= Ax +
(1
-
A)x'
,
f
=
Ar
,
where
c'^x
and x' is an optimal solution of Gp (x* is guaranteed to exist since D^ is
finite by hypothesis). Then A G [0, 1], and so x satisfies ,4x = b, x & P, and
by construction of A we have c^x = z* + e. Furthermore B{x, r) C P, and
r
=
Ar
=
^T~L.-
=
^ ™iri
1 1
,
^Ti_,.
]
1
proving the result .|
Define 53 to be the feasible region of P^, namely
53 := |x
.4x
I
=
6,
X 6 P, c^x
<
c^x
+
s\
.
GEOMETRY-BASED COMPLEXITY
Lemma
22
4.3 Suppose that x E S3. Then
\\x
-
< ^
x||
where
h :=
3A + (3.5(? + 2.25) g+4gD, {g +
Remeirk 4.2 h
is
D,) [(3.5^
+
2.25) 5
bounded from above by a polynomial
A+
+
679||
m 5,i?p,t'||
y
+
l]
\\,De,,
max
arid
max{f,l}.
Proof of
Lemma
Lemma
4.2.
4.3: Suppose that x
€
S3,
and
be as described
let (x, f)
in
Then
-
||x
x||
-
-
<
||x
ill
+
llx
<
||x-x||
+
A+
x'"||
+
llx""
-
x||
(47)
where the
Ijx
Q
-
x||.
Lemma 3.1.
invoke Lemma 4.1
last inequality uses (6)
To
this end,
we
(3.5i?
will
and
+
It
2.25)5
thus remains to bound
with a
=
+
z*
e.
Then
:= 2De satisfies
maxi{||x -
.4x
x||
=
6,
x 6 P, c^x
< a}
I
<
maxj{||x -
<
2D,
=Q
x'll
+
IJx'"
-
x||
|
Ar =
b.
x e P. c^x
<
z'
+
e}
,
and Q satisfy the hypotheses of Lemma 4.1. Now let t :=
—
and
so a + i > c^x + s. Then if x 6 53, x also satisfies Ax = b,
[c^x
s
a]'^,
X € P, c^x < a + t, and so from Lemma 4.1 we have:
and so
x, f, a,
-i-
||x-x||
<
Q(l
2
+ |)
A + ifi
(48)
< 2D, +
^^/^, maxfl. ^^^^^^^j
5 min{f,l}
<
—
'^'J'^''^,-':^^"'
2D,
'
+
smin{r,l)
(from
maxll, '^^^^}
I
'
(
J
Lemma
4.2
|
-,
1
GEOMETRY-BASED COMPLEXITY
23
But now observe that
c^x —
+
z*
s
=
(Fx — c^x*
+
s
<
s(||x
-
+
||i-*
-
x""!!)
+
s
<
s(||.f
-
+
||x'
-
x""!!)
+
s(6i?||
<
where the
j;'"||
x'll
[{3M +
s
(where x*solves Gp)
+
2.25)5
A+
+
II
6i?||
+
y
Lemma
and
last inequality uses (6)
(from Proposition A. 2)
(from Proposition 4.1)
1)
l]
We
3.1.
.
also
(49)
bound c^x —
z*
using Proposition A. 2:
c^x -
=
2*
c^x -
Jx* <
s{\\x
- x'W +
-
||x*
x'll)
<
~s[g
+
D,)
+
D,)
.
(50)
Substituting (49) and (50) into (48) we obtain
_
II
IK"
-
on
u.
^-^t +
<
2D,
<r
-II
•'^ll
-*g4(3.5i?+2.25)9+D,+6iJ||
||
+ 1] max{l /"'°+^''
}
min{f,l}
+
4D,g[{3.5d
+
2.25)g
+ D, +
6d^\
y
+
l]{g
max
{l,
f}
(51)
Finally, substituting (51) into (47)
Lemma
4.4
<
sym
where h
is
Proof: Let
:—
/?
5^
(3
and so A{x - pv)
:= min |l,
have x
^~}r
+ Qv E
X
P
2 251-/1
so,
s.
=
]
>
b.
(3.5t9
+
2.25)^
/t
(x, 53)
the quantity defined in
X - Pv e 53. To do
c^{x - j3v) < c^x +
Q
we obtain the desired bound.|
^^'-
'
Lemma
^"^ ^
4-3.
satisfy
x
+v
G
S3,
we must show that
we must show that .4(x — I3v) = b, x — /3v & P, and
Note that x G 53 and x + v G 53 imply that Av = 0,
And with
Also, from Lemma 4.3, we have ||i;|| <
/;,.
where z,6 are part of the output of algorithm FEAS, we
(since x
+ av -
rn
x'\
+
<
v
M
E
P
and a G
.„
X - xn
(0, 1]).
T
+ah<
Observe that
Pll
l-||i|l
1
e
e
e
K^-\-
.
1
GEOMETRY-BASED COMPLEXITY
and so x+av € 52
and note that
Q =
{see(22)).
mm (l.^P)
Oh
)
{
24
Then from Lemma 3.1 we have
>
—
^—tjI
minll.--—
(3.5<»+2.25)e/i
^
"^^"{ 1^ ^3.5^-2.25)
\
•^~
3 5,^°i25
Lemma
(from
^<1)
(since
}
^
-^2,
3.1)'
^
J
^
(52)
A(3.5i9+2.25)
Therefore
3^^^^ >
=
^(3.5^12.25)^
'^'
^nd so x-^t; G
whereby i - /3^; G P.
52,
Finally, note that
< c^x +
(F{x-Pv)
— 3v E
Therefore x
Proof of Theorem
<
c^x
+ s3h
<
(Fx
+ s^h
<
c^x
+
and the
S3,
Ps\\v\\
s
result
(from Proposition 4.1)
<
(since fih
.
is
1)
proved.|
To prove the theorem, we invoke the complexity
bound for the barrier method in optimization mode stated in (8). The barrier
F{x) for Ps defined in (41) has complexity value d < dp + I = Oidp). The
starting point x has symmetry bounded by Lemma 4.4:
4.1:
<{?,.od
+
2.2ofh
,
sym(x, 53)
this
4.2.
D^. and max{^, 1}. see Remark
bound being a polynomial in 5, dp, d\\
The range R of the objective function of Pj is bounded as follows:
||,
R<c^x + sTherefore -
z'
<
s{{3.b0
+
2.25)5
+ D, +
6!?||
y
+
1)
bounded above by a polynomial in max{j, 1}, dp,
g, and D^. Combining all of these terms and using Proposition A.l of
i?ll
the Appendix, we obtain the complexity bound of
from
(49).
is
||,
O [\/^ln
iterations of
(g
+ D, + dp +
Newton's method.|
d^
y
+ max | -•
1
GEOMETRY-BASED COMPLEXITY
On
5
25
Natural Norms, and Condition-Number
Complexity
Two
5.1
Natural
In this subsection
on
x*^
The
we
X
Norms on
briefly discuss
Norm.
B,o := [P
For the given point x" G intP, define the set
-
n
x°]
- P\ = {v
[j°
x°
I
Bj.o is
the origin in
norm
B
the smallest symmetric set
assumption that
this
that arise naturally based
and P.
First
Then
X
two norms on
P
contains no
B
line,
u
G P, x° -
1'
G
P}
.
which x" + 5 C P. Under the
be compact, convex, and contain
for
will
and so can be used
its interior,
+
Bj-o:
as the unit ball of a
norm. Indeed
constructed as follows:
is
\v\\xO
mmQ a
:=
s.t.
+
x'^
^ve P
X
(The norm
•
||
||jO,
in either
P
^v
e
a
expUcit or implicit form, appears throughout
shown that r(x°) =
dist(x°,9P) = 1, and so the explicit dependence of the complexity bounds in
Theorems 3.1 and 4.1 on dist(x°,5P) disappears. Also, we can construct a
much
of the analysis in
[5].)
Under
•
||
H^-o,
easily
it is
barrier function for the unit ball B^o using the barrier Fp(-) for P:
F||||(t.):=Fp(x°
whose complexity parameter
'd\\
y
+
i')
+ ^p(^°-^)>
bounded above
is
< 2dp
d\\
as follows:
.
II
Therefore the explicit dependence of the complexity bounds in Theorems 3.1
and
4.1
on
'd\\
y
disappears as well. With this choice of norm, then, the com-
bined complexity bound of the algorithms
O (^p\n
iterations of
(g
+ D, +
Newton's method.
i)p
FEAS
+ max |-,
and
l|
+
OPT
\\x°
-
becomes:
x'\\\]
GEOMETRY-BASED COMPLEXITY
(The norm
•
||
is
||j.o
||u||^o
for V
referred to as a generalization of the
P =
because in the case when
2G
3?"
and x°
=
e,
as
e K".)
The Second Norm. The
second norm we consider
barrier function Fp(-) for P. For the given point x^
||u||f,xO
where Hp{x^)
Theorem
Loo-norm
we recover the Loo-norm
is
:= ^v'^Hp{x°)v
[5]
that 5(.r°,
1)
C P and
so
constructed using the
G intP,
define the
norm
,
the Hessian of Fp(-) evaluated at x
2.1.1 of
is
=
x^. It
then follows from
dist(x^aP) >
1.
Also,
P||ll(^):=-ln(l-.^Lfp(x>)
is
a
=
d\\
1-self-concordant barrier function for the unit ball of this norm.
II
Therefore the explicit dependence of the complexity bounds in Theorems 3.1
and 4.1 on dist(x°,9P) and d\\ y disappear, and like the previous norm, the
combined complexity bound of the algorithms FEAS and OPT becomes:
O
iterations of
(\fo~p\n (g
+ D, + dp + maxl- ,l\ +
||x°
-^ll))
Newton's method.
Relation to Condition-Number based Complexity
5.2
Bounds
In this subsection
bound
for conic
we
indicate
how
the condition- number based complexity
convex optimization presented
in (1)
can be obtained as a
Theorems 3.1 and 4.1. To do so, assume that P is a closed
and for convenience we will assume that C is pointed and has
an interior.
assume that we have a ^c-self-concordant barrier Fc{-) for C,
on X is an inner-product norm
and we assume as in [7] that the norm
:= \/v'^v, and so the barrier function
special case of
convex cone
C
We
,
•
||
||
||'t;||
(u)
F||
:=
-ln(l -v^v)
II
is
a
=
d\\
l-self-concordant barrier for the unit ball.
II
Let us set
x''
rescaled j" so that
will satisfv:
:=
||x°||
and
=
1.
let
x° G intC be given, and assume that
Then from Theorem 17
of
[3],
it
we have
follows that g
GEOMETRY-BASED COMPLEXITY
27
|.T°1
g<3C{d)dist(xO,5C)
and from Theorem
1.1
and
Lemma
3.2 of
[6]
it
follows that
D,<C{df+C{d)-^
\\c\U
where C{d)
is
defined liere using
(2).
Then under
the combined complexity of algorithms
and
4.1
FEAS
the hypothesis that
and
OPT
<
||c||,,
from Theorems
is:
iterations of
e
Newton's method, which compares favorably to
(1).
3.1
GEOMETRY-BASED COMPLEXITY
28
APPENDIX
Proposition A.l:
addition a,b
Proof:
a
+
b
<
We
2
>
then
1,
have
max{a,
>
ln(a +
If a,b
<
\/'2ab
6}
<
2
theri |
6)
<
In 2
+
6^
\/ a^
max{a,
b}
+
ln2
+
+
+
(Ina
lab
min{a, b}
—
=
+
(Ina
^
In 6)
+
a
2ab.
In 6)
<
lii(a
+
b).
If in
.
If also
b.
The
a,b
>
1,
then
results then follow
by
taking logarithms.!
Proposition A. 2:
T„i
x
If x^,x'^ satisfy
J' -^'i
c^x^l
\c
<
s\\x^
Proof: From the definition of
Ic^x-i
-
c^x^l
The
=
\J[x'
-
x^)\
-
=
Ax^
:r^||
<
s
s in (37),
<
following proposition
s\\x'
is
-
(
=
Ax'^
llx^
-
b,
then
xHI
+
llx^
-x
we have
x^W
<
s (\\x'
-
a special case of the
x'W
+
\\x^
-
x'W)
.|
Hahn-Banach Theo-
rem; for a short proof of this proposition based on the subdifferential operator,
see Proposition 2 of
[3].
Proposition A. 3: For every z G
that \\z\\t = 1 and \\z\\ — z^ z. |
A',
there exists z
G
A'* with the property
GEOMETRY-BASED COMPLEXITY
29
References
On
the primal-dual geometry of level sets in linear and
conic optimization.
Operations Research Center Working Paper 999-01,
R.M. Freund.
Massachusetts Institute of Technology, July 2001.
[2
R.M. Freund and
Condition-based complexity of convex opti-
J.R. Vera.
mization in conic linear form via the ellipsoid algorithm.
SIAM
Journal
on Optimization, 10(1):155-176, 1999.
[3
R.M. Freund and
J.R. Vera.
Some
characterizations and properties of the
"distance to ill-posedness" and the condition measure of a conic linear
system. Mathematical Programming, 86:225-260, 1999.
[4:
M. Grotschel,
L.
Lovasz, and A. Schrijver.
Geometric Algorithms and
Combinatorial Optimization. Springer- Verlag, Berlin, 1988.
Xesterov and A. Nemirovskii.
\'.
Convex Programming. Society
(SIAM), Philadelphia, 1994.
in
[C
J.
Renegar.
ical
J.
Some
for Industrial
and Applied Mathematics
perturbation theory for linear programming. Mathemat-
Programming, 65(1):73-91, 1994.
Renegar. Linear programming, complexity theory, and elementary func-
tional analysis. Mathematical
[s:
Interior-Point Polynomial Algorithms
A
Programming, 70(3):279-351, 1995.
Mathematical View of Interior-Point Methods in Convex Optimization. Society for Industrial and Applied Mathematics, Philadelphia,
J.
Renegar.
2001.
2-2.
McM 2co^
Date Due
Lib-26-67
MIT LIBRARIES
DUPL
3 9080 02246 1278