:
:a-:S~~~c-~~,~~-
~B
~W~:~~F7
·::;::--:'~-
i:-
;
:r:`:-small_~
:-.:::::-:
~i~
!~~eR~~:Ns.R~~n~~AE ---Da e
;·:.·
.-
":
·1-.
.
:
Ij
.·
·;
:·
- ·:.-·.· ·.
;'·J:
·
·
- .·r-t
I)·
·-:
:.·
'i
.!·
i;·
;·
:'·
:·
·::
-,
..
·.'
i.--,,:-··
-·
r--
b.-
i.
i·
i.
:··--
>
MASSACHUSETTSBl~~~~i~
INSTITUT
.OF~0TcHNOLOGY
'
A METHOD FOR THE PARAMETRIC CENTER
PROBLEM, WITH A STRICTLY MONOTONE
POLYNOMIAL-TIME ALGORITHM FOR
LINEAR PROGRAMMING
by Robert M. Freund
and
Kok-Choon Tan
OR 192-89
March, 1989
v
r
Abstract
Given a system of linear inequalities and equalities Ax
b + dt and
Mx = g + ht where the right-hand-sides (RHS) are parametrically deformed
over the scalar t , the parametric center problem is to trace the parametric
family of approximate solutions
(t) to the center problems P(t) , where
m
P(t)
is the problem:
maximize
ln (bi + dit - Aix) subject to
Ax < b + dt
i=l
and
Mx = g + ht . We present an algorithm for tracing the parametric family
of solutions
(t) over the given range t E [ t, t] . At each iterate of the
algorithm, the value of the parameter t is strictly increased and a Newton step
is taken. The sequence of values of t exhibit the following geometric rate of
change: If tk and tk+1 are two successive values of the parameter t
generated by the algorithm, then either (tk+l - TMIN)
(tk - TMIN)(1 + 121m)
(TMAX - tk ' ) < (TMAX - tk)( 1
) where TMIN (TMAX) is a lower
128m)
(upper) bound on the smallest (largest) value of t for which Ax < b + dt,
Mx = g + ht has a solution. Thus the iterates exhibit either linear growth
away from TMIN or linear convergence toward TMAX , with a rate of change
1
of 128m , where m is the number of inequality constraints.
or
When applied to the linear programming problem, the algorithm is an
O(mL) iteration algorithm for linear programming, that strictly improves the
primal objective value at each iteration, and requires no dual feasible solution (or
even dual feasibility) to start. After O(mL) iterations, the algorithm either
detects primal unboundedness or produces an interior solution that can be
rounded to an optimal solution to the linear program.
Key Words:
Newton step, center, linear program, interior-point algorithm.
I
Introduction
Given a system of
Ax < b
and k
m
linear inequalities in
equations in
center of the system
Rn
of the form
(A, b, M g)
Rn
of the form
Mx = g,
the (analytic)
is the optimal solution to the convex
program:
m
P:
A in (bi - Aix)
maximize
i=l
s.t.
Ax < b
Mx= g
where
(Ai,
bi )
respectively.
denotes
ith
(See Sonnevend [12, 13] .
(x e Rn Ax < b, Mx = g )
system
rowof
(A, b, M g) ,
A
and
ith
component of
b,
Assuming
is nonemepty and bounded, the center of the
denoted
,
is uniquely defined.
The
computation of points near the center and their properties are important for
interior point algorithms for linear programming and extensions, see
Karmarkar [6],Renegar [11], Megiddo [8], Kojima et. al. [7], Vaidya [16],
Monteiro and Adler [10], Mehrotra and Sun [9], Jarre [5], Barnes et. al. [1],
and Todd and Ye [14], among others.
Algorithms for finding the center are
presented in Censor and Lent [2] , Vaidya [15] , and [4]
This study is concerned with the parametric analysis of the family of
centers as the right-hand-side
varies parametrically.
(RHS)
(b, g)
of the system
(A, b, M, g)
We define the parametric center problem
(PCP)
be the problem of tracing the paramitric family of optimal solutions
the problems:
1
xt
to
to
P(t):
, in (bi + dit - Aix)
maximize
i=l
s.t.
Ax < b
+
dt
Mx= g + ht
d e Rm
(where
interval
t E [,t]
and
,
t, t
and
varies over a given
t
are given), as
h e Rk
algorithm for generating a piecewise-linear function
the property that
We present an
are finite or infinite.
x(t): []
is an approximation to the center
x (t)
is an approximate solution to
P(t), as
the approximate path of solutions.
t
is varied.
Rn
-
with
(t)
x (t) , i.e.,
We refer to
(t)
as
(The sense of the approximation and its
properties are defined in Section 3.)
The algorithm starts with an approximate solution
P(t)
center problem
t = tk
t
t = t .
at
t = tk+1
is chosen as
to solve for
x ( tk + ).
where
The path
guaranteed increase in
(TMAX -
t1)
is
P(tk)
tk+1 > t k
( t)
t
and a Newton step is performed
is then extended for
at each iteration.
In particular,
tk)(i
where
-128m)
(upper) bound on the smallest (largest) value of
t
is
The next value of
t e [tk, tk+1 ]
or
Mx = g + ht , has a solution.
TMIN (TMAX)
t
__
is a lower
for which
Thus the iterate values of
demonstrate either geometric growth away from
2
can be
tk+1
(tk+1 - TMIN) > (tk - TMIN)(1 + 1)
< (TMAX -
Ax < b + dt ,
(tk) .
t
The important feature of the algorithm is the
by linear interpolation.
chosen so that either
k , the value of
At iteration
and the approximate center for
to the
( t)
TMIIN
or geometric
contraction toward
TMAX , with a rate of change of
1
128m
(where
m
is
the number of inequality constraints).
This algorithm can be applied to solve the linear programming
problem in a new way.
LP:
Suppose we wish to solve the linear problem
maximize
cT x
s.t.
Ax < b
Then we can use the algorithm for
PCP
to solve for the path of centers of
the system
LP(t):
maximize
I In (bi - Aix)
i=l
s.t.
Ax < b
cx= t
as
t
is increased.
This yields a new "central-trajectory-following"
algorithm for linear programming that differs from other central-trajectory
methods in two ways.
First, it is a strictly monotone algorithm for linear
programming, i.e., an algorithm that strictly increases the objective function
value of the primal at each iteration, unlike other central-trajectory-following
algorithms.
Second, it requires no prior information or bound on the
optimal objective value, and will process a linear program that is unbounded
in the primal objective value (i.e., dual infeasible), unlike other centraltrajectory methods.
However, the complexity of the algorithm is
iterations, as opposed to
(where
O(~:w L)
O(mL)
for most other central-trajectory methods
L is the bit-size of the problem instance) and so has an inferior
3
complexity bound (by
v ff) .
Perhaps it is the strict monotonicity of the
primal objective value in the algorithm that is responsible for the inferior
complexity bound.
This paper is organized as follows.
In Section 2, we present the main
results regarding the parametric center problem and we present the algorithm
for tracing the approximate parametric path of centers. The remaining three
sections are devoted to proofs of the results of Section 2.
notation and preliminary results.
Section 3 presents
Section 4 contains an analysis of one step
of the algorithm and presents. the results on the use of Newton's method.
Section 5 contains the results regarding bounds on feasible values of
are generated by the algorithm.
4
------
t
that
2.
The Parametric Center Problem
Given a system of
Ax < b
P:
Rn
of the form
Mx = g , the (analytic) center of the system
and equations
referred to as
linear inequalities in
m
(A, b, M, g)
is the optimal solution to the program
maximize
A In (bi - Aix)
i=l
s.t.
Ax< b
Mx = g
(see Sonnevend [12, 13] .)
Suppose
x
(RHS)
as the right-hand-side
center
P .
is the unique solution to
varies parametrically.
Our interest lies in tracing the
of the system
(A, b, M, g)
In particular, we are interested in generating the
x t to the problems
parametric family of optimal solutions
m
P'(t):
A in (bi + dit - Aix)
maximize
i=l
(2.1)
Ax < b + dt,
s.t
Mx + g + ht
In this section, we present an algorithm for generating a piecewise-linear path
of solutions
and as
t
x(t)
such that
x(t)
is close to
^xt
in a suitable measure,
is varied strictly monotonically over a prespecified range.
First note that there is no loss of generality in assuming that the
equations
Mx = g + ht
are not present.
without loss of generality that
kxk
and is nonsingular.
lM = [B, N]
To see this, we can assume
is a
By suitably partitioning
5
kxn
matrix where
A = [C, D]
and
B is
x = (y, z) , we can eliminate the
y
variables to obtain the equivalent
problem
m
maximize
, In ( i
+
d-it -
i=l
iz )
s.t.
where
b = b - CB 1g ,
A = D - CB1N ,
straightforward to show that
solves
P'(t),
solves
zt
and d = d - CB-'lh
P"(t)
if and only if
Yt = B 1 (g + ht - N2)
where
.
It is
X"t = (,
We thus can concentrate
on the more convenient problem
P(t):
maximize
,
n (bi + dit - Aix)
(2.2)
i=l
s.t.
Let X
=
Suppose
(x
Ax < b
xt
Rn Ax < b )
is the center of
st = b + dt - Axt,
Because
and let
P (t)
+
dt .
and
Xt = (x E Rn Ax < b + dt ) .
Xt , i.e.,
Xt
solves
P (t) .
St = diag(st)
is a convex program,
xt
solves
P (t)
the Karush-Kuhn-Tucker (K-K-T) conditions are satisfied at
where
e
if and only if
x , namely
st = b + dt - Axt > 0
(2.3a)
eT S t A = 0
(2.3b)
is the vector of ones of appropriate dimension.
We assume that our initial value of
of
Let
Xt = x e R n I Ax<b)
t
is
t = 0 ,
is nonemllpty and bounded.
6
and the interior
In this case, it is
)
straightforward to show that
X,
will be bounded for all values of t .
Then
T
will be an open interval, and it is
T = (t I int Xt * 0} .
Let
straightforward to extend the analysis in Megiddo [8] to show that the path of
parametric centers
Let
1: t -
is continuous and differentiable.
t, t e T ,
TMAX = sup (te T
TMIN = inf (te T) .
and
Itis also
tt
TMAX = +
straightforward to show that
if d = Ar
for some
of translations of
Xt
if and only
TMIN = -°
r e R n , in which case the sets
Xt
by the translation vector
We therefore assume
tr .
are just a family
does not lie in the column range of
d
throughout this paper that
TMIN > -oo , or
this case, either
and
TMAX < +
In
or both.
,
-
A .
The algorithm presented in this section will trace a piecewise-linear
x (t)
path
that is "dose" to the parametric center path
At each iteration the value of
measure.
Furthermore, at each
t .
algorithm is strictly monotone in the parameter
least one of two ways, as follows.
in a suitable
is strictly increased; thus the
t
iteration, the magnitude of the increase in
xt
is bounded (from below) in at
t
Suppose
is the current value of
tk
t
Then at the next iteration the algorithm will produce either a finite lower
bound
UB
LB
is produced, then
tk+1 _ tk >
i.e.,
TMIN
1
128m
or a finite upper bound
tk+ ,
(UB - tk),
(TMAX - tk+l) <(1 -128
will satisfy (t k +l - tk) >
geometrically, i.e., (tk+1
the new value of
so that
1
-TMIN)
(tk - LB) ,
>(I +
If
so that
7
If
will satisfy
decreases geometrically,
LB isproduced,then
(t -
)(tk - TMIN)
12 8mn
orboth.
TMAX,
t,
(TMAX - t)
(TNIAX - tk) .
128m
UB
TMIN)
grows
t+'
At least one of the above two bounds must be satisfied. Note that in either of
the two cases the geometric rate of change of t relative to a bound is at least
1,
where m is the number of inequality constraints.
128m
Given a linear program in the form:
LP:
cT x
maximize
x
s.t.
as the following equivalent problem
LP
we can reformulate
LP:
Ax < b
maximize
x, t
s.t.
t
Ax < b + Ot
cTx
= 0 + t
Thus an algorithm that traces the parametric path of centers to the above
system for strictly increasing values of
algorithm for solving
will be a strictly monotone
t
The application of the parametric center
LP .
problem algorithm to linear programming is presented at the end of this
section, and is an
O(mL)
iteration algorithm for linear programming.
2.1 Properties of the Parametric Center
For
St = b + dt - Axt , and let
space of
that
let
t e (TMIN, TMAX),
St A ,
i.e.,
d = St ut + Art
indicator function
ut
ut
L-
for some
f(t) as
xt
be the center of
be the projection of
A (ASt
rt E R n
f(t) = eT u t ,
8
AThen
A) AA
T
Xt , and let
St d
onto the range
St d
We now define the path
and note immediately that
note
eTSt A =
since
d ,
f(t) = eTut = eTSt t) eTSt=e=
by (2.3).
,
f (t) = eT St d
Therefore an alternative equivalent definition of f (t) is
follows. It is obvious that
is as
f(t)
The motivation for considering the path indicator function
is the optimal objective value of the
TMAX
following linear programming problem:
TMAX = maximize
x, t
LP:
t
Ax - dt < b
s.t.
Suppose for a given value of
st = b + dt - At , and
P (t) , let
solution to
any feasible solution
(x, t) to
-
be the
xt
Then for
St = diag(Ot) .
LP °
-
1^-1
f (t)
f (t) < 0 . Let
that
t
1
f (t)
f(t-
_ I [-m + tf(t)3
f(t)
TMAX < t -
whereby
(
f(t)
f(t) > 0
Similarly, if
Now suppose that for a given value of
Let
be the center of the system
t'
-t. = diag~b + dt~ - Aj;;)
St = diag(b + dt*-At*) .
(t, t*)
whereby
if both
that
TMIN
TMIN
X
and
TMAX
TMAX
t* , that
t- m
f(t)
f(t*) = O
Ax < b + dt* , and let
T--I
e Tt
eTS d = 0
e St A = 0 and
is the center of the system
and
>
TMIN
t , say
(A, -d) (x, t) < b , from (2.3).
X = ((x, t)E Rn+ l I Ax - dt < b
Thus the set
particular
Then
,
are finite.
are finite.
If
is bounded from below (above) in
9
We say
TIIN
t .
is bounded, and in
X
(TMAX)
is bounded in
t
is finite, we say
Note that if
X
is
bounded in
t ,
f (t*) = eT S t. d
then
0,
t ,
the value of
to be on the upper path if f (t) < 0 ,
0 .
which yields
is guaranteed to exist.
Returning to the path indicator
f (t)
t
f(t)
defined earlier, we define
xt
and to be on the lower path if
The intuition behind this definition is provided in the next two
propositions.
Proposition 2.1 The path indicator function
decreasing for
t e (TMIN, TMAX)
Proof: The K-K-T conditions of
eTSt
Let
t
and
respectively.
and
A
it
unless
A
= 0
P(t)
and
require that
At +t
st
and
xt at t ,
Then differentiating the above expressions yields
st St A = 0
t + st = d .
^-1
f (t) =e u = eT
eT St d ,
-2t
f'(t) = -stSt d =-t
St (A t +
st = 0 , in which case
out by the assumption that
d
t)
=
d = A it .
-St
-2
<
t < 0 ,
But this last possibility is ruled
does not lie in the column range of
10
I
= b + dt.
be the vector of derivatives of
Furthermore, since
then
f(t) = eTut is strictly
A .
m
2.2 ODP~ter and
Lower
Paths)
ProDosition
P
e
and
-low
rooir
(i)
(ii)
TMAX
and
t ER
such that
are both finite if and only if there exists
and
f(t) = O,
f(t) > 0
for all
t
f(t) < 0
for all
t e (t, TMAX);
is finite and
TMAX
all
(iii)
TMIN
TMIN
f(t) < 0
if and only if
-_00
=
for
t < TMAX ;
is finite and
TMIN
all
(TMIN, t),
TMAX =
if and only if f (t) > 0
0
for
t > TMIN
These three cases are illustrated in Figure 1, (a), (b) and (c).
Proof:
center of
and so
TMIN
f(t) = O,
(i) We have seen earlier that if
= ((x, t) IAx- dt
X
TMAX
and
b)
then
(t',
That being the case,
X
are both finite. Conversely, if
TMIN
are both finite, then the center
(x*, t*) of X
,
is the
t)
is bounded
TMAX
and
which is the
solution to the problem
maximize
x, t
s.t.
exists uniquely.
and
Ax - dt<b
The K-K-T conditions for
eT St d = 0 I i.e.,
St = diag(st) .
In (bi + dit - Aix)
i=1
f(t*) = 0,
P°
where
require that
st' = b + dt* - Ax'
The rest follows from Proposition 2.1.
11
eT St' A =0
and
t
- ..LfVIAA
-C(
r
.0
t
1\L/
t (t)
/LA\
r i kJ)
I
Ir
-1
.1
MIN
x
Figure I (a)
12
= 0
-
n
U
t
TMAX
(^ ,,t)
X
)
= ( x, t) I Ax-dt b)
x
Figure 1 (b)
13
t
1
f(t)<O
I
'I' MIN
x
Figure 1 (c)
14
_
__
I
(ii)
Suppose
is finite and
TMAX
We have seen earlier that
f(t) > 0 implies
f() < 0 .
f(t) = 0
But, from (i),if
contradiction.
then
Thus,
f(i) < 0 .
is finite.
TMAX
then
.
TMIN
is finite.
Thus,
is finite, which is a
TMIN
Conversely, if
Consequently, if
Let t < TMAX
TMIN = -
f () <
f (t) < 0 for all
for some
t ,
t < TMAX
then
follows from (i).
TMIN = -
The proof of (iii) is similar to (ii).
2.2 Algorithm
PCP
Before presenting the algorithm for the parametric center problem, we
introduce some more notation and the main improvement theorem.
Recall that
and
Xt
X =
x
Rr ,Ax < b}
= (x X Rn
We assume that
X
is bounded for all
Ax < b + dt) .
is bounded, and so
t . Let
Iv
A
has full column rank and
denote the Euclidean norm.
If
M
Xt
is a
symmetric positive-definite matrix,
wedenote
I y-zlIM =
let xe intX
where
(Y-Z)TM(y - z) .
be given, let
= b-Ax,
S = diag(S) . We say that
IIX- XlIQ(x)
<
1/21 .
Let
and let
be the center of
X,
Q(R) = ATS2 A,
is close to the center of
X if
The motivation for this criterion of closeness to the
center will be apparent from the following theorem, which also serves as a
basis for the
PCP
(Parametric Center Problem) algorithm.
15
Let
int X
Suppose x
Theorem 2.1 (Improvement Theorem).
=(X I Ax < b)
, S = diag () , and Q = ATS-2 A . Suppose
where x is the center of the system (A, b) . Define
= b-A
IIx - XIIQ < 211,
[I
=
S-1 A (A T -2 A)- A T -l]
-
(projection of s-' d)
d
r =
(A T -2 A) -1 AT -2 d
(translation)
a=
=11a
8o11~11
(step length)
Furthermore, define
Sa = b+Suca-AZ
!
S = diag ()
= -(AT S§2 A)1 ATS
XNEW = X
(New approximate center)
q + ar
< 211 . where
Then IIXNEW-XaI
= AT--EW2
SNEW A
(A, b + ad), and
Q = A
In this theorem,
x
X .
u
The vectors
so that
+
d = A + Su
,
Xa
is the center of the system
where SNEW = diag(b + ad - A XNEW). -
and
.
r
The increase in
and so
a
t
from
'Ar + u ,
S-ld =
are defined and satisfy
t = 0
I il ·. Because d
•
Ar
to
t = o
is
for any
is vwell-defined. The Newton step
16
of
x
is given and is assumed to be close to the center
defined next, and is a function of
reRn, I1ul > 0
(Newton step)
e
4
is then
defined using the modified slacks
(the Newton step) and
x, q
theorem,
,
and xNE
(a translation vector).
F
will be suitably close to
NEw
is composed, using
,
According to the
the center of
Xa .
This
theorem will be proved in Section 4.
The increase in
t
I iUll ·
is a function of
from
t = 0 is given as
1/llI U l
However, the fact that
makes good intuitive sense.
is the minimum (least-square) distance of
lies very close to the range space of
A ,
is small. Then changing the RHS by
by a r
Il l
and then changing the
i.e.,
d = S+
RHS by only
"most of
d "
lies outside of the range space of
RHS
ad
is shifting the shape of
X
I ill
Ar ,
d
and so If Iu
is the same as translating
a Su .
Then because
I ul
I
is large, then
is small, we can take a big step. If, however,
of
The quantity
i = 1,...,m . Suppose
1/ i,
ad
is
c
from the column range of
d
in the weighted norm with weights
A,
which
,
Ideally, we would like this increase to be as large as
possible, to speed algorithmic convergence.
proportional to
a = 80
X
A , so that a change in the
substantially, not just translating
the polyhedron. Thus, the step we can take will be smaller.
However, as the next theorem indicates, the value of
important bounds on the values of
show that if
a
Recall that
is small, so is
f (t) ,
TMAX
TMAX
or
and
TMIN ,
[I
Iu
also gives
and thus will
TMIN
the path indicator function, gives important
information about the boundedness/unboundedness of the path of centers
xt
in both directions, according to Proposition 2.2.
17
Theorem 2.2 (Bounds on
and
T AX
Under the conditions and
TM!N).
definitions of Theorem 2.1,
(i) if et/llIll > 1/20,
then
(ii) if eTu/IIlUll < -1/20, then
(iii) if
e
/ I1uII
- I < 1/20,
TMIN
LB
TMAX < UB
LB -
then
TMIN
and
TMAX
<
UB -
22Ym(m+1) + 1
2111ll
22Ym(m+1) + 1
2111 llII
1.6fm(m-1) + .6
IIll
1.6Vm(m-1) + .6
.
III"
This theorem will be proved in Section 5.
Note that either (i), (ii), or (iii) must be satisfied, so that either a finite lower
bound on
or a finite upper bound on
TMIN
course of increasing
t
t = 0
from
to
TMAX
t = ca,
is produced in the
or both are produced.
Case (i) corresponds to being (approximately) on the lower path at
i.e.,
f(0) > 0
path, i.e,.
Case (ii) corresponds to being (approximately) on the upper
f(0) < 0
close to the center of
Case (iii) corresponds to
is increased from
f(0) = 0,
so that
(, 0) is
X° .
Note also that these bounds are
t
t=0,
t = 0
one or both bounds is at least
to
0(m/
IiTII)
Thus, even though
the ratio of
a
t =
-= 1/8011llII
1
Therefore repeated increases in
128m
to
t
using the methodology of Theorem 2.1 will result in either geometric growth
18
in the quantity
with a growth rate of at least
t - TMIN ,
geometric contraction in the quantity
at least
(Geometric
Change
\___________
__ ___ in__
Prorosition
11
__ _ ________ 2.3
___
m
2 .
where
tNEW = a,
1
)
I
or
with a contraction rate of
as the next proposition indicates.
(1 - 1m) '
Suppose
TMAX - t ,
(1 +
t
Relative to
TuiNA
1_···_
or
TAY).
· 114-1-r
With the notation
XOLD
a
are defined as in Theorem 2.1, then
and
XNEW
- X,
tOLD = 0
either
(i)
(tNEW - TMIN)>
(ii)
(TMAX- tNEW) < (1
(
+ 1128 m)( tLD- TMIN)
12;8m)(TMAX
, or
- tOLD)
Suppose a lower bound, either
or LB , is generated through
2
Theorem 2.2. Notice that LB < LB if m
so in either case,
Proof:
TMIN
>
LB = -(1.6
(tNEW - TMIN) =
tOLD - TM IN
tNEW
m(m- 1) + .6)/ I11l 2>- 1.6m /I11 u
+ 1
-TMIN
>-(8oI1i
- 1.6m
)1
IIu II
+1 =
Thus
1+
1
128m ·
A parallel analysis demonstrates (ii) if an upper bound is generated.
U
Now let us return to our initial interest - to trace the parametric center
path
xt
for the program
ranges in an interval
P (t) .
t E [t, t]
Suppose we want to trace the path as
where
19
t >t
.
t
Suppose we are given a
point
(Such a point can be found by
X.
that lies close to the center of
x
using the algorithm in Vaidya [15] or in [4] .)
The following algorithm, denoted
for Parametric Center
PCP
Problem, is an iterative algorithm that invokes Theorems 2.1 and 2.2. At
Step 0, initial lower and upper bounds are set to their extreme values.
initial value of
is chosen as
t
t =I ,
and the counter
-x is set, as is the
r
RHS .
are computed as defined in Theorem 2.1.
which is the increase in
close to the center of
P(tk + a) ,
X +
(tk+'l) =
and
bounds on
TMIN
and
returns to Step 1.
x(t)
is then computed;
, i.e.,
xNEW
approximately solves
t e [t k , tk +
TMAX
a ,
is
] = [tk,
tk +l]
,
x(tk) =
using
In Step 4, the current
are updated, in accordance with Theorem
t kf l
t . If so, it stops.
If not, it
The output of the algorithm is the piecewise-linear path
and the incremental values of
t,
namely
PCP (Parametric Center Problem)
Input:
A
Rmxn , b, d
Step 0
(Initialization)
UB=+oo,
and
In Step 3, a piecewise-linear path
Algorithm
Set
tk
xNE-
x-Ew
In Step 5, the algorithm checks if
2.2.
In Step 2, the constant
as endpoints and interpolating.
NEW
is
In Step 1, the values
point,
according to Theorem 2.1.
is defined in the range
x(t)
t
at the iteration, is defined as in Theorem 2.1.
t
"close-to-center"
The next
(for the
The current value of
number of iterations) is set equal to zero.
The current value
k
The
LB=-o,
Rm
t° ,
t
..., t k ,
, t , t , x
k=O,
t ° =t,
20
x=x
,
RHS=b+dt.
Step 1
Set
(Projection of d)
s = RHS-Ax,
Compute
S = diag () .
u = [I - s A (AT -2 AI AT
-1]
S d;
r = (AT S-2 Arl AT S-2 d
Step 2
(Compute Step Length and Compute New Approximate Center)
Set
80 lull'
Sa = RHS +
S
- A
4 = - (AT S 2 A) 1 AT
XNEW =
Step 3
e
+ ar + q
(Extend Piecewise-Linear Path)
Set
tk+l =tk + a.
For
tk< t < tk+ l
Step 4
, Sa = diag(s,);
,
define
t) = - +[(t
](XNEW
-
X)
(Update Lower and Uvrer Bounds)
· ' a
.
(i) if eT U/1III > 1/20,
·* L~~~~~~
*
then
,~~~~~~~~~~~~~~~~~
LB = max LB, tk
21
(22 mm-- im+ 1Y+)/(
1+ )/22 1 111ul)
;
II
(ii) if eT/
< - 1/20, then UB = min UB, tk+(221m(m+l)+ 1)/(211iu
ll)
(iii) if jeTU/[IIj
< 1/20 ,thenLB = max(LB, tk-(1.6/mm-l)+.6)/Iull}
and UB = min (UB, tk +(1.6 Vm (m-1) + .6)/1ull }
Step5
tk + <t,
If
set
RHS = RHS+da ,
k = k+l,
x-
= XNEW,
and go to Step 1.
If tk+1
t,
STOP.
2.3 Algorithmic Performance
According to Theorem 2.1, if
for each
k = 1, 2, ...,
x (tk)
x
is close to the center of
will be close to the center of
break points of the piecewise-linear path
parameterized center.
Lemma 2.4
(t)
X
X t , then
Thus the
"
will be near the
Furthermore, we will prove in Section 4:
For all values of
t
generated by algorithm
near the center
Xt,
inthesensethat
Q = ATSt2 A ,
and
st = b +dt-A(t),
II (t) - x
kQ
tl
PCP ,
.585 ,
(t) is
where
St = diag(st).
.
We are now ready to discuss the performance of the algorithm.
According to Proposition 2.3, we obtain at each iteration either a geometric
decrease in the gap
the gap
t - TMIN .
TMAX - t
at each iteration, or a geometric increase in
We thus can measure algorithmic performance
according to the change in
TMAX - t ,
or
t - TMIN ,
on whether we are approximating the upper path
path
(f(t) > 0) ,
or both.
or both, depending
(f (t) < 0) ,
the lower
Suppose that in the course of running algorithm
22
;
PCP ,
that a lower bound on
is never generated.
TMIN
Then all iterates
will satisfy criterion (ii) or (iii) at Step 4, so that all iterates will generate an
upper bound, and all iterates will lie approximately on the upper path.
Lemma 2.5
(Algorithm Performance Based Only on
or if no iterates of the algorithm
TMIN = -
bound on
then the sequence of
TMIN ,
TMAX- tk < (1i
)(T
In particular, if
K =
t values
generate a lower
will satisfy
t < TMAX ,
the algorithm will stop after at most
t) / (TMAX- t)l1
iterations.
Under the hypothesis of the lemma, the algorithm must satisfy either
criterion (ii) or (iii) at Step 4. Thus, by Proposition 2.3,
(1
-
128 ) (TMAX
Lemma.
Then
If
- tk)
.
t < TMAX,
ln(TMAX-t
K
let
TMAX-
tk
<
K = F128mln ((TMAX-t)/(TMAX-t) )l
) < Kln(1
< -(In
TMAX - tk+ <
Thus we obtain the geometric decrease of the
< -K(i
Thus,
If
MAX-t)
l28m ln ((TMAX-
Proof:
PCP
TMAX)
+ In (TMAx-t)
l
)+
lnn (TMAx-t)
((TMAX - t)/(TMAX-t ))) + In (TMAX - t)
TMAX- t,
whereby
stop.
tk
t.
Thus the algorithm will
U
23
Suppose instead that none of the iterates of the algorithm generate an
upper bound at Step 4.
Analogous to Lemma 2.5 we have:
Lemma 2.6 (Algorithm Performance Based Only on TMTN)
or if none of the iterates of algorithm
TMAX
PCP
at Step 4, then the sequence of
tk-TMiN > 1 + 128m(*t(I
-TMiN
If TMAX = + °
generate an upper bound on
t values will satisfy
)
In particular, the algorithm will stop after
K = r128m n ((- T IN)/TMIN
))1
iterations.
We next examine the case when the algorithm generates both upper and
lower bounds. We first need the following result, which will be proved in
Section 5.
Lemma 2.7
If criterion (ii) of Step 4 of the algorithm
PCP
is satisfied at
iteration k, then in all subsequent iterations, criteria (ii) or (iii) of Step 4 will
be satisfied.
The significance of Lemma 2.7 is as follows: if at iteration
k
an upper
bound is generated, then an upper bound is generated at every subsequent
iteration.
Lemma 2.8
algorithm
(Algorithm Performance Based on Lower and Upper Bounds) If the
PCP
generates both lower and upper bounds, then there is some
24
such that for
iterate
j
for all
k >j
j , the algorithm generates lower bounds only and
the algorithm generates upper bounds, and
k < j,
(i)
for all
(ii)
for all k > j,
Furthermore, if
K=
k
tk -TMIN
TMAX-t
t < TMAX ,
2 5 6 mln(TMAX
-
k
(1 +
1) k(I-TMIN
128m)
< (1
(TMAX-t)
then the algorithm will stop after at most
TMIN) - 128m In (t-TMIN) -128m In (TMAX-t) +2
iterations.
Proof: The existence of
j
is guaranteed by Lemma 2.7.
The geometric
convergence rates are then a consequence of Proposition 2.3.
suppose
t < TMAX ,
and let
K
be as defined above.
note that
.5 (n (t - TMIN)
+
In (TMAX
- t))
< In
TMAX - TMIN
2
from the arithmetic-geometric mean inequality.
K 128m
1
Thus,
n( -TMIN)+ 128m In (TMAX- t)
- 128m In (TMAX -
) - 128m In (t - TMIN)
25
Let
Finally,
t = tj ,
and
- TMIN
TM
128m In tkTIN)
- -t
128 m In TMAX
\TMAX-t)
According to Lemma 2.5, with
128m In t - TMIN
t
replaced by
t >i
after at most
iterations.
t replaced by
Furthermore, according to Lemma 2.6, with
K
after at most
t ,
tj+1 ,
tk
t
iterations.
.
2.4 A Strictly Monotone Algorithm for Linear Programming that requires
O (mL)
iterations.
Suppose we wish to solve the problem
maximize
LP:
A
s. t.
where
t =
x,
T
x
e R n+ 1
, A'ERmx(n+l)
and eliminating one of the
< b
,
b E Rm
and
(n+1)
Upon setting
variables
of
,
LP is
easily transformed to the form
maximize
LP:
x, t
Ax < b + dt
S. t.
where
x £ R n, A E R m xn
transformation of the data
algorithm
PCP
b E R'
, and the data
(A, b, ) .
to trace the path
(t)
Ax < b + dt .
26
(A, b)
We can sol,ve
LP
are a linear
by using the
of center to the parametric problem
x, e R n
Suppose
is a given starting point for which
(x ° , t) = (x °, cTxo)
satisfies the starting criterion of the algorithm
namely
Q° < 2x
-xt°l
and
Q = A' T(S
path
x(t)
for
where
21
A .
so = b+dt
- A
xo > O
Then we can use algorithm
t E [to, TAX)
where
= [cTx,z*)
objective value of the linear program
PCP
LP .
z
PCP
S
= diag(s °)
to generate the
is the optimal
The sequence of values of
tk
k = 0,...,, will be strictly increasing, according to Theorem 2.1, i.e., the
objective value will be strictly increasing at each iteration.
total number of bits in a binary encoding of
LP .
Let
L
be the
In order to evaluate the
algorithm's complexity, we consider three cases.
Case (i): The linear program is unbounded.
never generate a finite upper bound on
After
k = O(mL)
iterations,
2 L , and we can conclude that
In this case, the algorithm will
TMAX , which equals infinity.
t > (to - TMIN)(1 + 128m)
LP
is unbounded.
Case (ii): The linear program is bounded and
indicator function at
t =
to
, is negative.
f(to) , the value of the path
This being the case, the algorithm
will always generate upper bounds, and after O(mL)
(z* - tk) < (z* - to)
is less than
(1 -
will exceed
2
,
iterations,
herebv
xt)
canbe
rounded to an optimal solution, see Karmarkar [6].
Case (iii): The linear program is bounded and
can show as in case (i) that after
otherwise the
LP
k = O(mL)
would be unbounded.
27
f (to) > 0 .
In this case, one
iterations, that
f(tk) < 0
,
for
Furthermore, after an additional
k = O (mL)
iterations, we will obtain via case (ii) that we can round to an
optimal solution.
Thus, after
iterations, we can round to an
O(mL)
optimal solution.
Note that in either of the three cases, that algorithm
process
LP
after O (mL)
PCP
will
(by detecting unboundedness or producing an optimal solution)
iterations.
This algorithm falls into the class of central-trajectory based algorithms,
but is inferior in that the bound of O(mL) iterations is worse than the bound
of O(lfi-L) iterations for algorithms such as Renegar [11] or Vaidya [16] that
trace the (weighted) center of the system
Ax < b
cx 2 6
as 8
O (I
is increased, or to the bound of
iterations for algorithms
L)
based on barrier penalty methods that trace the solution to
m
maximize
cT x + £
n si
i=l
s.t.
Ax+s = b
,
s>O
see Monteiro and Adler [10], among others.
All three methods follow the same path in their idealized version.
Yet the latter two obtain convergence in
superior to our algorithm.
O (Vi L)
iterations, which is
However, these other algorithms do not
guarantee strict improvement in the objective value, (but do guarantee strict
improvement in the duality gap).
In contrast, our algorithm will guarantee
strict improvement in the objective function of
28
(z' -cT X)/128m
at each
iteration.
Perhaps it is the implicit imposition of the strict improvement in
objective value that increases the iteration bound by a factor of
Furthermore, our algorithm does not assume that
Instead, our algorithm will detect unboundedness of
LP
O ( ff ) .
is bounded.
LP
directly.
As a final note, note that our algorithm can be used to mimic
Renegar's algorithm [11], tracing the center of
Ax
<b
-cx < -t
as
PCP
t
is increased.
Thus, Renegar's set-up is a special case of the problem
we are considering.
However, we see no way to cast problem (2.2) as a
special case of the set-up used by Renegar [11]; we allow all
RHS
values to
vary simultaneously, which is apparently more general than in his work
[11].
The remainder of the paper is devoted to proofs of the results
presented in this section.
preliminaries.
algorithm
In Section 3, we present notation and
Section 4 contains an analysis of a single step of the
PCP , and contains proofs of Theorem 2.1 and Lemma 2.4.
Section 5 contains an analysis of bounds generated by the algorithm, and
contains proofs of Theorems 2.2 and Lemma 2.7
29
Notation and Preliminary Results
3.
In this section we present notation and some preliminary results that
will be used in the proofs of Theorems 2.1 and 2.2 and in subsequent analysis.
Notation and Translations
3.1
For a vector
matrix
1IMl
v ,
I NI I
M ,
IlviI
denotes the Euclidean norm, and for a
denotes the usual matrix norm, i. e.,
= sup 11Mv1l/IllVI
v•:0
Note that if
If
M
M
{]M1
is a diagonal matrix,
= max
miil
i
is a positive definite matrix, the
M-norm
of
v
is
Iv IIM = vTMv
PM: = I-M (TM)
The matrix
1
denotes the orthogonal projection
MT
matrix which projects onto the null space of
MT
Let
Qt(x; u)
the negative of the Hessian of the function
m
ft(x; u): = X n (bi + tui - Aix),
where
u e Rm
is a given vector
i=1
parameter.
Let
A (x): = diag(b- Ax)
T-
Then
and
Qt (x; u) = A T A t (; tu) \
Q(x) = AT
At (x; u): = diag (b + tu - Ax) .
When
-2 (x) A .
30
t = 0 ,
e denote
denote
Let
t (u)
Ax < b + tu , and let
denote the center of the system
denote the center of the system
Suppose
d = u + Ar
Ax < b .
for some
At (x; d) = At (x - tr; u)
observe that
Rm
u
and
r
Rn
RHS
by
d
and by
u
tr , in that
simply corresponds to a translation of the inequality system by
R n I Ax < b+td} = (x E Rn
We
Qt (x; d) = Qt (x - tr; u)
and hence
Thus, the difference between modifying the
({x
x
b + tu} + tr
Ax
The following Lemma is therefore obvious.
Lemma 3.1:
(i)
Suppose
t (d) =
(ii) xx
- Xt
(iii) max
d = u + Ar
for some
u E Rm
and r E R n . Then
t (u) + tr ,
(d)|
Qt(Zt(d);d) =
t I Ax
b + td
(x -
tr) - Xt (u) Qt(Xt(u);u) for any
for some
x
= max
t
xE R n ,
Ax< b + tu for some
In the sequel, we shall be working with appropriate choices of
and
rER
instead of
d .
,
u E Rm
As we shall see in Section 5, this in fact is
central to the construction of the proof of Theorem 2.2.
u E Rm
For the appropriate
where convenient and the context is clear,
we let At(x): = At(x; u),
A ()
and
= A((),
31
St =
t (U) ,
= b-Ax ,
x
.l
S = diag (-)
and
s
b
diag () , st = b + tu - A ,
A-X
St = diag(st) .
We also will abbreviate
Qt (x; u) by Qt (x).
The next Lemma presents some basic inequalities. It is essentially
Proposition 7.2 of [4] with some simple extensions.
Q(x) = AT
E Rn
Suppose
Lemma 3.2:
-2
satisfies
and let
xe R n such that IIx-IIQ(5)
Then for any
A.
s = b - Ax > O
< 1,
we have
(i)
> 0 ,
= b-A
(ii)
II A- 1 (x) A (z) = IS-
(iii)
lIA-
1 ()A
and for any
(iv)
=
()I
v
I I IQ() <
1sI Sll
1
1-6
<1+6
,
Rn
1
1- 8
lv
(V) 11
VIIQ(X) < (1+a)l
3.2
<
HQ(-)
where
= AT -2 A , and
Q ()
VIQ(x)
.
Eauivalent Measures of Closeness
In this study, we measure how close a point
thesystem
Ax < b
with the norm
I IX -
IIQ()
32
_
x
is to the center
x
We shall also make
of
use of a different measure of closeness to
x
that was introduced in [4 , and
we will show a certain equivalence of the two measures.
Define
Q = 1 ATs-2 A, y=y(R) =
m
m AT S e
(3.1a)
(m - )yT Q y
and y = y(x)=
(3.lb)
1 -yT Q y
Note that
Q
Ax < b.
Then
Let
Q(X)
=
y(R) = 0
denote the center of the system
andso
is used to measure the closeness of
Lemma 3.3
Let
([4]).
h > 0
Proof:
I
-x
x
In [4], the scalar
to the center
7 = Y(x)
x .
be the center of the system
Ax < b .
be a given parameter.
Suppose Y = Y(X)
Then
Let
y())= 0 .
l
<
g
2 Q()
in ( +h))
h
(m - l)
where g(c) = 1- (c +
h 2 (1 +
-(xal)2+
2)
(1 - hy) 2
Follows from the proof Lemma 7.2 of [4] with weights
w = (1/m)e.
.
We shall say that
Corollary 3.1:
If
x
is approximately centered if
y(x) < .0072
)/2.
then
33
il X-
y(x) < .0072
Q(x) < 1/21 .
Proof:
Let
h = .03
and
IIX-XIJ2 Q(X) <
we have
Substituting for
ca =
m
m-1
h and 7y,
In (1 + h)
Then from Lemma 3.3,
h
(1 - hy) 2
and noting
m >2
gives the desired result.
Therefore, for appropriate values of
h ,
approximately centered implies
is close to the center by our criterion,
IIX
thatis
Q()
-
Lemma 3.4:
the system
<
1/21 .
Suppose
x
(e.g.,
Next, we show the converse implication.
X X1IQ(X) < 6 < 1/2
where
x
is the center of
Ax < b.
Then
=
(x) < a + 2f2 ,
Proof:
Let
= b - Ax
From Lemma=
of
and
62
2(1 - ) (1 - 26)
xwhere
s = b-Ax
x-xQ(2.1
From Lemma 2.1 of [ 4], with weights
I
1-6
,,
From Lemma 3.2,
II-x
iln Si
In 'S-
M--~
U
i=l
Z
34
iQ(X) < 61
-
< 1.
1-6
= (1/mn)e ,
m
i=1
X is
h = .03 )
I
where
-
1-6
<
1 .
.
On the other hand, from (2.4) of [4],
m
2 (mml1Ž
in si
i=
7=
(x) .
i=l
Therefore,
Thus,
where
-,J,/
+ 2)
2
a =(a
>
(1 + -
a= 2(1-c) -
Y< a +
where
,
2y).
a =
a2
2(1 - a)
62
2 (1 -
.
)(1 - 26)'
Finally, we present some elementary inequalities.
Lemma 3.5:
(i)
For all
(ii)
For all
Proof:
Since
Assume
Note that
m-1 +9
2
m
2
> 0,
F <
Let
Q, y
< £
,
implies
and Y be defined as in (3.1) .
(m - 1)yT Q - y
yT Q-1 V I F implies
(m -
- 1) yT Q-1 V
_ (m
9
.
whereby
To prove (ii), note that
(i)
£
< £ /( 1-)
yT Q-1 y =
1 - yT Q-1 y
m-1'
<
m-1 +y
follows immediately.
(m - 1) T Q- y < e
implies
£
U
-1)
35
2
.
Analysis of One Iteration of the Algorithm
4.
In this section, we analyze one iteration of the algorithm and prove the
First, we show that
Improvement Theorem (Theorem 2.1) and Lemma 2.4.
if
Itl
is small then the two centers
^x
and
are sufficiently close to
xt
each other with respect to some appropriate norm (so that Newton's method,
when applied, will converge).
Theorem 4.1:
denote the center of the inequality system
and let
xt
u^
() u .
=
-
b
Ax < b + tu .
Let
Itl < 1/(76 11 1) .
Suppose
11-
Then
XtQ
t()
< 1/12
The proof makes use of Lemma 3.3 of the previous section. We want
Proof:
= y(R)
to show that the quantity
note that
T
A
y
ATA
(^) AT
y()A:
-1
() e = 0
Ie
(2)e
-AT
Ax < (b + tu)
y
and
(see (2.3)), and so for the system
=
T A-l
[A
e
'
is
Q .
First
Ax < (b + tu),
- ( x) e
(XAT(x)(t u)
TA-1 1 xA-,
= 1 AT
Also,
for the system
We begin by giving expressions for
sufficiently small.
=
Ax
denote the center of the inequality system
Let
LQQ
( Q= ,)
Q
Alson
At
(x)(t u),
by definition of
AT At 2 (X,)A
n-M
36
u- above.
and so ,
yT Q-1 y =
l (t )T At () A (AT At () A) T A
AZ ()(t
because the eliminated matrix is a projection matrix.
y =
(X)
yT Q-1 y (m - 1)
1
It
-
1~tQ1-Corollary 4.1:
1
5,775
Hence
and
y < .0132 .
2h2 (1 + 2)
.
'12J
(1 - hy)2
Under the conditions of Theorem 4.1,
) < 1/I1
- Xt
Follows from Lemma 3.2(iv).
.
Furthermore, using Lemma 3.2(ii), the following corollary is
immediate.
Corollary 4.2:
Let
d
Rm
2
h = 1/18
IIX-
Proof:
<
1-lu]
1 - yT Q-1 y
Thus, from Lemma 3.3 with
- 112
) < 1 ||t-1
Suppose
I -XI Q)
be given and define
<
1/21
u: = PA
37
Sd .
Let
xt
denote the center of the inequality system
Suppose It
1
Then
8011u11ll
t A- 1 (x)Su
Ax < b + t S u .
< 1/76
and
X XtIIQ,(^,~t < 1/11
Proof:
Let
and
u = Su
u^ = A-1(x)S
,
u = A () u .
Note then that
so that
Iu
from Lemma 3.2(ii).
It[ <
A-1(x)S
l
201
u-11,
Thus,
1
< 21 .
80
from
Theorem
we
4.1, have 20
Hence, from Theorem 4.1, we have
I
1
<_
1
8011u11 - 7611 uII
-X tIlIQt(X) < 1/12 ,
llx-xtllQ,(^X,) < 1/11
4.1,
and by Corollary
.
We next use a theorem of Renegar [11] which gives the region and rate
of convergence of Newton's method for our problem.
Theorem 4.2
(Renegar [11]):
x E int Xt = {(x e R n
£
=
x -
Assume
Ax < b + tul
t IQt (t) < 1 w here
X.
and
is the center of the system Ax < (b + tu).
38
Let
,
st = b+ut-A
Qt = AT
Then
2
and
St = diag (t)
xt = x-Qt
1
where
( x ) AT -St ee ,, where
A.
|xt-xtIQ
(1 + 6)2
t (Xt)
.
-E
We are now ready to prove the Improvement Theorem.
For the
reader's convenience, we restate the theorem before proving it.
Theorem 2.1
(Improvement Theorem):
1.~~~~~~~~~~~~
Suppose xe int X = x I Ax<b}
Define
U: = P-S-1A
satisfies
1
Further, define
8011
=
I
XEW -
;
S, : = diag(s) );
(A T S--2 A) -1 ATS
IQa(Ew)
1/21
(step length) .
e
(Ne-wton step);
(Nelw approximate center) .
x-\w: = x + q + c.r
Then
<
(translation vector of system) ;
so,: = b + cc S u - A
-:
|Q(X)
(projection of S'-1 d)
S-'d
r: = (AT S-2 A)-1 A T S-2 d
a: =
IX -
1'/Il
39
Proof: First note that d = u + Ar,
I
(dl Qa ((d);
NEw -
and
Xa = X (U)
that
Q ()
II xa -
x
(d),
Qa (x)
< 1/22,
.
Lemma 4.1: Under the conditions of Theorem 2.1, define
Let
inequality system
Ax
]
tE [O,
where
which, by Theorem 4.2, implies
< .033 < 1/22
as in Theorem 2.1.
= XNEW
In Lemma 4.1 (iii) below we
(u); u) .
= Qa (
Qa (ca)
X xa
-
E: = I X- XaIIQ(() < .1462,
show that,
(u) I lQ (x(u); u)
it suffices to show that
6 = 1/22 ),
and
a
Hence, by Lemma 3.2 (iv) (with
+
where Xa =
IIxa -
d) =
u = Su . Thus by Lemma 3.1,
where
u = Su,
let
b +ut
and let
and
u,r,
denote the center of the
xt
Qt(x) = Qt(x;u)
For all
I
(i)
1II VQ(X)
- 10 76
(ii)
I vIIQt( )
-
(iii)
I
x
- t
12~
IQ,(~'
lvllQ(xt)
i7 I v I IQ(;)
;
for all
v E R
for all
v E R n ;and
• .1462
Proof: From Corollary 4.2, wve have
lt
A- (x)u
176 andc
< 76
1 - X; 1I()
<
1/11
(4.1)
Thus, by Lemma 3.2(ii) andl (iii)
.... )
III )
IA1 (xt)At ()
12
11
and
A ( ) Al (
40
< 11
10) '
(4.2)
Also,
A-t ()A
bi - Ai x
max
i
bi + t ui- Ai
()=
max
i
I
+t+ t
u)
bi- Ai _
1-l
because
I t A- ()
1
I < 76
75
t A- () u l
< 1
u
from (4.1).
76 '
(4.3)
Similarly,
max bi + tui-Ai
i
bi -Ai X
= max
I +t
i
<
Hence,
II ll IQ()
1
u
bi - Ai
tA-l(X)ull
< 727
(4.4)
76
= l A-1(x) A vI
A 1-)a ()At(
K
A 1 (X)A (t) l
Z77
)(i.)HV uQ(i)
76 101
This proves (i).
I
""
I AI-1
~ (Xt)t
from (4.2) and (4.4) .
The proof of (ii) is similar, using (4.2) and (4.3).
show (iii), we have by the triangle inequality,
41
Avii
Next, to
I1R-Rt1IQ'(^XJ < 11x -
x
Xt +11---tllQ,(^)
x
x
xt
IIQ,(^)
+ l1,
11
11 ) 75)• I-xIIQ(1)
· i111
Ix--x11Q(X)
since
201
775
+
<
211-xIQ(-)
-20
-
<
11
<
from (ii) and (4.1),
.1462
1
20 '
U
from Lemma 3.2(iv).
We are now ready to prove:
Lemma 2.4:
For all values of
near the center of
t
generated by algorithm
PCP ,
x(t) is
in the sense that
Xt ,
I x (t) - t I (,(t))
t
< .585
Proof:
Let
as in Theorem 2.1.
ox = 1/(8011U I)
I x (t) -
Lemma 3.2(ii), it suffices to show that
Xt
te [0, c] .
Let
IIQ ()
<
By
0.369
We have, from the proof of Theorem 2.1, and Lemma 4.1(iii),
| Xa
X I Q (a)
Hence,
IXa-
I
lXIIQ(X)
Z'I| IQ(x) + IIx - RC IIQa(^a) < .033 +.1462 < 0.18
<
1
<
75
(by Lemma 4.1(ii))
xa
72 10 7611
< 0.222
42
(
(by Lemmna 4.1(i))
Next, observe that
x (t)-
(t)-- X Q((a -
= t (xa-x)
O~
x)
l lQ (x,)
<
Therefore,
0.222(t/a)
Thus, by the triangle inequality and Lemma 4.1(iii),
IIx(t)-
t IQt ( t) < Il(t)-xQllQ,(,) + Ix -tlQt(Q,)
X
< .222 + .1462
X~~~~~~~~~
< .369.
5.
Lower and Upper Bounds
In this section, we analyze the upper and/or lower bounds generated by
PCP , and we will prove Theorem 2.2 and Lemma 2.7.
algorithm
Recall that
T = It
I intXt
0}
!
TMAX: = sup T
TMIN: = inf T . We shall derive upper bounds on
TMIN ·
iterate x satisfies
x
The current value of
. We decompose
d = u + A , where S-1 u
0 .
is
t= 0 ,
and the current
Ax < b ,
< 1/21
, S = diag(s) and
= b- A
d into
u =S-l'AS
d
and
A) 1 lAT s-2 d
2
(5.1)
By assumption,
u
t
= b-Ax > 0 and lX-XlIQ(X) = II(^ -)II
is the center of the system
S = diag()
r = (AT
and lower bounds on
We shall, for convenience, adopt the following notation and assumptions
throughout this section.
wvhere
TMAX
and
d
does not lie in the column space of
A
and so
We shall prove Theorem 2.2 and Lemma 2.7 in this section.
begin with three fundamental Lemmas.
Lemma 5.1:
eT S
Proof:
eTS
u - eTu
<
-1 U
111/20
II
IeT(S1 _
u
S)
- ;nS-S) U1
<
<
44
is-s)
(1:' 2)
Sull
i Il
We
The first inequality is the Cauchy-Schwartz inequality and the last follows
I x - x Q()
from Lemma 3.2 and the assumption that
Recall that
If
f(O) = eTS u
Therefore, the current center
x
1/21
is on the lower path and
eT
TMIN
_11 u 11/20
then
TMAX
is
- m/ (eT S-1 u)
is bounded above by
However, we typically cannot deduce the exact value of
- m/(T Su)
eT S u
<
t=0.
Ilull/ 2 0 >0.
eTS u 2 eTu-
bounded below, as we have seen in Section 2, by the bound
Similarly, if
.
is the path indicator function at
/20 then from Lemma 5.1,
eTu > u[
<
from the current iterate.
Therefore, we shall later in this section
derive an alternate bound using the next Lemma which is a variant of
Lemma 7.1 of [4].
Lemma 5.2:
x
Let
be the center of the system
intX = (x I Ax<b)
the ellipsoid
FOUT
FOUT: =
Then
Proof:
Let x E int X
IIX-XIIQ(X)
is given such that
Suppose
< s < 1.
Define
by:
x E Rn
intX
Ax < b .
I
J1X- 1 1 Q(X)
(1
+ 6) Vm (m-1) + }
(5.2)
FOUT .
be given and let s = b-Ax . Then from properties of
the center, (see for example [4] , Theorem 2.1),
45
Ix-Xl Q()
<
m (m-1) . From
Lemma 3.2(v), we also have
IIX-
IIQ()
<
IX-X IIQ(X)
+
(1 + ) m (m-)
I x- XIIQ(X)
<
II -
< (1 +
IIQ(X)
)/m
(m-l)
Thus,
+
.
6
The next Lemma concerns the well-known classical least-square
problem or minimum-norm problem.
Lemma 5.3:
Given an
mxn matrix M
and an
m-vector d ,
for all
scalar t,
IIMx-tdII
where
2
It IIPMdII
for all x
R ,
PM = I-M(MTM MT
We are now ready to prove Theorem 2.2 and Lemma 2.7.
We shall
prove Theorem 2.2 in two parts.
Proposition 5.1:
(i) if eT
(ii)
Proof:
if
(i)
eT
Under the definitions and conditions of Theorem 2.2,
l ull < - 1/20
/
II >
Suppose
TMAX < UB
then
TMIN 2 LB
then
/20
< -1/20
eTull l
.
(22
u
irnm+1Y+ 1)/21 11
-(22imm+1
We first show that
+ 1)/ 21 11 I .
E
is
close to the center of the following extended system with one additional
variable and one additional constraint:
A t
<
(5.3)
l j
46
A
A =
where
A
-u
0
0
I
= eTS' l
and
Note that from the remark following Lemma 5.1,
0<0
TMAX = SUP ( t I Ax < b + dt
for some x)
sup t I Ax < b + ut
for some x}
=
u .
Therefore,
t
I
t s|
<[ b
for some x
1
For system (5.3), define
A(x,t): = diag([l b
1
Then,
A(2, ) = 0 S
[eT, 1
Therefore,
X
{[]
I
A(, 0)=
[ T 1-O
(, 0) A = [eT S-1 A, -
[
and
[
Q(x,t): =
and
0 1
^-1
eT S
+
= 0
is the center of system (5.3) and
2]
x.
O
[o] -KoJ
Let
- xlt ])
E
=
IIx-x
I
Q(x,o)
Rn+l A[
t
<i'
b
I
47
Q(X)
I1
< 1/21
and
AT --
A
(2t)
(x,t)A .
E R n+
FOUT =
Then, from Lemma 5.2,
Furthermore, because
int X
-AT
S A
-U T g`2 A
-
0'
-
IQX,
- , 0)
< 22
21
m (m+1}
+
12
FOUT.
0 = 0ASAT -
T -2
Q(X,O) =
[ :]
l
S-2 u
= A
AT S-
K
2 u
2 + uT -- 2
, we have
ATS -2 A
0
0
02+llull 2
Thus
ttJ
12(Xo
-[
+ t2 (02+ I1UIl2)
= S A(x-)
)
> t211ll1
e
Also, for any
FouT ,
2
t[ -
[o
Q(X,O)
22 m (+
1+
21
22 Vm (m+1Y + 1
whereby
21 I1 II
It then follows from Lemma 3.1 that
TMAX =
UP
(
t
=sup(t
Psut
Ax - tu < b
for some
x
I Ax - tu < b for some x , Ot <1 )
t
int
X
for some
48
x
I
(since
< 0)
1
:5 max
<
(22
ttl [
1
FoUT
x}
for some
(since int X
FOUT)
m (m+1) + 1)/2111U11
.
The proof of (ii) exactly parallels that of (i).
Proposition 5.2:
<
(i)
TMAX
(ii)
>
TMIN LB
Proof:
Ie T u I/I Il
Suppose
UB -
<
/2o
Then
(1.6 /m (m-l) + .6 )/111
I
= -(1.6
We shall show that
m(m-1) + .6)/ 1u l
is approximately centered (in the sense of
0
Section 3.2) for the extended system
[A, -u
Let Q,y
Section 3.2.
Let
for system (5.4).
and
Y = y()
Q,y
and
(5.4)
for system
=
(, 0)
be as defined in
be the corresponding parameters
= b-Ax = s > 0
U_
S'
= -1
S
uU
and
AT S-Iu = 0
r
ja~= I
Ax < b
Observe that
b [A, -u]
Therefore, since
x
Q
0
-2[A, -u]
L
49
I
0
Il]2/m
I
and
and
y
S§- e
Ym
-U/M
-eT
Therefore,
-TQ-l
I x-
Recalling that
yTQ~y + (eT )2/m!lIll 2
x I
(-)
< 6 = 1/21,
2
.
< (a+2
<
++f
1
Therefore, by Lemma 3.5(ii),
h = .41
a =
1
760
and
Next from Lemma 3.5(i),
Thus,
'Y
(m-l)y Q
2a.
Y < a+
conclude from Lemma 3.4 that,
(m-l) yT Q-1 y <
wecompute
(m-l
)+M
IeTu II i 2
1
1
2
< .0054
< .0053
and so
y < .0735.
Taking
in Lemma 3.3, we have
h2 (1 + 2)
xl
*1
(
0
:
where
x]
M 1
0)
.36 < (.6)2 , (since
2
(1 -h)
is the center of system (5.4) and
m/(m-l)
Q (,0) = mQ
Note
that
Note that
Let
[ ]-
X: = [x
t
[o]
[Ax
=
u]
i
It i
S'
A(x - x)- tu I .
b• and
I
50
I
-
<
2)
FOUT: =
tt
I1S-' A (x - )- t
Then by Lemma 5.2,
it]
E FOUT
1.6 /m (m-1) + .6
By Lemma 5.3, for all
FOUT .
int X
I<
m (m-1)+ .6 t jj-S 1A(x-R-t-ujj z! I
, 1.6
since
IJ
Thus
ATS-1 U = 0
sup (ItI ti T/ =
sup
(It I X I E- int X
< max (Iti
x]
for some
x
for some
E FOUT
< ( 1.6 /m (m-1) + .6)/1l
x
II
This completes the proof.
.
Theorem 2.2 follows immediately from Propositions 5.1 and 5.2.
Finally, we
prove
Lemma 2.7:
iteration
k ,
criterion (ii)
If
Step 4
will be satisfied.
Proof:
Suppose
eT u / l
criterion (i)
Lemma 5.1, we have
< -1/20
in
is satisfied at
algorithm PCP
of
then in all subsequent iterations,
Step 4
that
of
criteria (ii)
iteration k .
or
51
of
It suffices to show
will not be satisfied in all subsequent iterations.
f (tk) = eT S-1 u < 0 .
(iii)
By Proposition 2.2,
By
f (t) < f (tk) <
criterion (i)
for all
t
tk .
Therefore, in all subsequent iterations,
will not be satisfied, as it would imply that
f (t)
0
for some
·
t > tk .
52
References
[1]
Barnes, E. R., S. Chopra, and D. L. Jensen (1988). "A polynomial time version
of the affine scaling algorithm." Working paper, Graduate School of
Business Administration, New York University, New York, NY.
[2]
Censor, Y. and A. Lent (1987). "Optimization of 'log x' entropy over linear
equality constraints." SIAM Journal of Control and Optimization 25,
921-933.
[3]
Freund, R. M. (1988). "Projective transformations for interior point methods,
Part I: Basic theory and linear programming." Working paper OR 17988, Operations Research Center, M.I.T., Cambridge, MA.
[4]
Freund, R. M. (1988). "Projective transformations for interior point methods,
Part II: Analysis of an algorithm for finding the weighted center of a
polyhedral system" Working paper OR 180-88, Operations Research
Center, M.I.T., Cambridge, MA.
[5]
Jarre, F. (1988). "On the
convex quadratic
Mathamatik und
Wiirzburg, West
[6]
Karmarkar, N. (1984). "A new polynomial time algorithm for linear
programming." Combinatorica 4, 373-395.
[7]
Kojima, M., S. Mizuno, and A. Yoshise (1987). "A primal-dual interior point
method for linear programming." Research report No. B-193, Dept. of
Information Sciences, Tokyo Institute of Technology, Japan.
[8]
Megiddo, N. (1986). "Pathways to the optimal set in linear programming."
Research report RJ 5295, IBM Almaden Research Center, San Jose, CA.
[9]
Mehotra, S. and J. Sun (1988). "A method of analytic centers for quadratically
constrained convex quadratic programs." Technical report 88-01, Dept.
of Industrial Engineering and Management Sciences, Northwestern
University, Evanston, IL.
convergence of the method of centers when applied to
programs." Manuscript, Institut fiiur Angewandte
Statistik, Universitit Wiirzburg, Am Hubland, 8700
Germany.
[10]
Monteiro, R. C. and I. Adler (1987). "An O(n 3 L) primal-dual interior point
algorithm for linear programming." Report ORC 87-4, Dept. of
Industrial Engineering and Operations Research, University of
California, Berkeley, CA.
[11]
Renegar, J. (1988). "A polynomial-time algorithm, based on Newton's
method, for linear programming." Mathematical Programming
40, 59-94.
[12]
Sonnevend, G. (1985). "An 'analytic' center for polyhedrons and new classes
of global algorithms for linear (smooth, convex) programming."
Preprint, Dept. of Numerical Analysis, Institute of Mathematics, E6tv6s
University, 1088, Budapest, Muzeum K6rut 6-8.
[13]
Sonnevend, G. (1985). "A new method for solving a set of linear (convex)
inequalities and its applications for identification and optimization."
Preprint, Dept. of Numerical Analysis, Institute of Mathematics, Edtv6s
University, 1088, Budapest, Muzeum K6rut 6-8.
[14]
Todd, M. J. and Y. Ye (1987). "A centered projective algorithm for linear
programming." Technical report No. 763, School of Operations
Research and Industrial Engineering, Cornell University, Ithaca, NY.
[15]
Vaidya, P. (1987). "A locally well-behaved potential function and a simple
Newton-type method for finding the center of a polytope." AT&T Bell
Laboratories, Murray Hill, NJ.
[16]
Vaidya, P. (1987). "An algorithm for linear programming which requires
O(((m+n)n 2 + (m+n) 1 .5 n) L) arithmetic operations." Proceeding of the
Nineteenth ACM Symposium on the Theory of Computing, 29-38.
b0
I