Generalized Linear
Programming
Jiří Matoušek
Charles University, Prague
The cool slides in this presentation are included by
the courtesy of Tibor Szabó.
Linear Programming
• Minimize cx subject to Ax  b.
• Geometry: Minimize a linear function over
the intersection of n halfspaces in Rd
(=convex polyhedron).
LP Algorithms
• Simplex method [Dantzig 1947]
– very fast in practice
– very good “average case”
– exponential-time examples for almost all pivot
rules
• Ellipsoid method [Khachyian], interior-point
methods [Karmakar],…
– weakly polynomial but no (worst-case) bound
in terms of n and d alone
Combinatorial LP algorithms
• wanted: time  f(d,n) for all inputs
• computations “coordinate independent”;
use only combinatorial structure of the
feasible set (polyhedron) or of the
arrangement of bounding hyperplanes
Combinatorial LP algorithms
Computational geometry: research started
with d fixed (and small)
– [Megiddo] exp(exp(d)).n
– [Clarkson] randomization; d2n+dd/2 log n
– [Seidel] simple randomized; d! n
– [Chazelle, M.] exp(O(d)).n deterministic
– parallel [Alon, Megiddo] [Ajtai, Megiddo]
A subexponential algorithm
Theory of convex polytopes
(Hirsch conjecture):
[Kalai] 1992
Computational geometry:
[Sharir, Welzl],
[M., Sharir, Welzl] 1992
exp((d log d)).n (randomized expected)
– known as RANDOM FACET :
In the current vertex of the feasible polytope, choose
a random improving facet, recursively find its
optimum, and repeat
– still the best known running time!
Abstract frameworks
• systems of axioms capturing some of the
properties of linear programming
• running time of algorithms counted in terms of
certain primitive operations
• to apply to a specific problem, need to
implement them …
• … and then algorithms become available (such
as Kalai/MSW, Clarkson)
Abstract frameworks
Abstract objective functions [Adler, Saigal
1976], [Wiliamson Hoke 1988], [Kalai
1988]
– P a (convex) polytope
– f : V(P) → R is an abstract objective function
if a local minimum of any face F is also the
unique global minimum of F
– every generic linear function induces an
AOF
– but there are nonrealizable AOF on the 3dimensional cube!
Abstract frameworks
Acyclic Unique Sink Orientations (AUSO)
– acyclic orientation of the graph of the
considered polytope such that every
nonempty face has exactly one sink (sink = all
edges incoming)
– same as abstract objective functions
Abstract frameworks
LP-type problems [Sharir, Welzl]
– also called Generalized Linear Programs
[Amenta]
– encompass many geometric optimization
problems [MSW,Amenta,Halman…]
• smallest enclosing ball of n points in Rd
• smallest enclosing ellipsoid of n points in Rd
• distance of two (convex) polyhedra in Rd
………
•
– plus some non-geometric (games on graphs)
LP-type problems
• H a finite set of constraints
• (W,) a linearly ordered set (such as the reals)
• w: 2H  W a value function; intuitively: w(G) is
the minimum value of a solution attainable under
the constraints in G
• Axiom M (monotonicity):
If F  G, then w(F)  w(G).
• Axiom L (locality):
If F  G and w(F) = w(G) =w(F{h}), then
w(G)=w(G{h}).
Example: Smallest enclosing ball
• H a finite set of points in the plane
• w(G) = radius of the smallest disk containing G
e
monotonicity trivial
a
d
b
c
locality depends on
uniqeness of the smallest
enclosing ball!
LP-type problems: more notions
• basis for G: inclusion-minimal B  G with
w(B)=w(G)
• dimension d of (H,w): maximum cardinality
of a basis
• computational primitives (B a given basis)
– violation test: value(B{h})>value(B)?
– pivoting: compute a basis for B{h}
Abstract frameworks
Abstract Optimization Problems [Gärtner]
– only one parameter: dimension d=|H| (no n)
– a linear ordering of 2H
– primitive operation: Is G optimal among all
sets containing F? If not, give a better G’
– nice randomized algorithm: exp(O(d))
[Gärtner]
– allows a (rather) efficient implementation of
“primitives” in Kalai/MSW, e.g., for the
smallest enclosing ball problem
Algorithms in the abstract
frameworks
• several algorithms (Kalai/MSW = RANDOM
FACET; Clarkson) work for AOF’s, same
analysis
– AUSO given by oracle: returns edge orientations for a
given vertex
– yields n.exp(O(d)) randomized algorithm
– analysis tight in this abstract setting [M.]
• for LP-type problems they work too (but…)
– O(n) algorithms for fixed d usually immediate
– but primitives “depend on d” … may be hard
– sometimes Gärtner’s algorithm helps
Algorithms in the abstract
frameworks
RANDOM EDGE
• the simplex algorithm that selects an
improving edge uniformly at random
• for AUSO: random outgoing edge
• great expectations: perhaps always
quadratic??? [Williamson Hoke 1988]
RANDOM EDGE
Expected running time
– on the d-dimensional simplex: (log d)
[Liebling]
– on d-dimensional polytopes with d+2 facets:
(log2d) [Gärtner et al. 2001]
– on the d-dimensional Klee-Minty cube:
• O(d2) Williamson Hoke (1988)
• (d2/log d) Gärtner, Henk, Ziegler (1995)
• (d2) Balogh, Pemantle (2004)
RANDOM EDGE can be
(mildly) exponential
There exists an AUSO of the d-dimensional cube such
that RANDOM EDGE, started at a random vertex,
makes at least exp(c.d1/3) steps before reaching the
sink, with probability at least 1- exp(-c.d1/3).
[M., Szabó, FOCS 2004]
The Klee-Minty cube
reversed
KMm-1
KMm
KMm-1
A blowup construction
Hypersink reorientation
A simpler construction
Let A be a d-dimensional cube on which
RANDOM EDGE is slow (constructed
recursively)
– take the blowup of A with random KMm‘s
whose sink is in the same copy of A, m=d
– reorient the hypersink by placing a random
copy of A
– thus, a step from d to d+d
A simpler construction
A
A
A
rand A
A typical RANDOM EDGE move
v
• Move in the frame:
– RANDOM EDGE move in KMm
– stay put in A
A
• Move within a hypervertex:
– RANDOM EDGE move in A
– move to a random vertex of
KMm on the same level
A
A
rand A
Random walk with reshuffles on KMm
RANDOM EDGE on A
Walk with reshuffles on KMm
• Start at a random v(0) of KMm
• v(i) is chosen as follows:
– with probability pi,step make a step of
RANDOM EDGE from v(i-1);
– with probability pi,resh randomly permute
(reshuffle) the coordinates of v(i-1) to obtain v(i)
– with probability 1- pi,step - pi,resh, v(i) = v(i-1).
Walk with reshuffles on KMm is slow
Proposition. Suppose that
min pi ,resh  11 max pi , step
m
Then with probability at least 1  e
the random walk with reshuffles makes
m
at least e steps (α and β are constants).
Reaching the hypersink
•
•
Either we reach the sink by reaching the sink of a
copy of A and then perform RANDOM EDGE on
KMm. This takes at least T(d) time.
Or we reach the hypersink without entering the
sink of any copy of A. That is, the random walk
with reshuffles reaches the sink of KMm . This
takes at least exp(m)  T(d) time.
The recursion
•
•
•
RANDOM EDGE arrives to the hypersink at a
random vertex. Then it needs T(d) more steps.
So passing from dimension d to d+d the
expected running time of RANDOM EDGE
doubles.
Iterating d - times gives T(2d)  2d T(d).
In order to guarantee that reshuffles are frequent
enough we need a more complicated
construction and that is why we are only able to
prove a running time of exp(c.d1/3).
Open questions
• Obtain any reasonable upper bound on
the running time of RANDOM EDGE
• Can one modify the construction such that
the cube is realizable? (Probably not …)
• Or at least it satisfies the Holt-Klee
condition?
• Or at least each three-dimensional
subcube satisfies the Holt-Klee condition?
More open questions
• Find an algorithm for AOF on the d-cube
better than exp(d)
• The model of unique sink orientations of
cubes (possibly with cycles) include LP on
an arbitrary polytope.
Find a subexponential algorithm!
THE END