I. Integer programming part of Clarkson-paper
II. Incremental Linear Programming, Section 9.10.1 in
Randomized Algorithms-book
presented by Jan De Mot
September 29, 2003
This presentation is based on: Clarkson, Kenneth L. Las Vegas Algorithms for Linear and Integer Programming When the Dimension
is Small. Journal of the ACM 42(2), March 1995, pp. 488-499. Preliminary version in Proceedings of the 29th Annual IEEE
Symposium on Foundations of Computer Science, 1988.
and Chapter 9 of: Motwani, Rajeev, and Prabhakar Raghavan. Randomized Algorithms. Cambridge, UK: Cambridge University
Press, 1995.
1/23
Outline
Part I: Integer Linear Programming (ILP)
• Previous work
• Algorithm for solving Integer Linear Programs [Clarkson 1995]
based on the mixed algorithm for LP (Susan)
– Concept
– Running Time Analysis
Part II: Incremental Linear Programming
• Concept
• SeideLP [Seidel 1991]
• BasisLP [Sharir and Welzl 1992]
2/23
Part I: Integer Linear Programming
3/23
Previous Work
•
•
•
[Lenstra 1983] showed how to solve an ILP in polynomial time
when the numbers of variables is fixed.
Subsequent improvements (e.g. by [Frank and Tardos 1987])
show that the fasted deterministic algorithm requires
operations on
-bit numbers.
Running time of new ILP algorithm:
This is substantially faster than Lenstra’s for
4/23
ILP Problem
•
Find the optimum of:
where
and
5/23
Notation and Preliminaries
•
Let:
–
–
•
denote the set of constraints defined by
and
denote the optimal solution of the ILP defined on
(not the corresponding LP relaxation).
Assume:
– Bounded solution by adding to
a new set of
constraints
:
where
and where we use a result by
[Schrijver 1986]: if an ILP has finite solution, then every coordinate
of that optimum has size no more than
where
is the facet
complexity of
– Unique solution by choosing the lexicographically largest point
achieving the optimum value.
6/23
ILP Algorithm: Concept
•
First it is established that an optimum is determined by a small
set ([Bell 1977] and [Scarf 1977]):
Lemma: There is a set
with
and with
•
ILP algorithms are variations on the LP algorithms, with sample
sizes using
rather than and using Lenstra’s algorithm in
the base case.
Here, we convert the mixed algorithm for LPs to a mixed
algorithm for ILPs, establishing the right sample sizes and
criteria for successful iterations in both the recursive and
iterative part of the mixed algorithm.
•
7/23
ILP Algorithm: Details
•
•
•
•
Lemma 2, related to the LP recursive algorithm, needs to be
redone due to the fact that
is not unique.
Reminder: why do we need lemma 2?
We want to make sure the set of violated constraints does not
become too big.
Lemma 2 (ILP version): Let
and let
be a
random subset of size
with
Let
be the set of constraints violated by
Then with
probability
Other necessary lemma’s remain valid or can be adapted easily,
yielding the following essential parameters for the ILP mixed
algorithm:
– Recursive part:
and require
– Iterative part:
of
use Lenstra’s algorithm for
for a successful iteration.
with a corresponding
bound
8/23
ILP Algorithm: Proof of Lemma 2 (ILP version)
•
Proof. Lemma 2 (ILP version): With probability
•
Assume is empty. For not empty: similar proof.
Let
and let
denote the
number of constraints in
violated by
We know that
for some
with
We want to find
such that the probability that
less then
This probability is bounded above by:
is
which is no more than:
9/23
ILP Algorithm: Proof of Lemma 2 (cont’d)
which is again no more than:
and using elementary bounds, this quantity is less than
for
10/23
ILP Algorithm: Running Time
•
We have the following theorem:
The ILP algorithm requires expected
row operations on
-bit vectors, and
expected operations on
-bit numbers, as
the constant factors do not depend on or
where
11/23
Part II: Incremental Linear Programming
12/23
Incremental LP
•
•
Randomized incremental algorithms for LP
Concept:
– add constraints in random order,
– after adding each constraint, determine the optimum of the
constraints added so far.
•
Two algorithms will be discussed:
– SeideLP
– BasisLP
13/23
Algorithm SeideLP
Input: A set of constraints
Output: The optimum of the LP defined by
0. if
output
1. Pick a random constraint
Recursively find
2.1. if
does not violate output
to be
the optimum
2.2. else project all the constraints of
onto
and
recursively solve this new linear programming problem;
14/23
SeideLP: Running Time
•
•
Let
denote an upper bound on the expected running
time for a problem with constraints in dimensions.
Then:
– First term: cost of recursively solving the LP defined by the
constraints
– Second term: checking whether violates
– Third term (with probability
): cost of projecting + recursively
solving smaller LP.
•
Theorem: There is a constant
satisfies the solution
such that the recurrence
15/23
SeideLP: Further Discussion
•
In Step 2.2. we completely
discard any information obtained
from the solution of the LP
•
From the above figure, it follows we must consider all
constraints in
But: Can we use
to “jump-start” the recursive call in
step 2.2.?
RESULT: Algorithm BasisLP
•
•
16/23
Algorithm BasisLP
Input:
Output: A basis
for
0. If
output
1. Pick a random constraint
BasisLP(
);
2.1. if does not violate
output
2.2. else output BasisLP( Basis(
Basis returns a basis for a set of
));
or fewer constraints.
17/23
BasisLP: Why does it work?
•
•
Each invocation of Basis occurs when the violation test in 2.1.
fails (i.e. does violate ).
What is the probability that we fail a violation test?
–
–
–
–
Let
Remember:
Pr( violates the optimum of
)
This probability decreases further if
contains some of the
constraints of
– This was indeed the motivation for modifying SeideLP to BasisLP.
18/23
BasisLP: Running Time
•
Notation:
– Given
enforcing in
– Let
denote
enforcing in
•
•
•
•
, we call
if
minus the number of constraints that are
is called the hidden dimension of
Lemma 1: If is enforcing in
then (i)
and (ii)
is extreme in all
such that
So, the probability that a violation occurs
can be bounded by
We establish that the
decreases by at least 1 at each
recursive call in step 2.2. It turns out
is likely to decrease
much faster.
Theorem: The expected running time of BasisLP is
19/23
BasisLP: Analysis Details
•
Proof of Lemma 1. If
– (i)
We have
subset of
– (ii) is extreme in all
Assume the contrary:
is enforcing in
then
which can not be true if
were a
such that
a
contradiction.
20/23
BasisLP: Analysis Details (Cont’d)
•
Lemma 2: Let
extreme constraint in
Then:
Let
and let
be a basis of
(i) Any constraint
that is enforcing in
(ii) is enforcing in
(iii)
be an
is also enforcing in
Proof:
– (i)
then:
– (ii) Since is extreme in
– (iii) Follows readily.
•
So, the numerator of
each execution.
decreases by at least 1 at
21/23
BasisLP: Analysis Details (Cont’d)
•
•
•
•
•
Show that this decrease is likely to be faster.
Given
and a random
we bound the
probability that violates
If it does, check the
probability distribution of the resulting hidden dimension.
Lemma 3: Let
be the extreme constraints of
that are not in
numbered so that
Then, for all and for
is enforcing in
Basis
(proof: immediate from lemma 2.)
In other words: when
then all of
will be
enforcing and the arguments of the recursive call will have
hidden dimension
Observation: since any
is equally likely to be
is uniformly
distributed on the integers in
and the resulting hidden
dimension is uniformly distributed on the integers in
22/23
BasisLP: Analysis Details (Cont’d)
•
•
•
Let
denote the maximum expected number of violation
tests for a call to BasisLP with arguments
where
and
We get:
This yields:
and consequently the expected running time of BasisLP is
Augmenting the analysis with Clarkson’s sampling technique
improves the running time of the mixed algorithm to
23/23