ADVANCED COMPUTATIONAL MODELS
AND ALGORITHMS
Instructor: Dr. Gautam Das
March 10, 2009
Class notes by Alexandra Stefan
Topics covered
• Linear Programming (LP)
– Problem instance:
• Set of n real variables
• Set of restrictions in the form of linear inequalities
• Goal/optimizing function
• Integer Programming (IP)
• Set of n integer variables
• Set of restrictions in the form of linear inequalities
• Goal/optimizing function
Linear programming
• Example:
– Find real values x,y, s.t.
– Constraints:
(1) x+y <= 5
(2) x + 2y <= 6
(3) x >= 0
(4) y >= 0.2
– Goal:
(G) 4x + y is maximized
– Solution for this problem: (x,y) = (5, 0)
Geometric interpretation of the linear problem
y
(3) = 0
(solution plane)
5
4
3
G
(1) = 0
Feasible region
2
(4) = 0
Solution point
1
0.2
x
1
(G) = 0
2
3
(G) = 4
4
5
6
(2) = 0
• Feasible region is convex
– Convex region: given any two points from the
region, the line segment between them is part of
the region
– Proof idea: the feasible region is the intersection
of half-planes which are convex regions.
• Each constraint generates at most one edge
in the feasible region
• Intuition on finding the solution:
– as you move the goal line, G, to increase it’s
value, the point that will maximize it is one of the
‘corners’.
Naive algorithm – special case (n =2)
• Problem definition:
– 2 variables and m constraints
• Solution:
– For all pairs of constraints ci,cj
• Find the intersection point: pij (pij satisfies both ci
and cj)
• Check whether pij is part of the feasible region
If yes, apply goal function. Keep track of which point
gave maximum value for the goal function.
• Complexity: O(m3)
– O(m2) pairs (ci, cj). For each pair perform
O(m) evaluations to see if the point pij
satisfies all the m constraints
Naive algorithm – general case
• General problem definition:
– n variables and m constraints,
• Solution
– A corner is now defined by intersecting n
hyper-planes (that is satisfying n constraints)
– This gives a total of m choose n corners
=> complexity O(mn) (exponential in n)
Methods that solve LP
• Simplex (by George B. Dantzig, 1947)
– Finds optimum: (because feasible region is convex)
– Runtime: average time is linear; worst case time is
exponential (covers all feasible corners)
• Ellipsoidal method (by Leonid Khachiyan, 1839)
– Runtime: polynomial
– Theoretical value; not useful in practice
• Interior point method (by Narendra Karmarkar, 1984)
– Runtime : polynomial
– Practical value
Simplex
• Start from a corner
– Note that note all corners are part of the
feasible region. The algorithm will find a
feasible one.
– Look at the neighboring corners
• Intuition of how you find the neighboring corners:
remove one constraint and add another
– Move to the one that gives the highest value
to the goal function
• STOPPING condition
– : none of the neighboring points gives a higher value
than the current one.
Integer Programming (IP)
• Integer Programming (IP)
• Set of n integer variables
• Set of restrictions in the form of linear inequalities
• Goal/optimizing function
• IP is NP-Complete
– Decision problem:
• Input: IP and a target value C
• Output: is there a point in the region that evaluates
the goal function to a value greater than C?
• Easy to verify it is NP
– NP-Complete
• reduce Vertex Cover to IP