CS5234
Combinatorial and Graph Algorithms
Welcome!
CS5234 Overview
 Combinatorial & Graph Algorithms
http://www.comp.nus.edu.sg/~cs5234/2013/
 Instructor: Seth Gilbert
Office: COM2-323
Office hours: by appointment
Algorithms for
Combinatorial Optimization
Algorithms
What is an algorithm?
– Set of instructions for solving a problem
“First, wash the tomatoes.”
“Second, peel and cut the carrots.”
“Third, mix the olive oil and vinegar.”
“Finally, combine everything in a bowl.”
– Finite sequence of steps
– Unambiguous
– English, Chinese, pseudocode, Java, etc.
“If you need your software to run twice as fast,
hire better programmers.
But if you need your software to run more than
twice as fast, use a better algorithm.”
-- Software Lead at Microsoft
Goals: Algorithmic
1. Design
(problem solving)
2. Analysis
(rigorous, deep understanding)
3. Implementation
(able to put it to use)
Goals: Algorithmic
1. Design
(problem solving)
2. Analysis
3.
Implementation
(rigorous, deep understanding)
(able to put it to use)
Algorithms for
Combinatorial Optimization
Combinatorial Optimization
Optimization:
Find the minimum/maximum of…
Combinatorial Optimization
Optimization:
Find the minimum/maximum of:
Continuous optimization: a function f
Discrete optimization: a collection of items
Combinatorial Optimization
Discrete Optimization:
Find the “best” item in a large set of items.
Combinatorial Optimization:
Find the “best” item in a large set of items generated
by a combinatorial process.
Combinatorial: finite discrete structure
Combinatorial process: counting, combining,
enumerating
Combinatorial Optimization
Discrete Optimization:
Find the “best” item in a large set of items.
Combinatorial Optimization:
Find the “best” item in a large set of items generated
by a combinatorial process.
Graphs
Matroids
Similar structures…
Combinatorial Optimization
Find the “best” item in a large set of items:
Problem
Set of items
Size
Difficulty
Searching
List of integers
Linear
Easy
Shortest paths
All paths in a graph
Exponential
Easy
Minimum spanning tree
All spanning trees
Exponential
Easy
Steiner tree
All steiner trees
Exponential
Hard
Travelling salesman
All possible tours
Exponential
Hard
Matching
All possible matchings Exponential
Easy
Bipartite vertex cover
All possible covers
Exponential
Easy
Vertex cover
All possible covers
Exponential
Hard
Maximum clique
All possible subsets
Exponential
Very Hard
Combinatorial Optimization
Find the “best” item in a large set of items:
Problem
Difficulty
Maintain student records
Easy
Data compression
Easy
Program halting problem
Impossible
VLSI chip layout
Hard
Exam timetable scheduling
Hard
Job assignment problem
Easy
Computer deadlock problem
Easy
Finding patterns in a database
Easy
Combinatorial Optimization
Operations Research:
How to make better decisions (e.g., maximize profit)
Project planning / critical path analysis
Facility location: where to open stores / plants
Floorplanning: layout of factory or computer chips
Supply chain management
Berth assignment problem (BAP): port management
Assignment problems (e.g., weapon target assignment)
Routing / transportation problems: buses, subways, trucking.
Airline ticket pricing
Combinatorial Optimization
What do we do when problems are NP-hard?
1. Find exponential time solutions
2. Average performance
3. Approximate
– Algorithm is efficient
– Solution is sub-optimal
– Provable guarantee: ratio of output to optimal
Combinatorial Optimization
Five Representative Problems
1. Interval Scheduling
Jobs don’t overlap
Input: Set of jobs with start and finish times
Output: Maximum cardinality subset of compatible jobs
a
b
c
d
e
f
g
h
0
1
2
3
4
5
6
7
8
9
10
11
Time
2. Weighted Interval Scheduling
Input: Set of weighted jobs with start and finish times
Output: Maximum weight subset of compatible jobs
23
12
20
26
13
20
11
16
0
1
2
3
4
5
6
7
8
9
10
11
Time
3. Bipartite Matching
Input: Bipartite graph
Output: Maximum cardinality matching
A
1
B
2
C
3
D
4
E
5
4. Independent set
subset of nodes such that no two
joined by an edge
Input: Graph
Output: Maximum cardinality independent set
2
1
4
5
3
6
7
5. Competitive facility location
Input: Graph with weighted nodes
Game: Two players alternate in selecting nodes.
Cannot select a node if any of its neighbors
have been selected.
Goal: Select a maximum weight subset of nodes.
10
1
5
15
5
1
Second player can get 20, but not 25.
5
1
15
10
Five problems
Variations on a theme: Independent Set
Interval scheduling:
Weighted Interval scheduling:
Bipartite matching:
Independent set:
Competitive facility location:
Greedy O(n log n)
Dynamic programming O(n log n)
O(nk) max-flow algorithm
NP-complete
PSPACE-complete
Combinatorial Optimization
• General combinatorial problem
– Given a finite, discrete set S of objects
– Minimize/maximize some function f (S)
• Algorithmic Issues…
– Representation of the set S
– Efficient manipulation of the set S
– Efficient algorithm to compute f (S)
Strategy
• Given a problem P,
–
Computability
Can it be solved?
• If “Yes”, give an algorithm A for solving P,
–
Is algorithm A correct ?
–
How good is algorithm A ?
–
Can we find a better algorithm A’ ?
Verification
Efficiency
• How do we define good?
–
How much time it takes.
Time Complexity
–
How much space it uses.
Space Complexity
Algorithms for Combinatorial Optimization
Goals of this course:
– Advanced design and analysis of algorithms:
• Efficiency
– Time: How long does it take?
– Space: How much memory? How much disk?
– Others: Energy, power, heat, parallelism, etc.
• Scalability
– Inputs are large : e.g., the internet.
– Bigger problems consume more resources.
– Solve important (fun!) problems in combinatorial
optimization
Algorithms for Combinatorial Optimization
Target students:
– Beginning graduate students
– Advanced (4th year) undergraduates
– Anyone planning to do research in algorithmic
design
Prerequisites:
– CS3230 (Analysis of Algorithms), or equivalent
– Strong programming skills
Algorithms for Combinatorial Optimization
You must already know these:
• Data Structures (with analyses)
– Stacks, Queues, Lists,
– Binary search trees, balanced trees,
– Heaps and priority queues
• Algorithm Design Paradigms (with Analysis)
– Standard sorting and searching algorithms
– Graph algorithms: DFS, BFS,
– Shortest Path Algorithms, MST Algorithms
– Greedy Algorithms, Divide-and-Conquer
Algorithms for Combinatorial Optimization
You must already know these:
• Analysis of Algorithms
– Expertise with Big-O, ,  notations
– Summation of series, Master Theorem
– Competent with Algorithmic Analysis:
Quicksort, Heapsort, Divide-and-Conquer algorithms
DFS, BFS, Shortest Path & MST algorithms
Algorithms for Combinatorial Optimization
Assumed knowledge:
– Data structures and algorithms
– Good programming skills
– CS3230 (Analysis of Algorithms)
Not a course to learn algorithms:
– If not, take CS3230 instead.
CS5234 Overview
 Mid-term exam
October 8
In class
 Final exam
November 19
Reading Week
Exams will be graded and returned.
CS5234 Overview
 Grading
40% Problem sets
25% Mid-term exam
35% Final exam
 Problem sets
– 6-7 sets (about every 1-2 weeks)
– A few will have programming components (C++).
CS5234 Overview
 Problem sets released by tomorrow morning
PS0:
Covers background knowledge.
Do not submit.
PS1:
Routine problems --- easy practice.
Do not submit.
Standard problems --- to be submitted.
Advanced problems --- for a challenge/fun.
Do not submit.
CS5234 Overview
 Problem set grading
Distributed grading scheme:
Each of you will be responsible for grading
one week during the semester.
Grading supervised (and verified) by the TA.
CS5234 Overview
 What to submit:
Concise and precise answers:
Solutions should be rigorous, containing all
necessary detail, but no more.
Algorithm descriptions consist of:
1. Summary of results/claims.
2. Description of algorithm in English.
3. Pseudocode, if helpful.
4. Worked example of algorithm.
5. Diagram / picture.
6. Proof of correctness and performance
analysis.
CS5234 Overview
 Policy on plagiarism:
Do your work yourself:
Your submission should be unique, unlike
anything else submitted, on the web, etc.
Discuss with other students:
1. Discuss general approach and techniques.
2. Do not take notes.
3. Spend 30 minutes on facebook (or equiv.).
4. Write up solution on your own.
5. List all collaborators.
Do not search for solutions on the web:
Use web to learn techniques and to review
material from class.
CS5234 Overview
 Policy on plagiarism:
Penalized severely:
First offense: minimum of one letter grade lost on
final grade for class (or referral to SoC disciplinary
committee).
Second offense: F for the class and/or referral to
SoC.
Do not copy/compare solutions!
Textbooks
Introduction to Algorithms
– Cormen, Leiserson, Rivest, Stein
– Recommended…
Textbooks
Algorithm Design
– Kleinberg and Tardos
– Recommended…
CS5234 Overview
 Topics (tentative, TBD)
Introduction to combinatorial optimization
Vertex cover, set cover, Steiner tree, TSP
Flows and matching
Maximum flow, bipartite matching
Graph partitioning
Heuristics, spectral bisection
Linear programming
LPs, duality, relaxations, rounding