Graph Partitioning using Natural Cuts
Daniel Delling
Ilya Razenshteyn
Andrew Goldberg
Renato Werneck
Microsoft Research Silicon Valley
DIMACS 2012
1
Graph Partitioning
• Informally: split graph into loosely connected regions (cells).
2
Graph Partitioning
• Formal definition:
• Input: undirected graph G = (V , E )
• Output: partition of V into cells V1 , V2 , . . . , Vk
• Goal: minimize edges between cells
• Standard variant: enforce |Vi | ≤ U for fixed U:
• #cells may vary (≥ dn/Ue).
• Balanced variant: fix #cells k and imbalance :
• exactly k (maybe disconnected) cells, size ≤ (1 + )dn/Ue.
3
Natural Cuts
Road networks: dense regions (grids) interleaved with natural cuts
rivers, mountains, deserts, forests, parks, political borders, freeways, . . .
4
Natural Cuts
Road networks: dense regions (grids) interleaved with natural cuts
rivers, mountains, deserts, forests, parks, political borders, freeways, . . .
Partitioner Using Natural-Cut Heuristics
4
Natural Cuts
Road networks: dense regions (grids) interleaved with natural cuts
rivers, mountains, deserts, forests, parks, political borders, freeways, . . .
PUNCH: Partitioner Using Natural-Cut Heuristics
4
Algorithm Outline
1. Filtering phase:
• find natural cuts at appropriate scale
5
Algorithm Outline
1. Filtering phase:
• find natural cuts at appropriate scale
• keep cut edges, contract all others
5
Algorithm Outline
1. Filtering phase:
• find natural cuts at appropriate scale
• keep cut edges, contract all others
2. Assembly phase:
• partition (smaller) contracted graph
• greedy + local search [+ combinations]
5
Filtering: Finding Natural Cuts
• Must find sparse cuts between
dense regions:
• Sparsest cuts?
• Too expensive.
• Compute random s–t cuts?
• Mostly trivial: degrees are small.
• We need something else:
• s–t cuts between regions
6
Filtering: Finding Natural Cuts
1. Pick a center v .
v
7
Filtering: Finding Natural Cuts
1. Pick a center v .
2. Grow BFS of size U around v :
• First U/10 nodes: core
v
U/10 nodes
7
Filtering: Finding Natural Cuts
1. Pick a center v .
2. Grow BFS of size U around v :
• First U/10 nodes: core
• Unscanned neighbors: ring
v
U/10 nodes
U nodes
7
Filtering: Finding Natural Cuts
1. Pick a center v .
2. Grow BFS of size U around v :
• First U/10 nodes: core
• Unscanned neighbors: ring
3. Find minimum core/ring cut:
v
U/10 nodes
• standard s–t mincut.
U nodes
7
Filtering: Finding Natural Cuts
1. Pick a center v .
2. Grow BFS of size U around v :
• First U/10 nodes: core
• Unscanned neighbors: ring
3. Find minimum core/ring cut:
v
U/10 nodes
• standard s–t mincut.
4. Repeat for several “random” v :
• until each vertex in ≥ 2 cores
7
U nodes
Filtering: Finding Natural Cuts
1. Pick a center v .
2. Grow BFS of size U around v :
• First U/10 nodes: core
• Unscanned neighbors: ring
3. Find minimum core/ring cut:
v
U/10 nodes
• standard s–t mincut.
4. Repeat for several “random” v :
• until each vertex in ≥ 2 cores
Preprocess tiny cuts explicitly:
• identify 1-cuts and 2-cuts
7
U nodes
Filtering: Finding Natural Cuts
1. Pick a center v .
2. Grow BFS of size U around v :
• First U/10 nodes: core
• Unscanned neighbors: ring
3. Find minimum core/ring cut:
v
U/10 nodes
• standard s–t mincut.
4. Repeat for several “random” v :
• until each vertex in ≥ 2 cores
Preprocess tiny cuts explicitly:
• identify 1-cuts and 2-cuts
7
U nodes
Filtering: Finding Natural Cuts
1. Pick a center v .
2. Grow BFS of size U around v :
• First U/10 nodes: core
• Unscanned neighbors: ring
3. Find minimum core/ring cut:
v
U/10 nodes
• standard s–t mincut.
4. Repeat for several “random” v :
• until each vertex in ≥ 2 cores
Preprocess tiny cuts explicitly:
• identify 1-cuts and 2-cuts
7
U nodes
Filtering: Finding Natural Cuts
1. Pick a center v .
2. Grow BFS of size U around v :
• First U/10 nodes: core
• Unscanned neighbors: ring
3. Find minimum core/ring cut:
v
U/10 nodes
• standard s–t mincut.
4. Repeat for several “random” v :
• until each vertex in ≥ 2 cores
Preprocess tiny cuts explicitly:
• identify 1-cuts and 2-cuts
7
U nodes
Filtering: Finding Natural Cuts
1. Pick a center v .
2. Grow BFS of size U around v :
• First U/10 nodes: core
• Unscanned neighbors: ring
3. Find minimum core/ring cut:
v
U/10 nodes
• standard s–t mincut.
4. Repeat for several “random” v :
• until each vertex in ≥ 2 cores
Preprocess tiny cuts explicitly:
• identify 1-cuts and 2-cuts
• reduces road networks in half
• accelerates natural cut detection
7
U nodes
Properties of Filtering
1. many edges are never cut
2. cut edges partition graph into fragments
3. fragment size ≤ U (usually much less)
8
Properties of Filtering
1. many edges are never cut
2. cut edges partition graph into fragments
3. fragment size ≤ U (usually much less)
• Build fragment graph:
• fragment → weighted vertex
• adjacent fragments → weighted edge
8
Properties of Filtering
1. many edges are never cut
2. cut edges partition graph into fragments
3. fragment size ≤ U (usually much less)
• Build fragment graph:
• fragment → weighted vertex
• adjacent fragments → weighted edge
U
4 096
65 536
1 048 576
fragments
frag size
605 864
104 410
10 045
30
173
1 793
(Europe: 18M nodes)
8
Properties of Filtering
1. many edges are never cut
2. cut edges partition graph into fragments
3. fragment size ≤ U (usually much less)
• Build fragment graph:
• fragment → weighted vertex
• adjacent fragments → weighted edge
Assembly phase can operate on much smaller graph.
8
Assembly: Constructive
• Algorithm:
9
Assembly: Constructive
• Algorithm:
• start with isolated fragments;
9
Assembly: Constructive
• Algorithm:
• start with isolated fragments;
• combine adjacent cells;
9
Assembly: Constructive
• Algorithm:
• start with isolated fragments;
• combine adjacent cells;
9
Assembly: Constructive
• Algorithm:
• start with isolated fragments;
• combine adjacent cells;
9
Assembly: Constructive
• Algorithm:
• start with isolated fragments;
• combine adjacent cells;
9
Assembly: Constructive
• Algorithm:
• start with isolated fragments;
• combine adjacent cells;
9
Assembly: Constructive
• Algorithm:
• start with isolated fragments;
• combine adjacent cells;
• stop when maximal.
9
Assembly: Constructive
• Algorithm:
• start with isolated fragments;
• combine adjacent cells;
• stop when maximal.
• Randomized greedy:
• join fragments that are well-connected...
• ...relative to their sizes.
9
Assembly: Constructive
• Algorithm:
• start with isolated fragments;
• combine adjacent cells;
• stop when maximal.
• Randomized greedy:
• join fragments that are well-connected...
• ...relative to their sizes.
Reasonable solutions, but one can do better.
9
Assembly: Local Search
• For each pair of adjacent cells:
• disassemble into fragments;
• run constructive on subproblem;
• keep new solution if better.
10
Assembly: Local Search
• For each pair of adjacent cells:
• disassemble into fragments;
• run constructive on subproblem;
• keep new solution if better.
• Variant adds assembled neighbors:
• more flexibility;
• best results (default).
10
Assembly: Local Search
• For each pair of adjacent cells:
• disassemble into fragments;
• run constructive on subproblem;
• keep new solution if better.
• Variant adds assembled neighbors:
• more flexibility;
• best results (default).
• Could also disassemble neighbors:
• subproblems too large;
• worse results.
10
Assembly: Local Search
• For each pair of adjacent cells:
• disassemble into fragments;
• run constructive on subproblem;
• keep new solution if better.
• Variant adds assembled neighbors:
• more flexibility;
• best results (default).
• Could also disassemble neighbors:
• subproblems too large;
• worse results.
Evaluate each subproblem multiple times (use randomization).
10
15000
Assembly: Better Solutions
• Multiple tries for each pair
• local search is randomized
13000
12000
CUT SIZE
14000
●
●
●
●
●
●
11000
●
●
●
●
●
10000
Local search retries
0.1
1
10
100
1000
ASSEMBLY TIME (s)
(Europe, U = 216 )
11
10000
15000
Assembly: Better Solutions
• Multiple tries for each pair
• local search is randomized
14000
13000
CUT SIZE
12000
●
●
●
●
●
●
11000
• Multistart:
• constructive+local search;
• pick best of multiple runs.
●
●
●
●
●
10000
Local search retries
0.1
1
10
100
1000
ASSEMBLY TIME (s)
(Europe, U = 216 )
11
10000
15000
Assembly: Better Solutions
• Multiple tries for each pair
• local search is randomized
14000
13000
CUT SIZE
12000
●
●
●
●
●
●
11000
• Multistart:
• constructive+local search;
• pick best of multiple runs.
●
●
●
●
●
●
●
●
10000
Local search retries
Multistart
0.1
1
10
100
1000
ASSEMBLY TIME (s)
(Europe, U = 216 )
11
10000
15000
Assembly: Better Solutions
• Multiple tries for each pair
• local search is randomized
14000
13000
CUT SIZE
12000
●
●
●
●
●
11000
• Combination:
• combine some solutions;
• merge + local search.
●
●
●
●
●
●
●
●
Local search retries
Multistart
10000
• Multistart:
• constructive+local search;
• pick best of multiple runs.
●
0.1
1
10
100
1000
ASSEMBLY TIME (s)
(Europe, U = 216 )
11
10000
15000
Assembly: Better Solutions
• Multiple tries for each pair
• local search is randomized
14000
13000
CUT SIZE
12000
●
●
●
●
●
11000
• Combination:
• combine some solutions;
• merge + local search.
●
●
●
●
●
●
●
●
●
●
●
Local search retries
Multistart
Combination
10000
• Multistart:
• constructive+local search;
• pick best of multiple runs.
●
0.1
1
10
100
1000
ASSEMBLY TIME (s)
(Europe, U = 216 )
11
10000
15000
Assembly: Better Solutions
• Multiple tries for each pair
• local search is randomized
14000
13000
CUT SIZE
12000
●
●
●
●
●
11000
• Combination:
• combine some solutions;
• merge + local search.
●
●
●
●
●
●
●
●
●
●
●
Local search retries
Multistart
Combination
10000
• Multistart:
• constructive+local search;
• pick best of multiple runs.
●
0.1
1
10
100
1000
ASSEMBLY TIME (s)
(Europe, U = 216 )
More processing time → better solutions
11
10000
200
Running Times
●
●
160
180
Tiny cuts
Natural cuts
Greedy + Local search
Total
●
100
●
●
●
●
60
80
Time (s)
120
140
●
●
●
20
40
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
18
20
●
0
●
10
2
12
2
14
2
16
2
2
Maximum cell size (U)
Europe (18M vertices), 12 cores
12
2
222
200
Running Times
●
●
160
180
Tiny cuts
Natural cuts
Greedy + Local search
Total
●
100
●
●
●
●
60
80
Time (s)
120
140
●
●
●
20
40
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
18
20
●
0
●
10
2
12
2
14
2
16
2
2
2
222
Maximum cell size (U)
Europe (18M vertices), 12 cores
Bottlenecks: assembly for small U, filtering for large U
12
Solution Quality
U
A
B
1 024
4 096
16 384
65 536
262 144
1 048 576
4 194 304
895
3 602
14 437
57 376
222 626
826 166
3 105 245
16.8
27.6
45.6
72.7
103.7
134.3
127.9
√
B/ U
√
B/ 3 U
0.52
0.43
0.36
0.28
0.20
0.13
0.06
1.66
1.73
1.80
1.80
1.62
1.32
0.79
(Europe, 16 retries, no multistart/combination)
U: maximum cell size allowed
A: average cell size in PUNCH solution
B: average boundary edges per cell
13
Solution Quality
U
A
B
1 024
4 096
16 384
65 536
262 144
1 048 576
4 194 304
895
3 602
14 437
57 376
222 626
826 166
3 105 245
16.8
27.6
45.6
72.7
103.7
134.3
127.9
√
B/ U
√
B/ 3 U
0.52
0.43
0.36
0.28
0.20
0.13
0.06
1.66
1.73
1.80
1.80
1.62
1.32
0.79
(Europe, 16 retries, no multistart/combination)
U: maximum cell size allowed
A: average cell size in PUNCH solution
B: average boundary edges per cell
Road networks have very small separators!
13
Experimental Comparison
Existing packages:
• METIS [KK99]
• SCOTCH [PR96]
• Kappa [HSS10], KaSPar [OS10], Kaffpa [SS11], KaffpaE [SS12]
They work on the balanced variant:
• find k cells with size ≤ (1 + )dn/Ue.
PUNCH can find balanced partitions:
1. run standard PUNCH
with U = (1 + )dn/Ue;
14
Experimental Comparison
Existing packages:
• METIS [KK99]
• SCOTCH [PR96]
• Kappa [HSS10], KaSPar [OS10], Kaffpa [SS11], KaffpaE [SS12]
They work on the balanced variant:
• find k cells with size ≤ (1 + )dn/Ue.
PUNCH can find balanced partitions:
1. run standard PUNCH
with U = (1 + )dn/Ue;
14
Experimental Comparison
Existing packages:
• METIS [KK99]
• SCOTCH [PR96]
• Kappa [HSS10], KaSPar [OS10], Kaffpa [SS11], KaffpaE [SS12]
They work on the balanced variant:
• find k cells with size ≤ (1 + )dn/Ue.
PUNCH can find balanced partitions:
1. run standard PUNCH
with U = (1 + )dn/Ue;
2. pick k base cells,
reassign the rest (randomized multistart)
14
Experimental Comparison
Existing packages:
• METIS [KK99]
• SCOTCH [PR96]
• Kappa [HSS10], KaSPar [OS10], Kaffpa [SS11], KaffpaE [SS12]
They work on the balanced variant:
• find k cells with size ≤ (1 + )dn/Ue.
PUNCH can find balanced partitions:
1. run standard PUNCH
with U = (1 + )dn/Ue;
2. pick k base cells,
reassign the rest (randomized multistart)
14
Experimental Comparison
Existing packages:
• METIS [KK99]
• SCOTCH [PR96]
• Kappa [HSS10], KaSPar [OS10], Kaffpa [SS11], KaffpaE [SS12]
They work on the balanced variant:
• find k cells with size ≤ (1 + )dn/Ue.
PUNCH can find balanced partitions:
1. run standard PUNCH
with U = (1 + )dn/Ue;
2. pick k base cells,
reassign the rest (randomized multistart)
14
Experimental Comparison
Existing packages:
• METIS [KK99]
• SCOTCH [PR96]
• Kappa [HSS10], KaSPar [OS10], Kaffpa [SS11], KaffpaE [SS12]
They work on the balanced variant:
• find k cells with size ≤ (1 + )dn/Ue.
PUNCH can find balanced partitions:
1. run standard PUNCH
with U = (1 + )dn/Ue;
2. pick k base cells,
reassign the rest (randomized multistart)
14
Experimental Comparison
Existing packages:
• METIS [KK99]
• SCOTCH [PR96]
• Kappa [HSS10], KaSPar [OS10], Kaffpa [SS11], KaffpaE [SS12]
They work on the balanced variant:
• find k cells with size ≤ (1 + )dn/Ue.
PUNCH can find balanced partitions:
1. run standard PUNCH
with U = (1 + )dn/Ue;
2. pick k base cells,
reassign the rest (randomized multistart)
14
Experimental Comparison
Existing packages:
• METIS [KK99]
• SCOTCH [PR96]
• Kappa [HSS10], KaSPar [OS10], Kaffpa [SS11], KaffpaE [SS12]
They work on the balanced variant:
• find k cells with size ≤ (1 + )dn/Ue.
PUNCH can find balanced partitions:
1. run standard PUNCH
with U = (1 + )dn/Ue;
2. pick k base cells,
reassign the rest (randomized multistart)
14
Balanced Partitions
PUNCH finds better solutions...
2.0
1.5
1.0
0.0
0.5
Relative cut size (PUNCH=1)
2.5
3.0
Solution Quality
PUNCH
KAFFPA
KASPAR
KAPPA
SCOTCH
METIS
K=4
K=16
(Europe, = 0.03)
15
K=64
Balanced Partitions
PUNCH finds better solutions...
...in reasonable time.
3000
Running Times
2000
1000
1500
Running Time [s]
2.0
1.5
500
1.0
0
0.0
K=4
K=16
(Europe, = 0.03)
15
PUNCH
KAFFPA
KASPAR
KAPPA
SCOTCH
METIS
2500
PUNCH
KAFFPA
KASPAR
KAPPA
SCOTCH
METIS
0.5
Relative cut size (PUNCH=1)
2.5
3.0
Solution Quality
K=64
K=4
K=16
(Europe, = 0.03)
K=64
Balanced Partitions
PUNCH finds better solutions...
...in reasonable time.
Running Times
PUNCH
KAFFPA
KASPAR
KAPPA
SCOTCH
METIS
128
32
1.5
Running Time [s]
2.0
512
2048
PUNCH
KAFFPA
KASPAR
KAPPA
SCOTCH
METIS
0.0
2
0.5
8
1.0
Relative cut size (PUNCH=1)
2.5
3.0
Solution Quality
K=4
K=16
(Europe, = 0.03)
15
K=64
K=4
K=16
(Europe, = 0.03)
K=64
Vancouver by METIS
16
Vancouver by PUNCH
17
Portland by METIS
18
Portland by PUNCH
19
DIMACS Instances
Setup:
• = 0.03
• 9 runs
• default PUNCH
instance
luxembourg
belgium
netherlands
italy
great-britain
germany
asia
europe
20
2
16
72
40
36
84
113
7
140
5
46
167
81
91
225
283
20
312
median solution
8
16
82
148
316
565
191
380
201
349
393
638
509
881
48
112
523
955
32
245
923
679
690
1175
1512
249
1536
64
377
1436
1210
1187
1846
2332
470
2576
DIMACS Instances
Setup:
• = 0.03
• 9 runs
• default PUNCH
instance
luxembourg
belgium
netherlands
italy
great-britain
germany
asia
europe
20
2
1.2
16.0
28.1
97.8
60.4
128.6
67.5
1051.0
5
2.4
19.9
17.1
78.6
60.6
125.8
76.6
814.0
average time [s]
8
16
2.4
1.9
20.8
20.4
15.2
15.0
65.0
51.9
57.7
50.8
104.7
91.5
60.1
50.9
627.4
512.8
32
1.5
15.7
12.1
41.7
43.6
74.3
46.1
427.7
64
2.2
18.3
16.9
40.0
47.6
76.4
43.7
375.0
DIMACS Instances
Setup:
• = 0.03
• 9 runs
• strong PUNCH
instance
luxembourg
belgium
netherlands
italy
great-britain
germany
asia
europe
20
2
16
71
40
36
83
111
7
139
5
46
163
81
90
220
279
20
311
median solution
8
16
80
142
313
548
191
369
200
339
381
636
503
852
48
111
522
923
32
238
900
662
673
1140
1488
242
1517
64
377
1421
1199
1175
1821
2317
462
2538
DIMACS Instances
Setup:
• = 0.03
• 9 runs
• strong PUNCH
instance
luxembourg
belgium
netherlands
italy
great-britain
germany
asia
europe
20
2
5
7.2
16.4
51.2
99.9
132.2
57.3
157.2 173.8
103.6 165.5
195.6 347.7
83.4 200.0
2217.9 1451.8
average
8
18.1
113.6
52.8
174.3
189.8
291.8
95.3
939.8
time [s]
16
32
13.7
11.1
115.0
94.9
59.2
50.1
135.1
110.2
167.0
135.3
253.9
214.1
73.7
66.4
732.5
604.0
64
8.6
58.5
48.4
80.7
108.5
153.0
58.4
494.6
DIMACS Instances
Setup:
• = 0.03
• 9 runs
• strong PUNCH
instance
luxembourg
belgium
netherlands
italy
great-britain
germany
asia
europe
20
2
16
70
40
36
82
108
7
138
5
46
161
81
89
213
276
20
311
best solution
8
16
79
139
308
532
191
360
198
338
377
633
485
845
47
110
515
905
32
235
880
652
665
1118
1475
238
1488
64
369
1401
1186
1166
1796
2282
452
2509
Final Thoughts
• PUNCH can be used to find multilevel partitions
top-down works best
• How to improve balancing?
• Can it be made faster?
though fast enough for our purposes
• How far is it from optimal?
• Does it work well on other graph classes?
• Crucial ingredient for Bing Maps driving directions engine
21
Thank you!
22