The Traveling Salesman Problem
in Theory & Practice
Lecture 5: Tour Construction Heuristics
18 February 2014
David S. Johnson
dstiflerj@gmail.com
http://davidsjohnson.net
Seeley Mudd 523, Tuesdays and Fridays
Outline
Tour Construction Heuristics
• Definitions and Worst-Case Results
• NN, Greedy, Savings, Insertion Algorithms, etc.
• Performance in Practice (Tour quality)
Nearest Neighbor (NN):
1.
Start with some city.
2. Repeatedly go next to the nearest unvisited neighbor of
the last city added.
3. When all cities have been added, go from the last back
to the first.
Theorem
[Rosenkrantz, Stearns, & Lewis, “An analysis of several heuristics
for the traveling salesman problem,” SIAM J. Comput. 6 (1977), 563-581]:
1.
There is a constant c, such that, for any instance I obeying
the ∆-inequality, NN(I) ≤ clog(N)Opt(I).
[Proved last week]
2. There exists a constant c’, such that for all N > 3, there are
N-city instances I obeying the triangle inequality for which
we have NN(I) > c’log(N)Opt(I).
[To be proved today]
Definition: For any algorithm A and instance size N, the worstcase ratio for A is
RN(A) = max{A(I)/OPT(I): I is an N-city instance}
Corollary: RN(NN) = Θ(log(N)).
Lower Bound Examples
(Unspecified distances determined by shortest paths)
Number N1 of vertices = 3
1
F1:
OPT(F1) = 3+ε
1+ε
NN-path(F1) = 2
1
NN path starts at left, ends in middle
2
F2:
2
1
1+ε
1
1
1+ε
1
1+ε
1
1+ε
N2 = 2N1 + 3 = 9
OPT(F1) = N2+5ε = 9+5ε
NN-path(F2) = 10
1+ε
1
Fk-1 without shortcut between left and right endpoints
2k-1
Fk:
1+ε
2k-1
1
1+ε
1
1+ε
Number Nk of vertices = 2*Nk-1 + 3
OPT(Fk) = Nk + ½(Nk+ 1)ε (for k ≥ 2)
NN-path(Fk) = 2NN-path(Fk-1)+2k+2
General Formula for Nk
Nk = 3(2k-1)
Proof by Induction:
Initialization: N2 = 9 = 3(22-1)
Assume true for k’ < k. Then
Nk = 2Nk-1 + 3 = 2(3(2k-1-1)) + 3
= 32k – 6 +3 = 3(2k-1)
Note for future reference: Nk is always odd.
General Formula for Opt(Fk) when k ≥ 2
Opt(Fk) = Nk + ½(Nk+ 1)ε
Proof: All edge lengths must be 1 or 1+ε, which means they
must lie along the base of Fk, where the edges start with 1+ε
and then alternate, and since Nk is odd, this means that means
we will get one more 1+ε than 1.
Note that this means that, as ε → 0 and k → ∞, OPT(Fk) → Nk
= 3(2k-1), and hence log(OPT(Fk)) → k+1 + log(3/2), and hence
for sufficiently large k we have
k+1 > ½log(Opt(Fk))
General Formula for NN-path(Fk)
NN-path(Fk) = (k+1)2k – 2
Proof by Induction:
Initialization: NN-path(F2) = 10 = 322 - 2
Assume true for k’ < k. Then
NN-path(Fk) = 2NN-path(Fk-1) + 2k + 2
= 2(k2k-1 – 2) + 2k + 2 = (k+1)2k – 2.
Corollary: as ε → 0 and k → ∞,
NN-path(Fk)/OPT(Fk) → (k+1)2k/(32k) > log(OPT(Fk))/6
Fk-1 without shortcut between left and right endpoints
2k-1
Fk:
1+ε
2k-1
1
1+ε
1
1+ε
Number Nk of vertices = 2*Nk-1 + 3
OPT(Fk) = Nk + ½(Nk+ 1)ε (for k ≥ 2)
NN-path(Fk) = 2NN-path(Fk-1)+2k+2
But are these two vertices really nearest neighbors?
2k-1
Fk
2k-1
Fk-1
1+ε
1+ε
1
2k-2
Fk-1
1
2k-2
B
1+ε
1
1+ε
C
1
Is d(B,C) + 1+ε > 2k-1
Results for General k
2k-1
Fk
2k-1
B
A
1+ε
Define: D1(Fk) = d(A,C),
1
C
1+ε
1
D2(Fk) = d(B,C) = d(A,B)
We shall prove by induction that D1(Fk) > 2k+1 - 3 and D2(Fk) > 2k - 1
Note: A shortest path from A to B need never go leftwards
(by the triangle inequality)
Assuming Induction Hypothesis (for k’ < k):
D1(Fk’) > 2k’+1 - 3
D2(Fk’) > 2k’ – 1
There are two choices for the shortest A-C path in Fk
2k-1
2k-1
B
A
1+ε
1+ε
1
2k-1
A
1+ε
C
1
2k-1
1B
1+ε
1
C
Consequently, D1(Fk) = D1(Fk-1) + 1+ε + 2k-1 + D2(Fk-1)
Similarly, D2(Fk) = 2+ε + D1(Fk-1)
Basis for the induction: F2
2
2
1
1+ε
1
1+ε
1
1
1+ε
1
1+ε
1+ε
D1(F2) = 5 + ε
D2(F2) = 3 + ε
D1(Fk) = D1(Fk-1) + 1+ε + 2k-1 + D2(Fk-1)
D2(Fk) = 2+ε
k
2
3
4
5
D1(Fk)
5+ε
13 + 3ε
29 + 6ε
61 + 11ε
D2(Fk)
3+ε
7+ε
15 + 4ε
31 + 7ε
1
Let us ignore the precise values of the  terms
k
2
3
4
5
k
D2(Fk)
3+
7+
15+
31+
(2k-1)+
D2(Fk-1) +
1+
4+
8+
16+
(2k-1)+
2k-1
4
8
16
2k-1
So the correct NN choice is the 2k-1 edge, as claimed.
NN Running Time
• To find the kth vertex, k > 1, find the
shortest distance among N-k-1
candidates.
• Total time = Θ(
) = Θ(N2)
Greedy (Multi-Fragment) (GR):
1.
Sort the edges, shortest first, and treat them in that
order. Start with an empty graph.
2. While the current graph is not a TSP tour, attempt to
add the next shortest edge. If it yields a vertex degree
exceeding 2 or a tour of length less than N, delete.
• Theorem [Ong & Moore, “Worst-case analysis of two travelling
salesman heuristics,” Inform. Proc. Letters 2 (1984), 273-277]:
– For all instances I obeying the -Inequality,
GR(I)/OPT(I) = O(logN).
• Theorem [A. M. Frieze, Worst-case analysis of algorithms for
travelling salesman problems,” Methods of Operations Research 32
(1979), 93-112.]:
– There are N-city instances IN for arbitrarily large N
that obey the -Inequality and have
GR(IN)/OPT(IN) = Ω(logN/loglogN).
Greedy Running Time
• Initial sort takes Θ(N2log(N)).
• Processing an edge takes constant time to
check for degree 3, and two union-find
operations to check for short cycles, for a
total of at most Θ(N2α(N)).*
• Total time = Θ(N2log(N)).
*Note: The union-find operations can be avoided by storing with each vertex u its
degree in the current forest (initially 0) and, if the degree is 1, the identity of the
vertex v at the other end of the path containing it, which rules out edge {u,v} as a
potential choice of next edge. This information is easily updated in constant time
whenever we add an edge.
Clarke-Wright “Savings” Heuristic
1.
Start with a pseudo-tour in which an arbitrarily chosen city is the
“hub” and the salesman returns to the hub after visiting each city (a
multigraph in which every hub city is connected by two edges to the
hub).
2.
For each pair of non-hub cities, let the “savings” be the amount by
which the pseudo-tour would be shortened if we added an edge
between the two and deleted one edge to the hub from each.
3.
As long as we do not yet have a tour, find a pair of non-hub cities that
have not yet undergone two such shortcuts, and yields the most
savings, and perform the shortcut for them.
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
Clarke-Wright “Savings” Heuristic
Theorem [A. M. Frieze, Worst-case analysis of algorithms for travelling salesman
problems,” Methods of Operations Research 32 (1979), 93-112]:
•
There are N-city instances IN for arbitrarily large N that obey the -Inequality
and have
Savings(IN)/OPT(IN) = Ω(logN/loglogN).
No upper bounds are known under the triangle inequality.
We do have RN(A) = Θ(log(N)) for a sequential variant on the Savings heuristic, in
which we
1.
Pick an initial non-hub vertex v as the “current” vertex.
2.
Perform the best legal shortcut involving v and some other non-hub vertex that
has not yet been involved in a shortcut.
3.
Declare the other non-hub vertex involved in the shortcut to be the new
“current” vertex.
4.
Repeat until all non-hub vertices have been involved in a shortcut.
[Ong & Moore, 1984] [B. Golden, “Evaluating a sequential vehicle routing algorithm,”
AIIE Transactions 9 (1977), 204-208]
Savings Heuristic Running Time
• Mirrors Greedy:
– Sort all (N-1)(N-2)/2 potential shortcuts by decreasing
savings: Θ(N2log(N)).
– Processing a shortcut takes constant time for involvement in
more than two shortcuts, and two union-find operations to
check for short cycles, for a total of at most Θ(N2α(N)).*
– Total time = Θ(N2log(N)).
• Sequential version mirrors NN analogously.
– Total time = Θ(N2).
*See Footnote for Greedy running time – same observations apply.
Nearest Addition (NA):
1.
Start with 2-city tour consisting of some city and its
nearest neighbor.
2. Repeatedly insert the non-tour city u that is closest to a
tour city v into one of the tour edges involving v (the one
that yields the shorter tour).
Insertion Variants
Nearest Insertion (NI):
1.
Start with 2-city tour consisting of some city and its nearest
neighbor.
2. Repeatedly insert the non-tour city u that is closest to a tour
city v into the tour edge that yields the shortest new tour (not
necessarily one involving v).
Cheapest Insertion (CI):
1.
Start with 2-city tour consisting of some city and its nearest
neighbor.
2. Repeatedly perform the insertion of a non-tour vertex w into a
tour edge {u,v} that results in the shortest new tour.
More Insertion Variants
Farthest Insertion (FI):
1.
Start with 2-city tour consisting of the two cities at maximum
distance from each other.
2.
Repeatedly insert the non-tour city u, whose minimum distance to a
tour city is maximum, into the tour edge that yields the shortest
new tour.
Arbitrary Insertion (AI):
1.
Start with some 2-city tour.
2. Repeatedly pick some non-tour vertex u and insert it into the
tour edge {u,v} that yields the shortest new tour.
Farthest Insertion
Theorem [Rosenkrantz, Stearns, & Lewis, 1977]:
If instance I obeys the triangle inequality, then
1.
RN(AI) = O(log(N)).
2.
RN(NA) = RN(NI) = RN(CI) = 2.
The best lower bound known for RN(AI) is Ω(log(N)/loglog(N))
[Y. Azar, “Lower bounds for insertion methods for TSP,”
Combinatorics, Probability & Computing 3 (1994) 285-292].
The best lower bound known for RN(FI) is RN(FI) ≥ 6.5. [C. A. J.
Hurkens, “Nasty TSP instances for farthest insertion,” IPCO
Proc. (1992), 346-352].
Upper Bound Proofs
• The proof that RN(AI) = O(log(N)) mirrors that for NN.
• For the RN(A) ≤ 2 bounds, the results for NA and NI are
straightforward:
– Note that the vertices are added to the tour in the same order that
they are added to the tree in Prim’s algorithm for computing an MST,
and the optimal tour is at least as long as the MST.
– Thus we need only show that the increase in tour length when a vertex
u is added under NA is no more than twice the increase in tree length
in Prim’s algorithm, or 2d(u,v), where v is u’s nearest neighbor in the
tour. The result for NI will follow since its cost at each step never
exceeds NA’s cost.
w
d(u,w) ≤ d(u,v) + d(v,w) by -inequality, so
u
v
d(u,w) + d(u,v) – d(v,w) ≤ 2d(u,v)
Upper Bound for CI
• Basic plan: We will associate each insertion with a unique edge of
the MST, with the cost of the insertion being no more than twice
the cost of the edge.
• Label the vertices v1, v2, …, vN in the order by which they are
added to the tree under CI. Let T be the MST and let Ti be the
tree just before vi is added.
• Say vj is compatible with vi if j < i and if vk is an internal vertex on
the unique path in T between vj and vi, then k > i. (In other words,
vk is not in Ti).
• For each i > 1, the “critical vertex” for vi is the compatible vertex
for vi with largest index, and the ”critical edge” for vi is the first
edge on the path to vi from its critical vertex.
Critical Vertices and Edges
1
7
4
8
3
9
2
10
6
5
Note: In this example, every edge is critical for precisely one vertex.
We can prove that this will always be the case.
Unique Criticality
Suppose an edge {vi,vj}, i < j, is critical for two distinct vertices, vh and vk, with h < k.
vk
vh
vj
vi
Since vi is the critical vertex for vh, we have that h > i and that vj and all the
internal vertices in the path from vi to vh must exceed h. (We cannot have h = j
since that would imply j < k and so vi would not be compatible with vk.)
Since {vi,vj} is critical for vk, all the internal vertices of the path from vi to vk
must exceed k > h. (In the case when k = j there are no internal vertices.)
Hence, all the internal vertices on the (blue) path from vh to vk must have
indices exceeding h, and so vh is compatible with vk, with index h > i, implying
that vi is NOT the critical vertex for vk and {vi,vj} is not the critical edge, a
contradiction.
Completing the CI proof
We first show that the cost of inserting vertex vk is no more than twice the
cost of its critical edge {vi,vj}, i < j, for vk, 2 ≤ j ≤ N.
•
Let the critical edge be {vi,vj}, i < j. By definition of critical edge we must
have j ≥ k.
•
Thus, at the time vk is added, vi was in the tour but vj was not.
•
Hence d(vi,vj) must be at least as large as the length of the shortest edge
joining a tour vertex u to a non-tour vertex v.
•
By our NA argument, the insertion of v into a tour edge incident on u will
cost at most 2d(u,v) ≤ 2d(vi,vj). The cheapest insertion can cost no more.
Summing over all inserted vertices we get that the total tour length for CI is
at most two times the sum of the lengths of all the critical edges, which by the
uniqueness lemma is simply the length of the MST.
QED
Lower Bound Examples
2-ε
2-ε
2-ε
2-ε
Adjacent points have distance 1, all other distances
are the distance along the line minus ε.
2-ε
2-ε
2-ε
NA(I) = NI(I) = CI(I) = 2N-2-(N-2)ε
2-ε
OPT(I) = N+1-ε
More Lower Bounds:
Double MST and Christofides
Recall: Start by computing a minimum spanning tree [O(N2) time]
Then add edges so that every vertex has even degree.
Double MST
Double the edges of the
spanning tree [O(N) time]
Christofides
Add minimum-weight matching on
odd-degree vertices [O(N3) time]
Find an Euler tour for the resulting graph [O(N) time]
Traverse the Euler tour, taking shortcuts to avoid revisiting vertices
[O(N) time]
More Lower Bounds: Double MST Algorithm
MST
Length = n + (n+1)(1-ε) + 2ε
= 2n + 1 – (n-1)ε
DoubleMST Tour
Length ∼ 2n + (2n)(1-ε) + 2ε
= 4n – 2(n-1)ε
OptimalTour
Length ∼ 2n + 2
More Lower Bounds: Christofides Algorithm
1
N cities on bottom, N+1 on top.
Distance to nearest city in same row = 1.
Distance to nearest city on other row = 1 – ε’ .
OPT: Length = 2N + 1 - 2ε’
MST: Length = 2N(1 – ε’)
Christofides: Length = 2N(1 – ε’) + N = 3N – 2Nε’
Subquadratic Algorithms for Euclidean Instances:
Strip
1.
Let R be a minimum rectangle containing all the cities.
2.
Partition R into floor(sqrt(N)/3) vertical strips.
3.
Sort the cities within each strip by y-coordinate.
4.
Starting with the bottom-most point in the leftmost strip, traverse
the cities up one strip and down the next until all have been visited,
and then return to the starting point.
Total Running time O(Nlog(N)).
∂
∂
∂
∂
∂
∂∂
∂
∂
∂
∂
∂
∂
∂
∂
∂
∂∂
∂
∂
∂ ∂
∂ ∂
∂ ∂
∂∂
∂
∂
∂∂
∂
∂
∂
∂
∂∂
∂
∂
∂
∂
∂
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
= Ω(sqrt(N)OPT)
Strip(IN) > sqrt(N)/3)(N/4)
Assuming neighboring points
are one unit apart, OPT = N
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂∂
Lower Bounds for Strip
Subquadratic Algorithms for Euclidean Instances:
Spacefilling Curve
Visit the cities in the order that they occur along a spacefilling
curve for a square that contains them all.
Wikipedia Entry for Spacefilling Curve
Running time O(Nlog(N)).
For details, see [Platzmann & Bartholdi, “Spacefilling curves and the
planar travelling salesman problem,” J. ACM 36 (1989), 719-737].
More Geometric Tour Construction Heuristics
• Insertion variants for geometric instances
– Cheapest Insertion into the convex hull (CHCI)
– “Greatest Angle” insertion into the convex hull (CHGA)
– Convex Hull, Cheapest Insertion with greatest angle (CCA)
• Double Strip (best of both horizontal and vertical strip tours) (DST)
• Karp’s Partitioning Algorithm (KP)
• Litke’s Clustering Algorithm
• Bentley’s Fast Recursive Partitioning Heuristic (FRP)
Performance “In Practice”
Data Sources
•
Johnson, Bentley, McGeoch, & Rothberg, “Near-Optimal Solutions to
Very Large Traveling Salesman Problems,” unpublished (and as-yetuncompleted) monograph (1994).
•
Johnson & McGeoch, “The traveling salesman problem: A case study
in local optimization,” chapter in Local Search in Combinatorial
Optimization, Aarts & Lenstra (editors), Princeton University Press,
Princeton, NJ, 2003, 215-310 [Also available on DSJ’s website].
•
Johnson & McGeoch, “Experimental analysis of heuristics for the
STSP,” chapter in The Traveling Salesman Problem and its
Variations, Gutin & Punnen (editors), Kluwer Academic Publishers,
Dordrecht, 2002, 369-443 [Also available on DSJ’s website].
•
Website for the “8th DIMACS Implementation Challenge: The
Traveling Salesman Problem”
[http://dimacs.rutgers.edu/Challenges/TSP – only minor updates
since 2002].
Performance “In Practice”
• Testbed 1: Random Euclidean Instances
N = 10,000
(Results appear to be reasonably well-correlated
with those for our real-world instances.)
Performance “In Practice”
• Testbed 2: Random Clustered Instances
N = 3,162
1,000
10,000
Choose N/100 centers uniformly, then generate 100
normally-distributed points around each.
Performance “In Practice”
• Testbed 3: Instances from TSPLIB
Printed Circuit Boards
Geography
Laser Logic
Performance “In Practice”
• Testbed 4:
Random Symmetric Distance Matrices
(Unlikely to obey Triangle Inequality.)
Let’s start with Random Euclidean and just a few algorithms…
Nearest Neighbor
Greedy
Percent Excess over Held-Karp bound
50
40
30
20
10
0
100
1000
Double MST
Nearest Insertion
Christofides
Nearest Neighbor
Smart-Shortcut Christofides
Greedy
10000
Number of Cities
100000
Smart-Shortcut Christofides
1e+06
Standard Shortcuts
Smart Shortcuts
Random Euclidean Performance for N = 1,000,000
(% Excess over HK Bound)
Algorithm
Smart-Shortcut Christofides
%
9.8
Savings
12.2
Farthest Insertion
13.5
Greedy
14.2
Classic-Shortcut Christofides
14.5
Random Insertion
15.2
Convex Hull, Cheapest Insertion
22.0
Cheapest Insertion
22.1
Nearest Neighbor
23.3
Nearest Insertion
27.0
Strip
30.2
Nearest Addition
32.6
Spacefilling Curve
35.1
Double MST w. Smart Shortcuts
39.9
Solution Quality Distributions
Random
Euclidean
N = 10,000
Smart-Shortcut Christofides
Savings
Farthest Insertion
Greedy
Savings versus Classic Christofides
Nearest Neighbor versus Nearest Insertion
Greedy versus Nearest Insertion
Savings versus Nearest Insertion
Conclusion:
Many of the heuristics that are
famous theoretically are totally
dominated in practice.
Question:
Why care about these dominated
heuristics?
Tour Quality versus Running Time Tradeoffs
Random
Euclidean
N = 1,000