6.046 Recitation 8 April 6, 2012 Outline Course Topics 1 1 Aliens Problem 1.1 Understanding the Problem 1.2 Trying to Solve the Problem 1.3 Working Out the Details . . 1.4 Reflect and Improve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 2 4 5 2 Simple Path of Length 7 2.1 Understanding the Problem 2.2 Trying to Solve the Problem 2.3 Working Out the Details . . 2.4 Reflect and Improve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 6 6 8 9 Course Topics Over the course of the semester, we have seen a number of tech­ niques that are useful for designing algorithms. In the first part of the semester, we learned algorithms that used greedy choice, dy­ namic programming, divide-and-conquer, and reductions to previ­ ously solved problems. In the second third of the semester, we learned about how to use randomization and amortization, as well as how to analyze online algorithms in comparison to the optimal solution. Any or all of these techniques may be useful in designing your own algorithms for Quiz 2. In the case of reductions, it’s good to have in mind another list: a list of problems that you know how to solve. To the right there is a list of problems and the runtimes of the algorithms that solve them. This list is, of course, not complete — it’s missing problems such as scheduling, matrix multiplication testing, and so forth. However, each of the algorithms on the list has been (a) covered this semester in lecture, and (b) has at least one chapter devoted to it in CLRS. So these problems are in some sense the big ones. 1 Algorithm Design Tools. • • • • • • • greedy choice dynamic programming divide and conquer reduction to solved problem randomization amortization competitive analysis (Selected) Solved Problems. • • • • • • • select: Θ(n) sorting: Θ(n lg n) minimum spanning tree: Θ(E + V lg V ) discrete Fourier transform: Θ(n lg n) all-pairs shortest paths: Θ(V E + V 2 lg V ) hashing: O(1) insert, delete, lookup network flow: Θ(f ∗ · V ) or Θ(V 3 ) 6.046 Recitation 8 1 April 6, 2012 Aliens Problem Because the quiz is entirely focused on problem-solving, a good way to review for the quiz is to try to solve problems. To that end, let’s consider the following problem: Problem 1. Ripley is holed up inside a building trying to hide from aliens. The building consists of n rooms joined by m doors. Some rooms have a door to the outside, which could allow the aliens to enter. Other rooms are currently being occupied by a small band of survivors, whom Ripley is determined to protect. Luckily, Ripley has k « n guard robots that she can place in var­ ious rooms around the building (including rooms that contain sur­ vivors or rooms that may contain aliens) to keep the aliens from get­ ting to the survivors. If a room contains a guard robot, no aliens can pass through the room. Devise an algorithm to help Ripley and her band of survivors last the night (or report that survival is impossible). 1.1 • building with n rooms, m doors • known subset of rooms may contain aliens • known subset of rooms contain survivors • k « n guard robots to place in rooms • GOAL: protect survivors from aliens with guards Understanding the Problem The very first step in problem-solving is understanding the prob­ lem. A good first step is coming up with an example, preferably one that is small enough to understand, but large enough to capture some of the complexity. The example shown on the right has seven rooms, with a large number of doors connecting them. In this exam­ ple, if k = 1, then there are two possible locations in which to put the guard robot: one in the giant room at the center, or one in the room where the aliens might enter from. But it’s hard to model this as rooms and doors. Rooms have dimension, doors have locations, but the only thing we care about (according to the problem) is connectivity: whether there’s a way for the aliens to enter through some allowable room, and then pass from room to room via doors until they reach a survivor without ever passing a guard robot. So, because each door connects two rooms, we model this as a graph problem, as depicted at right. We can now restate the problem as follows: given an unweighted undirected graph G = (V, E) and subsets H ⊆ V and A ⊆ V , find a set R ⊆ V of vertices with size k such that any path from an alien a ∈ A to a survivor h ∈ H contains some vertex in R. 1.2 Aliens Problem. Trying to Solve the Problem With the problem rephrased this way, there is something about it that rings a bell. Our goal is to find a subset R of vertices such that any path from an alien to a human includes one of the vertices in R. This is somewhat reminiscent of the edges crossing an s-t cut: any path from s to t must contain some edge crossing the cut. What algorithms do we know that involve s-t cuts? Well, we know that s-t cuts are closely linked to s-t flows: that the capacity 2 Example. aliens survivors survivors ⇓ 6.046 Recitation 8 April 6, 2012 of the minimum cut is equal to the maximum flow. In fact, there is a way to compute the minimum s-t cut from the maximum s-t flow. So if we could find a way to reduce the aliens problem to the minimum s-t cut problem, then we would be able to find an algorithm for the aliens problem. What might a reduction look like? Well, although we have dis­ cussed some of the similarities between the minimum s-t cut and the aliens problem, we must also keep in mind the differences. The min­ imum s-t cut problem involves finding a cut that minimizes the sum of the capacities of the edges crossing the cut in the direction from s to t. So we can think of the s-t cut problem as trying to find a set of edges of minimum total capacity that are necessary to get from s to t. But in the aliens problem, we want to find a set of vertices of size k that are necessary to get from A to H. There are several differences here. First, one problem involves finding a set of edges; the other in­ volves finding a set of vertices. So if we want to take advantage of the known algorithm for min s-t cut, we need some way to convert between vertices and edges. We might consider doing the following: find a set of edges constituting a minimum cut, and then split the vertices incident to those edges into two groups based on which side of the cut that they fall into. If we pick the smaller of the two groups, we might get a good set of vertices. Unfortunately, the counterex­ ample to the right shows that this technique doesn’t work. Instead, we’ll try to look for another way. It would be nice to have a one-to-one correspondence such that for each vertex, there is an edge in the transformed graph such that we choose a vertex to guard exactly when we choose to cut the edge. One way to do this would be to split every vertex in the original graph into two vertices in the transformed graph, connected by a single edge. We will figure out the details of this splitting process later. Second, one problem involves minimizing the sum of the weights of items in the set; the other involves picking a set whose size is k. Luckily, we’re trying to convert from the unweighted case to the weighted case, so it would seem like we could handle part of this difference by setting all weights / capacities equal to 1. Furthermore, note that if there is a set of vertices of size < k, then we can construct a set of vertices of size k that also separates the humans from the aliens by adding extra vertices to the set. So if we find the minimumsized set R∗ of vertices that separates the humans from the aliens, then we can solve the problem as follows: if |R∗ | > k, then the survivors will die; if |R∗ | ≤ k, then we can add extra guard posts until we get R with |R| = k. Third, one problem involves cutting off paths from a single source to a single sink; the other involves multiple humans and multiple aliens. However, we saw in lecture how to handle this kind of prob­ lem: create a new source that points to all of aliens, and create a new sink that is pointed to by all of the humans. 3 Reduce to Minimum s-t Cut. • • • • • transform vertices into edges give edges capacity 1 use minimum to find a set of size = k s is a supernode connected to all aliens t is a supernode connected to all survivors Counterexample. A S A S A S A S A S A S 6.046 Recitation 8 1.3 April 6, 2012 Working Out the Details Now that we’ve worked out an idea for an algorithm, it’s time to formalize it. We want a reduction that takes as input an instance of the aliens problem (G = (V, E), A, H, k) and outputs an instance (G/ = (V / , E / ), s, t) of the minimum s-t cut problem. This reduction should have the property that we can use a minimum s-t cut of (G/ , s, t) to produce a way to protect the humans in H from the aliens in A. Our goal is to set up G/ so that there is a nearly one-to-one corre­ spondence between paths in G and paths in G/ . We have previously decided to take each vertex v ∈ V and create two vertices in V / con­ nected by an edge in E / . For reasons which will later become clear, call those two vertices vin and vout . Due to this known mapping between vertices in both graphs, it seems like it would be useful to make it so that each path in G/ that passes through vin or vout cor­ responds to a path in G that passes through v. In fact, because the edge (vin , vout ) was created to be the equivalent of the vertex v, it would be nice to make sure that any path in G that passes through v corresponds to a path G/ that passes through the edge (vin , vout ). In other words, we want to make it so that any path in G/ that passes through vin or vout should pass through the edge (vin , vout ). Direct the edge (vin , vout ) to point from vin to vout . To make sure that there’s no way to construct a path that goes into vin and leaves without going to vout , it is sufficient to make vin have no other outgoing edges. To make sure that there’s no way to construct a path that goes through vout without having just come from vin , it is sufficient to make vout have no other incoming edges. This gives us the constraint we want. With this in mind, we can now write the reduction to the min s-t cut problem: AlienReduction(G = (V, E), A, H) 1 create an empty directed graph G/ = (V / , E / ) with capacities 2 for each vertex v ∈ V 3 create vertices vin , vout ∈ V / 4 create directed edge (vin , vout ) ∈ E / with capacity 1 5 for each edge (u, v) ∈ E 6 create directed edge (uout , vin ) ∈ E / with capacity ∞ 7 create directed edge (vout , uin ) ∈ E / with capacity ∞ 8 create a new node s ∈ V / 9 for each vertex v ∈ A 10 create directed edge (s, vin ) ∈ E / with capacity ∞ 11 create a new node t ∈ V / 12 for each vertex v ∈ H 13 create directed edge (vout , t) ∈ E / with capacity ∞ 14 return (G/ , s, t) And using this reduction as a subroutine, we can write the algo­ 4 Transformation Details. • turn vertex v into vin , vout • create edge (vin , vout ) with capacity 1 • for each edge (u, v), new edges: – edge (uout , vin ) with capacity ∞ – edge (vout , uin ) with capacity ∞ • total: 2n vertices, 2m + n edges Transformation Example. ⇓ 6.046 Recitation 8 April 6, 2012 rithm for solving our alien problem: AlienProblem(G = (V, E), A, H, k) 1 set (G/ , s, t) = AlienReduction(G, A, H) 2 run Ford-Fulkerson(G/ , s, t) to find the minimum cut 3 if the cut size is > k 4 return Nil 5 else 6 create an empty set R 7 for each edge (vin , vout ) in the cut 8 add the vertex v to R 9 while |R| < k 10 add an arbitrary new vertex to R 11 return R The runtime of this algorithm is Θ(n+m) to perform the transforma­ tion, and Θ(f ∗ (n + m)) to run Ford-Fulkerson. The maximum flow f ∗ is equal to the minimum cut, which is less than all other cuts. Because of the way that we have constructed the graph, the cut S = {vin |v ∈ V } ∪ {s} ensures that the total capacity of all edges crossing from S to V / − S is |V | = n, which gives us an upper bound on the flow. So our algorithm will get Θ(n(n + m)), which is Θ(mn) for a connected graph. 1.4 Reflect and Improve We have now constructed an algorithm. There is no randomiza­ tion involved — the algorithm is always correct, so there isn’t room for improvement there. But what about the runtime? Is there some way to improve it? Well, let’s look back at the runtime analysis. We said that the algorithm takes time Θ(f ∗ · m), and that f ∗ ≤ n. But in many cases, we’ll have even tighter bounds. If there is a way to position the k guards, then the value of the flow that we get is the capacity of the cut that we get, which is ≤ k. So it’s only when the algorithm returns Nil that it might have to spent Θ(mn) time — in all other cases, we can spend only Θ(km) time, which is better because k « n. Is there some way that we can improve the runtime in that one bad case? The answer is yes. Once we know that the value of the flow is ≥ k + 1, we don’t need to keep running Ford-Fulkerson. So we can either modify Ford-Fulkerson to use k + 1 iterations, or we can modify the graph that we create so that the flow from s to t is limited to ≤ k + 1 (by adding a replacement sink t/ and a new edge (t, t/ ) with capacity k + 1). This yields an algorithm with runtime Θ(km) in all cases. 5 Runtime. • • • • Ford-Fulkerson: O(f ∗ m) runtime: O(mn) IMPROVEMENT: stop after k+1 iterations runtime: O(km) 6.046 Recitation 8 2 April 6, 2012 Simple Path of Length 7 Now that we’ve settled the alien problem, we can move on to a new problem. Problem 2. Given an unweighted undirected graph G = (V, E), re­ turn any simple path of length 7, or return Nil if no such path exists. 2.1 • given unweighted undirected graph • return any simple path of length 7 • return Nil if no such path exists Understanding the Problem We begin, of course, by constructing an example such as the one at the right. This example will help us understand what it is that we’re searching for. The bold lines show a simple path of length 7. Note that the problem doesn’t specify anything about the start or the end of the path — as long as it has length 7. So in the depicted graph, there are multiple solutions. One crucial element of understanding this problem is understand­ ing what it means for a path to be simple. A non-simple path in a graph is a path in which some vertex occurs more than once. In other words, a non-simple path contains a loop. A simple path contains no loops, and does not contain any vertex more than once. Therefore, every simple path of length 7 consists of a sequence of unique vertices of length 8 connected by edges. With this knowledge, we can come up with a very rough upper bound. If we examine all sequences of 8 unique vertices and check to see whether the necessary edges exist for that sequence, then we can find the correct answer. The runtime of this brute-force algorithm is O(V 8 ), which gives us a very rough upper-bound for the problem. But at least we know it’s polynomial! 2.2 Simple Path of Length 7. Example. Upper Bound. • for each length-8 sequence of unique vertices – check whether all 7 edges exist – if so, return the path • runtime: O(V 8 ) Trying to Solve the Problem Most of the path problems that we’ve seen so far involve shortest paths of some sort. Dijkstra’s and Bellman-Ford can be used to find single-source shortest paths in weighted graphs. Floyd-Warshall and Johnson’s algorithm can be used to solve the all-pairs shortest path problem. In an unweighted graph, performing breadth-first search is equivalent to computing single-source shortest paths. Depth-first search (which you probably saw in 6.006) is one of the only graph search algorithms that doesn’t involve shortest paths. So it’s important to think about how the simple paths we want in this problem relate to shortest paths. For instance, can we use shortest paths to solve this problem? At the right, we show the shortest path between the ends of the path of length 7 in the example that we picked. As you can see, the shortest path has length 3. In fact, careful examination reveals that all shortest paths in this graph have length ≤ 3. Unfortunately, this means that we’re unlikely to be able to use shortest path algorithms to solve this problem. 6 Shortest Path. 6.046 Recitation 8 April 6, 2012 In fact, after further examination of the problem, it seems that the problem might actually be more closely related to the longest simple path problem. Suppose that we have the longest simple path in the graph G. If that path has length = 7, then we’re set — we’ve found the path we want. If that path has length > 7, then we can take a subpath of length 7. Otherwise, there is no simple path of length 7, and we can safely return Nil. Unfortunately, the longest simple path problem is quite difficult to solve. (For those of you who know complexity, you may recall that it’s NP-hard to find the length of the longest simple path in general graphs.) So this line of reasoning (reducing to the longest path problem, which is actually even harder for us to solve) looks like something of a dead end. However, the relationship between longest paths and shortest paths is an interesting one. We have a bunch of shortest-path al­ gorithms that will find the shortest path. Some of these algorithms even handle negative-weight edges. One fairly well-known technique for finding longest paths in certain graphs is negating all of the edge weights and running a shortest-path algorithm. What happens if we give all of our edges a weight of −1 and try to find longest paths? Unfortunately, that’s not going to work — in undirected graphs, it’s possible to zig-zag across any edge of weight −1 to make a path arbitrarilty short, so none of our shortest-path algorithms work when the graph is undirected. But even if we had a directed graph, this technique wouldn’t work — any negative-weight loop will cause our shortest-path algorithms to fail. And because all of our edges have weight −1, every loop has negative weight. So in order to be able to compute longest paths in this fashion, we’d need a directed acyclic graph. Let’s simplify the problem by imagining that our input is not in fact an undirected graph, but is instead a DAG. Fortunately, comput­ ing path lengths in a DAG is easy because of the topological ordering associated with each DAG. This topological ordering makes it easy to maintain the invariant that when you are examining vertex v, all of the predecessors of v in the DAG have already been processed. As a result, the runtime of shortest or longest paths in a DAG is O(V + E). (At right is a sketch of the algorithm.) Unfortunately, while this DAG result is nice, it doesn’t solve the original problem. But it would be nice if we could use it in some way. For instance, is there a way to take our original graph G = (V, E) and use it to construct a DAG so that the DAG contains a simple path of length 7 exactly when the original graph does? It seems like it would be ideal to make a DAG containing all of the vertices of our original graph, with directed versions of all the edges. That way we don’t accidentally toss out an edge that belongs to the path of length 7. But how can we figure out which way to direct all of the edges? We can use a permutation on all of the vertices: use the permutation as the topological order, and then make all of the 7 Relationship to Longest Path. • • • • let g be the length of the longest path if g = 7, return path if g > 7, return subpath if g < 7, return Nil Longest Path on a DAG. • GOAL: longest path in a DAG G = (V, E) • for each vertex v in topological order: – set d(v) = 0 – for each edge (u, v): ∗ set d(v) = max{d(v), 1 + d(u)} • runtime: O(V + E) Using DAGs. • • • • GOAL: reduce to DAG solution each permutation yields a different DAG no principled way of orienting edges pick a random permutation 6.046 Recitation 8 April 6, 2012 edges point from the node earlier in the selected ordering to the node later in the selected ordering. But how would we pick such an ordering? There are n! different orderings, where n = |V |, and each ordering could potentially yield a different DAG. So we certainly can’t try all of them. And we don’t really know anything about how to pick a “good” permutation — one containing a path of length 7. So rather than trying to figure out what a “good” permutation looks like, we’re just going to pick a random one, and see if we get something reasonable when we compute the probabilities. 2.3 Working Out the Details We can put together the details of everything that we’ve worked out to get the following pseudocode: SimplePath7(G = (V, E)) 1 create a directed graph G/ = (V / , E / ) 2 add all of the vertices in V to V / 3 pick a random ordering π of V 4 for each edge (u, v) ∈ E 5 if π(u) < π(v) 6 add (u, v) to E / 7 else 8 add (v, u) to E / 9 for each vertex v ∈ V in order of π 10 set d(v) = 0 11 for each edge (u, v) ∈ E / 12 set d(v) = max{d(v), 1 + d(u)} 13 if d(v) = 7 14 return the corresponding path 15 return Nil The runtime of this is Θ(V + E), which seems pretty good compared to our very high upper bound. But before we can conclude that this algorithm is good, we must analyze the probability of correctness. Note that the structure of this algorithm means that it cannot com­ pute false positives: the only error that it can make is concluding that there is no path when there actually is one. Suppose that (v1 , v2 ), (v2 , v3 ), . . . , (v7 , v8 ) is a path of length 7 in our original graph. What’s the probability that the path will also exist in the random DAG that we generate? Well, the path will also exist in the DAG if the process of creating the DAG caused all of the edges in the path to point in the same direction. In order for that to be true, the random order that we select should contain as a subsequence either v1 , v2 , . . . , v8 or v8 , v7 , . . . , v1 . In other words, of the 8! different orderings of v1 , . . . , v8 within the larger permuta­ tion, exactly two orderings give us the result we want. This means 8 Probability of Correctness. • worst case: only one path • (v1 , v2 ), (v2 , v3 ), . . . , (v7 , v8 ) • need all edges oriented correctly • need path to occur in permutation in correct order 2 1 • probability: 8! = 20160 6.046 Recitation 8 April 6, 2012 1 that 8!2 = 20160 of the n! different permutations are going to pre­ serve the path. The permutation is selected at random, so we have 1 . Pr[success] ≥ 20160 2.4 Reflect and Improve The probability of success that we have is pretty small. Usually, it’s good to have success probability ≥ 12 + E. So let’s try to improve our previous algorithm by repeating it t times. If at least one of those t iterations results in a simple path of length 7, we can return the path. Otherwise, we’ll return Nil. So what is the probability of correctness of this amplified algo­ rithm? Well, the only way we could return the wrong answer is by saying there isn’t a path when there is one. This would require us to not find that path in all t iterations. So the probability is: Pr[failure of all trials] = (1 − Pr[success of one trial])t = 20159 t 20160 If we want this failure probability to be < 13 , then we can solve for t to get t ≈ 23000. This yields an algorithm with Pr[correctness] > 32 and runtime Θ(t(V + E)) = Θ(V + E). 9 Better Probability. • • • • run algorithm t times 20159 t probability of always failing: 20160 when t ≈ 23000, probability of failure < 1/3 runtime: O(t(V + E)) = O(V + E) MIT OpenCourseWare http://ocw.mit.edu 6.046J / 18.410J Design and Analysis of Algorithms Spring 2012 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.