2620001 Data Structures Graph Data Structure What is a graph? A data structure that consists of a set of nodes (vertices) and a set of edges that relate the nodes to each other The set of edges describes relationships among the vertices 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Formal definition of graphs A graph G is defined as follows: G=(V,E) V(G): a finite, nonempty set of vertices E(G): a set of edges (pairs of vertices) 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Directed vs. undirected graphs When the edges in a graph have no direction, the graph is called undirected 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Directed vs. undirected graphs (cont.) When the edges in a graph have a direction, the graph is called directed (or digraph) Warning: if the graph is directed, the order of the vertices in each edge is important !! E(Graph2) = {(1,3) (3,1) (5,9) (9,11) (5,7) (11,1) (9,9)} 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Trees vs graphs Trees are special cases of graphs!! 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Graph terminology Adjacent nodes: two nodes are adjacent if they are connected by an edge 5 is adjacent to 7 7 is adjacent from 5 Path: a sequence of vertices that connect two nodes in a graph Complete graph: a graph in which every vertex is directly connected to every other vertex 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Graph terminology (cont.) What is the number of edges in a complete directed graph with N vertices? N * (N-1) 2 O( N ) 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Graph terminology (cont.) What is the number of edges in a complete undirected graph with N vertices? N * (N-1) / 2 2 O( N ) 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Graph terminology (cont.) Weighted graph: a graph in which each edge carries a value 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Graph implementation Array-based implementation A 1D array is used to represent the vertices A 2D array (adjacency matrix) is used to represent the edges 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Array-based implementation 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Graph implementation (cont.) Linked-list implementation A 1D array is used to represent the vertices A list is used for each vertex v which contains the vertices which are adjacent from v (adjacency list) 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Linked-list implementation 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Graph searching Problem: find a path between two nodes of the graph (e.g., Austin and Washington) Methods: Depth-First-Search (DFS) or Breadth-First-Search (BFS) 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Depth-First-Search (DFS) What is the idea behind DFS? Travel as far as you can down a path Back up as little as possible when you reach a "dead end" (i.e., next vertex has been "marked" or there is no next vertex) DFS can be implemented efficiently using a stack 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Use of a stack It is very common to use a stack to keep track of: nodes to be visited next, or nodes that we have already visited. Typically, use of a stack leads to a depth-first visit order. Depth-first visit order is “aggressive” in the sense that it examines complete paths. 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Graph Traversals •Both take time: O(V+E) 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot start end (initialization) 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot (BFS) What is the idea behind BFS? Look at all possible paths at the same depth before you go at a deeper level Back Breadth-First-Searching up as far as possible when you reach a "dead end" (i.e., next vertex has been "marked" or there is no next vertex) BFS can be implemented efficiently using a Queue 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Use of a queue It is very common to use a queue to keep track of: nodes to be visited next, or nodes that we have already visited. Typically, use of a queue leads to a breadth-first visit order. Breadth-first visit order is “cautious” in the sense that it examines every path of length i before going on to paths of length i+1. 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot start end (initialization) 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot next: 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Breadth First Search 2 s 5 3 620001 – Data Structures 4 8 7 6 9 Mr. Deven Patel , AITS, Rajkot Breadth1 First Search Shortest path from s 0 2 s 4 5 3 8 7 6 9 Undiscovered Discovered Queue: s Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Breadth1 First Search 2 0 s 4 5 3 8 7 6 9 1 Undiscovered Discovered Queue: s 2 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Breadth1 First Search 2 0 4 5 s 8 7 1 3 6 9 1 Undiscovered Discovered Queue: s 2 3 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Breadth1 First Search 2 0 s 4 5 8 7 1 3 6 9 1 Undiscovered Discovered Queue: 2 3 5 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 4 5 8 7 1 3 6 9 1 Undiscovered Discovered Queue: 2 3 5 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 4 5 already discovered: 7 don't enqueue 5 1 3 8 6 9 1 Undiscovered Discovered Queue: 2 3 5 4 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 4 5 8 7 1 3 6 9 1 Undiscovered Discovered Queue: 2 3 5 4 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 4 5 8 7 1 3 6 9 1 Undiscovered Discovered Queue: 3 5 4 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 4 5 8 7 1 3 6 1 2 9 Undiscovered Discovered Queue: 3 5 4 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 4 5 8 7 1 3 6 1 2 9 Undiscovered Discovered Queue: 3 5 4 6 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 4 5 8 7 1 3 6 1 2 9 Undiscovered Discovered Queue: 5 4 6 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 4 5 8 7 1 3 6 1 2 9 Undiscovered Discovered Queue: 5 4 6 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 4 5 8 7 1 3 6 1 2 9 Undiscovered Discovered Queue: 4 6 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 5 8 7 1 3 6 1 2 9 Undiscovered Discovered Queue: 4 6 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 5 8 7 1 3 6 1 2 9 Undiscovered Discovered Queue: 4 6 8 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 1 2 9 Undiscovered Discovered Queue: 6 8 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Undiscovered Discovered Queue: 6 8 7 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Undiscovered Discovered Queue: 6 8 7 9 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Undiscovered Discovered Queue: 8 7 9 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Undiscovered Discovered Queue: 7 9 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Undiscovered Discovered Queue: 7 9 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Undiscovered Discovered Queue: 7 9 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Undiscovered Discovered Queue: 7 9 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Undiscovered Discovered Queue: 9 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Undiscovered Discovered Queue: 9 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Undiscovered Discovered Queue: 9 Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Undiscovered Discovered Queue: Top of queue Finished 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot 2 Breadth1 First Search 2 0 s 3 4 8 5 7 1 3 3 6 9 1 2 3 Level Graph 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Applications Using the template method pattern, we can specialize the BFS traversal of a graph G to solve the following problems Compute the connected components of G Compute a spanning forest of G Find a simple cycle in G, or report that G is a forest 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot DFS vs. BFS Applications DFS BFS Spanning forest, connected components, paths, cycles Shortest paths Biconnected components A B C E F DFS 620001 – Data Structures D L1 L0 A B L2 C E D F BFS Mr. Deven Patel , AITS, Rajkot Paths and cycles A path is a sequence of nodes v1, v2, …, vN such that (vi,vi+1)E for 0<i<N The length of the path is N-1. Simple path: all vi are distinct, 0<i<N A cycle is a path such that v1=vN An acyclic graph has no cycles 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Cycles BOS SFO DTW PIT JFK LAX 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot More useful definitions In a directed graph: The indegree of a node v is the number of distinct edges (w,v)E. The outdegree of a node v is the number of distinct edges (v,w)E. A node with indegree 0 is a root. 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Trees are graphs A dag is a directed acyclic graph. A tree is a connected acyclic undirected graph. A forest is an acyclic undirected graph (not necessarily connected), i.e., each connected component is a tree. 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Example DAG Under shorts Socks Watch Shoes Pants Shirt Belt Tie a DAG implies an ordering on events Jacket 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Minimum Spanning Trees 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Problem: Laying Telephone Wire Central office 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Wiring: Naive Approach Central office Expensive! 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Wiring: Better Approach Central office Minimize the total length of wire connecting the customers 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Minimum Spanning Tree (MST) A minimum spanning tree is a subgraph of an undirected weighted graph G, such that it is a tree (i.e., it is acyclic) it covers all the vertices V contains |V| - 1 edges the total cost associated with tree edges is the minimum among all possible spanning trees not necessarily unique 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot How Can We Generate a MST? 9 a 2 5 4 c 620001 – Data Structures 6 d 4 5 9 b 5 e a 2 5 4 c b 6 d 4 5 5 e Mr. Deven Patel , AITS, Rajkot Prim’s Algorithm Initialization a. Pick a vertex r to be the root b. Set D(r) = 0, parent(r) = null c. For all vertices v V, v r, set D(v) = d. Insert all vertices into priority queue P, using distances as the keys 9 a 2 5 4 c 620001 – Data Structures b 6 d e a b c d 4 5 5 Vertex Parent e - 0 e Mr. Deven Patel , AITS, Rajkot Prim’s algorithm e d b c a 9 a 2 5 6 d 4 4 c b 0 Vertex Parent e b c d - 5 5 e d b c a 4 5 5 Vertex Parent e b e c e d e The MST initially consists of the vertex e, and we update the distances and parent for its adjacent vertices 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Prim’s algorithm d b c a 9 a 2 5 4 c b 4 5 5 6 d 4 5 5 e a c b 2 4 5 620001 – Data Structures Vertex Parent e b e c e d e Vertex Parent e b e c d d e a d Mr. Deven Patel , AITS, Rajkot Prim’s algorithm a c b 2 4 5 9 a 2 5 4 c b 6 d 4 5 5 e c b 4 5 620001 – Data Structures Vertex Parent e b e c d d e a d Vertex Parent e b e c d d e a d Mr. Deven Patel , AITS, Rajkot Prim’s algorithm c b 9 a 2 5 4 c b 4 5 6 d 4 5 5 e b 5 620001 – Data Structures Vertex Parent e b e c d d e a d Vertex Parent e b e c d d e a d Mr. Deven Patel , AITS, Rajkot Prim’s algorithm b 9 a 2 5 4 c b 5 6 d 4 5 5 e The final minimum spanning tree 620001 – Data Structures Vertex Parent e b e c d d e a d Vertex Parent e b e c d d e a d Mr. Deven Patel , AITS, Rajkot Running time of Prim’s algorithm (without heaps) Initialization of priority queue (array): O(|V|) Update loop: |V| calls • Choosing vertex with minimum cost edge: O(|V|) • Updating distance values of unconnected vertices: each edge is considered only once during entire execution, for a total of O(|E|) updates 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Prim’s Algorithm Invariant At each step, we add the edge (u,v) s.t. the weight of (u,v) is minimum among all edges where u is in the tree and v is not in the tree Each step maintains a minimum spanning tree of the vertices that have been included thus far When all vertices have been included, we have a MST for the graph! 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Another Approach • Create a forest of trees from the vertices • Repeatedly merge trees by adding “safe edges” until only one tree remains • A “safe edge” is an edge of minimum weight which does not create a cycle 9 a 2 5 6 d 4 4 c 620001 – Data Structures b 5 5 forest: {a}, {b}, {c}, {d}, {e} e Mr. Deven Patel , AITS, Rajkot Kruskal’s algorithm Initialization a. Create a set for each vertex v V b. Initialize the set of “safe edges” A comprising the MST to the empty set c. Sort edges by increasing weight 9 a 2 5 4 c 620001 – Data Structures b 6 d 4 5 5 e F = {a}, {b}, {c}, {d}, {e} A= E = {(a,d), (c,d), (d,e), (a,c), (b,e), (c,e), (b,d), (a,b)} Mr. Deven Patel , AITS, Rajkot Kruskal’s algorithm For each edge (u,v) E in increasing order while more than one set remains: If u and v, belong to different sets U and V a. add edge (u,v) to the safe edge set A = A {(u,v)} b. merge the sets U and V F = F - U - V + (U V) Return A Running time bounded by sorting (or findMin) O(|E|log|E|), or equivalently, O(|E|log|V|) (why???) 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Kruskal’s algorithm 9 a 2 5 4 c b 6 d E = {(a,d), (c,d), (d,e), (a,c), 4 5 5 e Forest {a}, {b}, {c}, {d}, {e} {a,d}, {b}, {c}, {e} {a,d,c}, {b}, {e} {a,d,c,e}, {b} {a,d,c,e,b} 620001 – Data Structures (b,e), (c,e), (b,d), (a,b)} A {(a,d)} {(a,d), (c,d)} {(a,d), (c,d), (d,e)} {(a,d), (c,d), (d,e), (b,e)} Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1? Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2? 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5? 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8? 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9? 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13? 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14? 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17? 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19? 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21? 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25? 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Run the algorithm: 2 8 14 21 620001 – Data Structures 19 25 9 17 5 13 1 Mr. Deven Patel , AITS, Rajkot Kruskal’s Algorithm Invariant After each iteration, every tree in the forest is a MST of the vertices it connects Algorithm terminates when all vertices are connected into one tree 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot Dijkstra Algorithm: Finding shortest paths in order Closest node to s is 1 hop away 2nd closest node to s is 1 hop away from s or w” 3rd closest node to s is 1 hop away from s, w”, or x Find shortest paths from source s to all other destinations w' z w s 620001 – Data Structures x w" z' x' Mr. Deven Patel , AITS, Rajkot Execution of Dijkstra’s algorithm 2 1 5 1 6 5 2 3 3 2 4 4 1 2 N D4 D5 D6 Initial {1} 3 5 1 {1,3} 3 2 4 3 2 {1,2,3} 3 2 4 7 3 3 {1,2,3,6} 3 2 4 5 3 4 {1,2,3,4,6} 3 2 4 5 5 {1,2,3,4,5 ,6} 3 2 4 5 620001 – Data Structures D2 D3 2 5 4 Iteratio n 6 3 2 5 1 3 2 3 4 1 1 3 2 2 3 3 Mr. Deven Patel , AITS, Rajkot Shortest Paths in Dijkstra’s Algorithm 2 1 2 3 3 2 2 1 2 1 2 1 3 2 4 2 3 1 6 2 4 1 3 2 5 5 5 2 2 3 3 4 1 4 4 6 3 2 1 5 5 6 2 3 1 3 5 2 3 2 5 3 4 4 620001 – Data Structures 1 2 2 2 3 4 6 5 1 4 1 2 3 1 2 5 4 3 6 5 2 1 3 3 4 1 2 1 6 5 1 1 3 4 2 5Mr. Deven Patel , AITS, Rajkot That’s All! 620001 – Data Structures Mr. Deven Patel , AITS, Rajkot