Lectures on
Graph Algorithms:
searching, testing and sorting
COMP 523: Advanced Algorithmic Techniques
Lecturer: Dariusz Kowalski
Graph Algorithms
1
Overview
Previous lectures:
• Recursive method (searching, sorting, …)
This lecture: algorithms on graphs, in particular
• Representation of graphs
• Breadth-First Search (BFS)
• Depth-First Search (DFS)
• Testing bipartiteness
• Testing for (directed) cycles
• Ordering nodes in acyclic directed graphs
Graph Algorithms
2
How to represent graphs?
Given an undirected graph G = (V,E) , how to represent it?
1. Adjacency matrix: ith node is represented as ith row
and ith column, edge between ith and jth nodes is
represented as 1 in row i and column j, and vice versa
(0 if there is no edge between i and j)
2. Adjacency list: nodes are arranged as array/list, each
node record has the list of its neighbors attached
1
4
2
3
1 2 3 4
1 0 1 1 1
2 1 0 1 0
3 1 1 0 1
4 1 0 1 0
Adjacency matrix
Graph Algorithms
1
2
3
4
2
1
1
1
3 4
3
2 4
3
Adjacency list
3
Adjacency matrix
Advantages:
–
Checking in constant time if an edge belongs to the graph
Disadvantages:
–
–
Representation takes memory O(n2) - versus O(m+n)
Examining all neighbors of a given node requires time O(n)
- versus O(m/n) in average
Disadvantages especially for sparse graphs!
1
4
2
3
1 2 3 4
1 0 1 1 1
2 1 0 1 0
3 1 1 0 1
4 1 0 1 0
Adjacency matrix
Graph Algorithms
1
2
3
4
2
1
1
1
3 4
3
2 4
3
Adjacency list
4
Adjacency list
Advantages:
–
–
Representation takes memory O(m+n) - versus O(n2)
Examining all neighbors of a given node requires time
O(m/n) in average - versus O(n)
Disadvantages:
–
Checking if an edge belongs to the graph requires time
O(m/n) in average - versus O(1)
Advantages especially for sparse graphs!
1
4
2
3
1 2 3 4
1 0 1 1 1
2 1 0 1 0
3 1 1 0 1
4 1 0 1 0
Adjacency matrix
Graph Algorithms
1
2
3
4
2
1
1
1
3 4
3
2 4
3
Adjacency list
5
Breadth-First Search (BFS)
Given graph G = (V,E) of diameter D and a root node
Goal: find a spanning tree such that the distances between nodes
and the root are the same as in graph G
Idea of the algorithm:
• For layers i = 0,1,…,D
– while there is a node in layer i + 1 not added to the partial BFS tree :
add the node and an edge connecting it with a single node in layer i
root
layer 0
layer 2
layer 1
Graph Algorithms
6
Implementing BFS
Structures:
– Adjacency list
– Lists L0,L1,…,LD
– Array Discovered[1…n]
Algorithm:
• Set L0 = {root}
• For layers i = 0,1,…,D
– Initialize empty list Li+1
– For each node v in Li take next edge adjacent to v and if its second end w is not
marked as Discovered then add w to Li+1 and {v,w} to partial BFS
root
layer 0
layer 2
layer 1
Graph Algorithms
7
Analysis of BFS
Correctness: (observing layer property)
By induction on layer number: each node in layer i is in the list Li
Each node w in layer i has its predecessor on the distance path in layer i-1,
consider the first of such predecessors v in list Li-1: w is added to list Li by
the step where it is considered as the neighbor of v
Complexity:
• Time: O(m+n) - each edge is considered at most twice - since it occurs
twice on the adjacency list
• Memory: O(m+n) - adjacency list takes O(m+n), lists L0,L1,…,LD have at
most n elements in total, array Discovered has n elements
root
layer 0
layer 2
layer 1
Graph Algorithms
8
Depth-First Search (DFS)
Given graph G = (V,E) of diameter D and a root node
Goal: find a spanning tree such that each edge in graph G corresponds to the
ancestor relation in the tree (i.e., no cross edges between different branches)
Recursive idea of the algorithm:
Repeat until no neighbor of the root is free
• Select a free neighbor v of the root and add the edge from root to v to partial
DFS tree
• Recursively find a DFS* in graph G restricted to free nodes and node v as the
root* and add it to partial DFS tree
root*
root
Graph Algorithms
root**
9
Implementing DFS
Data structures:
– Adjacency list
root
with pointer .next
– List (stack) S
– Array Discovered[1…n]
root*
Algorithm:
• Set S = {root}
• Repeat until S is empty
Consider the top element v in S
root**
– if v.next is not Discovered then put v.next into the stack S and set v := v.next and
v.next to be the first neighbour of v
– if v.next is Discovered then
set v.next to be the next neighbour of v if such neighbour exists
otherwise remove v from the stack, add edge {v,z} to partial DFS, where z is the
current top element in S
Remark: after considering the neighbor of node v we remove this neighbor from adjacency
list to avoid considering it many times!
Graph Algorithms
10
Analysis of DFS
Correctness: (observing ancestor property)
By induction on the number of recursive calls:
Consider edge {v,w} in G : suppose v joints DFS before w. Then w is free
and since it is in a connected component with v it will be in DFS* with
root* (which is v), so v is the ancestor of w
Complexity:
• Time: O(m+n) - each edge is considered at most twice - while adding or
removing from the stack
• Memory: O(m+n) - adjacency list consumes O(m+n), stack S and array
Discovered have at most n elements each
root
root*
Graph Algorithms
root**
11
Textbook and Exercises
READING:
• Chapter 3 “Graphs”, Sections 3.2 and 3.3
OBLIGATORY EXERCISES:
• Prove that obtained DFS and BFS trees are spanning
trees.
• Prove that if DFS and BFS trees in graph G are the
same, then G is also the same like they (does not
contain any other edge).
• Prove that if G has n nodes, where n is even, and each
node has at least n/2 neighbours then G is connected.
Graph Algorithms
12
Testing graph properties
• Testing bipartiteness
• Testing for directed cycles
Graph Algorithms
13
How to represent graphs?
Given an undirected graph G = (V,E) , how to represent it?
1. Adjacency matrix: ith node is represented as ith row
and ith column, edge between ith and jth nodes is
represented as 1 in row i and column j, and vice versa
(0 if there is no edge between i and j)
2. Adjacency list: nodes are arranged as array/list, each
node record has the list of its neighbours attached
1
4
2
3
1 2 3 4
1 0 1 1 1
2 1 0 1 0
3 1 1 0 1
4 1 0 1 0
Adjacency matrix
Graph Algorithms
1
2
3
4
2
1
1
1
3 4
3
2 4
3
Adjacency list
14
Adjacency list
Advantages:
–
–
Representation takes memory O(m+n) - versus O(n2)
Examining all neighbours of a given node requires time
O(m/n) in average - versus O(n)
Disadvantages:
–
Checking if an edge belongs to the graph requires time
O(m/n) in average - versus O(1)
Advantages especially for sparse graphs!
1
4
2
3
1 2 3 4
1 0 1 1 1
2 1 0 1 0
3 1 1 0 1
4 1 0 1 0
Adjacency matrix
Graph Algorithms
1
2
3
4
2
1
1
1
3 4
3
2 4
3
Adjacency list
15
Bipartiteness
Graph G = (V,E) is bipartite iff it can be partitioned into two sets of
nodes A and B such that each edge has one end in A and the other
end in B
Alternatively:
• Graph G = (V,E) is bipartite iff all its cycles have even length
• Graph G = (V,E) is bipartite iff nodes can be coloured using two
colours
Question: given a graph represented as an adjacency list, how to test if
the graph is bipartite?
Note: graphs without cycles (trees) are bipartite
bipartite:
non bipartite
Graph Algorithms
16
Testing bipartiteness
Method: use BFS search tree
Recall: BFS is a rooted spanning tree having the same layers as the original graph G
(each node has the same distance from the root in BFS tree and in graph G)
Algorithm:
• Run BFS search and colour all nodes in odd layers red, others blue
• Go through all edges in adjacency list and check if each of them has two different
colours at its ends - if so then G is bipartite, otherwise it is not
We use the following alternative definitions in the analysis:
• Graph G = (V,E) is bipartite iff all its cycles have even length, or
• Graph G = (V,E) is bipartite iff it has no odd cycle
bipartite
non bipartite
Graph Algorithms
17
Testing bipartiteness - why it works
Property of layers:
•
Every edge is either between two consecutive layers or in a single layer (two
ends are in the same layer) - it follows from BFS property
1.
Suppose that graph G is not bipartite. Hence there is an odd cycle. This cycle
must have an edge in one layer, and so the ends of this edge have the same
colour. Thus the algorithm answers not bipartite correctly.
2.
Suppose that graph G is bipartite. Hence all its cycles have even length.
Suppose, to the contrary, that the algorithm answers incorrectly that G is not
bipartite. Hence it found a monochromatic edge. This edge must be in one
layer. Consider one of such edges which is in the smallest possible layer.
Consider a cycle containing this edge and the BFS path between its two ends.
The length of this cycle is odd, thus G is not bipartite. Contradiction!
bipartite
not bipartite
Graph Algorithms
18
Complexity
Time: O(m+n)
• BFS search: O(m+n)
• Checking colours of all edges: O(m+n)
Memory: O(m+n)
• Adjacency list with colour labels: O(m+n)
• Array Active and lists Li for BFS algorithm: O(n)
Graph Algorithms
19
Testing for directed cycles
Directed graph G (edges have direction - one end is the beginning,
the other one is the end of the edge)
Reversed graph Gr has the same nodes but each edge is a reversed
edge from graph G
Representing directed graph: adjacency list - each node has two
lists: of in-coming and out-coming edges/neighbours
Problem: does the graph have a directed cycle?
Graph Algorithms
20
How to represent directed graphs?
Given a directed graph G = (V,E) , how to represent it?
1. Adjacency matrix: ith node is represented as ith
row and ith column, edge from ith to jth node is
represented as 1 in row i and column j, (0 if there is
no directed edge from i to j)
2. Adjacency list: nodes are arranged as array/list,
each node record has two lists: one stores
in-neighbours, the other stores out-neighbours
1
4
2
3
1 2 3 4
4 1 2 3 4
1 0 1 1 1
1 2 3
2 0 0 1 0
4 2 1 3
3 0 0 0 0
1 4 1 3
4 1 0 1 0
Adjacency matrix
Adjacency list
Graph Algorithms
21
Testing cycles
Technique: use directed DFS tree for graph G
Algorithm:
• Find a node with no incoming edges - if not found answer cycled
• Build a directed DFS tree, with the following modification:
– consider only edges in proper direction;
– if during building a DFS tree two nodes which are already in the stack are
considered then answer cycled; otherwise answer acyclic at the end
General remark: if graph G is not connected then do the same until
no free node remains
Graph Algorithms
22
Testing cycles - analysis
Property of acyclic graph (to be proved later):
• There is a node with no incoming edges
Analysis of algorithm correctness:
1. Suppose that graph G is acyclic. Hence there is no directed cycle and then
while building a directed DFS tree we never try to explore the already visited
node. Answer acyclic is then proper.
2. Suppose that graph G is not acyclic. It follows that there is a cycle in it. Let v
be the first node in a cycle visited by DFS search. By the property of DFS, all
nodes from this cycle will be reached during the search rooted in v, and so the
in-neighbour of v from this cycle (or even other node before) will attempt to
visit v, which causes that the algorithm stops with the correct answer cycled.
Graph Algorithms
23
Complexity
Time: O(m+n)
• Finding a node with no incoming edges: O(m+n)
• DFS search with checking cycle condition: O(m+n)
Memory: O(m+n)
• Adjacency list: O(m+n)
• Array Active and stack S for DFS algorithm: O(n)
Graph Algorithms
24
Textbook and Exercises
READING:
• Chapter 3, Sections 3.4 and 3.5
OBLIGATORY EXERCISES:
• Prove the following property of layers: every edge is either between two
consecutive layers or in a single layer (two ends are in the same layer).
• Prove the following property of acyclic graphs: there is a node with no
incoming edges. Is this property true for out-going edges?
OTHER EXERCISES:
• Design and analyse time and memory taken by BFS for directed graphs.
• What happens if you try to use a different method of searching to check
bipartiteness or acyclicity?
Graph Algorithms
25
Ordering nodes in acyclic graphs
(topological order)
Graph Algorithms
26
Directed Acyclic Graphs (DAG)
Directed graph G (edges have directions - one end is the beginning,
the other one is the end of the edge)
Reversed graph Gr has the same nodes but each edge is a reversed
edge from graph G
Test if a given direct graph is acyclic (DAG): done already,
directed DFS approach in time and memory O(m+n)
Problem: is it possible to order all nodes topologically, which
means that if (v,w) is a directed edge in G then v is before w?
Note: there may be many topological orders
graph G
Graph Algorithms
reversed graph Gr
27
Opposite property
Fact:
If graph G has a topological order then it is DAG.
Proof:
Suppose, to the contrary, that
G has a topological order and it also has a cycle.
Let v be the smallest element in the cycle according to the
topological order. It follows that its predecessor in the cycle must
be smaller (in the final order) than v, but this contradicts the fact
that v is the smallest element on the cycle.
We get a contradiction with our assumption, which means that G
must not have a cycle.
Graph Algorithms
28
Key property
Property:
• In DAG there is a node with no incoming edges
• After removing this node with the incident edges, the remaining graph is DAG
Proof:
• Node with no incoming edges:
Suppose, to the contrary, that every node has at least one incoming edge.
It follows that starting from any node we can always “leave” a node by incoming
edge (in opposite direction) until we return to a visited node, which creates a
cycle. This is a contradiction.
• After removing node still a DAG remains:
There could not be a cycle in smaller graph, since it would be in the original
graph.
Graph Algorithms
29
Algorithm finding topological order
Algorithm:
• Repeat until all nodes are in the ordered list
1. Find a node with no incoming edges and place the node next in the order
2. Remove this node and all its outgoing edges from the graph
Efficient implementation:
• Preprocessing:
– for each node keep info if it is active or not (additional array Active of size n needed) - at the
beginning all nodes are active
– go through the list and for every node store the number of incoming edges from other active nodes
(additional array Counter of size n needed)
– select from list S those nodes that have no incoming edges
•
During the algorithm:
– While removing the node (make inactive), go through its outgoing active neighbours and decrease
their counters by one - if some becomes zero put this active node into list S
– Node with no incoming edges is taken (and then removed) from list S
3
2
1
3
2
1 Graph Algorithms
4
2
1
1
30
Complexity
Time: O(m+n)
• Preprocessing:
– Initialization of array Active: O(n)
– Filling array Counter - counting edges: O(m+n)
– Initialization of list S - checking array Counter: O(n)
• During the algorithm:
– While removing a node - go through its out-neighbours: total O(m+n)
– Selecting from list S: total O(n)
Memory: O(m+n)
• Adjacency list: O(m+n)
• Arrays Active and Counter, list S: O(n)
Graph Algorithms
31
Conclusions
Graph algorithms in time O(m + n)
• Searching in graphs
– BFS
– DFS
• Testing graph properties
– bipartiteness (based on BFS)
– if a directed graph is acyclic (based on DFS)
• Topological order
Graph Algorithms
32
Textbook and Exercises
READING:
• Chapter 3, Section 3.6
EXERCISES: (reminder)
• Prove that if DFS and BFS trees in graph G are the same, then G is also the
same like they (does not contain any other edge).
• Prove that if G has n nodes, where n is even, and each node has at least n/2
neighbours then G is connected.
• Prove the following property of layers: every edge is either between two
consecutive layers or in a single layer (two ends are in the same layer).
• Prove the following property of acyclic graphs: there is a node with no
incoming edges. Is this property true for out-going edges?
• Design and analyze time and memory taken by BFS for directed graphs.
• What happens if you try to use a different method of searching to check
bipartiteness or acyclicity?
Graph Algorithms
33