Today – More about Trees
• Spanning trees
– Prim’s algorithm
– Kruskal’s algorithm
Introduction to Computers and
Programming
• Generic search algorithm
Prof. I. K. Lundqvist
– Depth-first search example
– Handling cycles
– Breadth-first search example
Lecture 8
April 5 2004
A
B
A
B
C
D
A
B
B
A
D
E
B
D
C
A
D
C
D
D
A
B
D
D
E
B
D
E
B
C
E
A
B
E
C
D
C
D
E
A B
A 0 1
B 1 0
C 1 0
D 1 1
E 0 1
C D
1 1
E
0
0
0
1
1
1
0
1
0
1
0
1
0
C
D
A B
A 0 1
B 0 0
C 0 0
D 0 0
E 0 1
C D
1 1
E
0
0
0
1
1
0
0
0
1
0
0
0
0
Trees
Spanning Trees
• A tree is a connected graph without
cycles
• A Spanning tree of a graph G, is a tree
that includes all the vertices from G.
• A connected graph is a tree iff it has N
vertices and N-1 edges
Airline Routes
Boston
SFO
The
The resulting
resulting spanning
spanning tree
tree
isis not
unique
not unique
Wash DC
• A graph is a tree iff there is one and
only one path joining any two of its
vertices
LA
Dallas
Boston
SFO
Wash DC
LA
Minimum Spanning Tree
• Prim’s Algorithm
– Finds a subset of the edges (that form a tree)
including every vertex and the total weight of
all the edges in tree is minimized
• Choose starting vertex
Initialization
• Create the Fringe Set
• Loop until the MST contains all the vertices in the
graph
Body
– Remove edge with minimum weight from Fringe Set
– Add the edge to MST
– Update the Fringe Set
Dallas
Prim – Initialization
• Pick any vertex x as the starting vertex
• Place x in the Minimum Spanning Tree
(MST)
• For each vertex y in the graph that is
adjacent to x
– Add y to the Fringe Set
• For each vertex y in the Fringe Set
– Set weight of y to weight of the edge
connecting y to x
– Set x to be parent of y
2534
Prim – Body
While number of vertices in MST < vertices in
the graph
860
ORD
1855
SFO
957
349
722
DEN 908
760
2451
834
BOS
191
JFK
1090
606
ATL
Find vertex y with minimum weight in the Fringe Set
Add vertex and the edge x,y to the MST
Remove y from the Fringe Set
For all vertices z adjacent to y
LAX
595
MIA
If z is not in the Fringe Set
ORD
Add z to the Fringe Set
Set parent to y
Set weight of z to weight of the edge connecting z to y
DEN 908
SFO
Else
349
If Weight(y,z) < Weight(z) then
Set parent to y
Set weight of z to weight of the edge connecting z to y
722
834
606
ATL
LAX
595
Minimum spanning tree – Prim
MIA
2534
Minimum Spanning Tree
– Finds a minimum spanning tree for a
connected weighted graph
• Create a set of trees, where each vertex in the
graph is a separate tree
• Create set S containing all edges in the graph
• While S not empty
– Remove edge with minimum weight from S
– if that edge connects two different trees, then add
it to the forest, combining two trees into a single
tree
– Otherwise discard that edge
860
ORD
1855
SFO
• Kruskal’s Algorithm
BOS
191
JFK
957
349
760
2451
834
191
JFK
722
DEN 908
BOS
1090
606
ATL
LAX
595
MIA
ORD
722
BOS
191
JFK
DEN 908
SFO
349
834
606
ATL
LAX
595
Minimum spanning tree – Kruskal
MIA
More about Trees
Depth First Search (DFS)
Idea:
•Explore descendants before siblings
•Explore siblings left to right
• Spanning trees
– Prim’s algorithm
– Kruskal’s algorithm
1
S
2
A
• Generic search algorithm
3
– Depth-first search example
– Handling cycles
– Breadth-first search example
6
D
4 C 5 G
7
B
11
8
C
D
9 C
G
10
G
Where do we place the children on the queue?
• Assume we pick first element of Q
• Add path extensions to ? of Q
Simple Search Algorithm
Let Q be a list of partial paths,
Let S be the start node and
Let G be the Goal node.
Pick first element of Q; Add path extensions to front of Q
Q
1. Initialize Q with partial path (S)
2. If Q is empty, fail. Else, pick a partial path N from Q
3. If head(N) = G, return N
(goal reached!)
4. Else:
a) Remove N from Q
b) Find all children of head(N) and
create all the one-step extensions of N to each child.
c) Add all extended paths to Q
d) Go to step 2.
Depth-First
1
C
(S)
2
3
4
5
G
A
1
D
S
B
Simple Search Algorithm
Depth-First
Let Q be a list of partial paths,
Let S be the start node and
Let G be the Goal node.
Pick first element of Q; Add path extensions to front of Q
Q
1. Initialize Q with partial path (S)
1
2. If Q is empty, fail. Else, pick a partial path N from Q
2
3. If head(N) = G, return N
3
(goal reached!)
4. Else:
C
(S)
1
D
S
4
a) Remove N from Q
G
A
B
5
b) Find all children of head(N) and
create all the one-step extensions of N to each child.
c) Add all extended paths to Q
d) Go to step 2.
Depth-First
Depth-First
Pick first element of Q; Add path extensions to front of Q
Pick first element of Q; Add path extensions to front of Q
Q
1
2
(S)
4
G
A
(A S)
3
Q
C
1
1
2
D
S
B
C
(S)
(A S) (B S)
3
4
5
5
Added paths in blue
Added paths in blue
G
A
1
D
S
B
Simple Search Algorithm
Depth-First
Let Q be a list of partial paths,
Let S be the start node and
Let G be the Goal node.
Pick first element of Q; Add path extensions to front of Q
Q
1. Initialize Q with partial path (S)
1
2. If Q is empty, fail. Else, pick a partial path N from Q
3. If head(N) = G, return N
2
(goal reached!)
C
(S)
2
(A S) (B S)
1
3
4. Else:
D
S
4
a) Remove N from Q
G
A
B
5
b) Find all children of head(N) and
create all the one-step extensions of N to each child.
Added paths in blue
c) Add all extended paths to Q
d) Go to step 2.
Depth-First
Depth-First
Pick first element of Q; Add path extensions to front of Q
Pick first element of Q; Add path extensions to front of Q
Q
1
2
3
(S)
2
4
G
A
(A S) (B S)
(C A S) (D A S) (B S)
Q
C
1
D
S
B
1
C
(S)
2
(A S) (B S)
3
(C A S) (D A S) (B S)
4
5
5
Added paths in blue
Added paths in blue
2
G
A
1
D
S
B
Depth-First
Depth-First
Pick first element of Q; Add path extensions to front of Q
Pick first element of Q; Add path extensions to front of Q
3
3
Q
1
2
3
Q
C
(S)
2
G
A
(A S) (B S)
1
(C A S) (D A S) (B S)
D
S
4
B
1
C
(S)
2
(A S) (B S)
3
(C A S) (D A S) (B S)
4
(D A S) (B S)
2
G
A
1
D
S
B
5
5
Added paths in blue
Added paths in blue
Depth-First
Depth-First
Pick first element of Q; Add path extensions to front of Q
Pick first element of Q; Add path extensions to front of Q
3
3
Q
1
2
(S)
2
(C A S) (D A S) (B S)
4
(D A S) (B S)
5
Added paths in blue
G
A
(A S) (B S)
3
Q
C
2
4
1
1
D
S
B
C
(S)
2
(A S) (B S)
3
(C A S) (D A S) (B S)
4
(D A S) (B S)
5
(C D A S)(G D A S)
(B S)
G
A
4
1
D
S
B
Simple Search Algorithm
Depth-First
Let Q be a list of partial paths,
Let S be the start node and
Let G be the Goal node.
Pick first element of Q; Add path extensions to front of Q
3
Q
1. Initialize Q with partial path (S)
1
2. If Q is empty, fail. Else, pick a partial path N from Q
3. If head(N) = G, return N
2
(goal reached!)
4. Else:
a) Remove N from Q
b) Find all children of head(N) and
create all the one-step extensions of N to each child.
C
(S)
2
(A S) (B S)
3
(C A S) (D A S) (B S)
4
(D A S) (B S)
5
(C D A S)(G D A S)
(B S)
G
A
4
1
D
S
B
c) Add all extended paths to Q
d) Go to step 2.
Depth-First
Depth-First
Pick first element of Q; Add path extensions to front of Q
Pick first element of Q; Add path extensions to front of Q
3
3
Q
1
2
(S)
2
(C A S) (D A S) (B S)
4
(D A S) (B S)
G
A
(A S) (B S)
3
Q
C
2
4
1
1
D
(S)
(C A S) (D A S) (B S)
4
(D A S) (B S)
5
(C D A S)(G D A S)
(B S)
5
(C D A S)(G D A S)
(B S)
6
(G D A S)(B S)
6
(G D A S)(B S)
B
2
G
A
(A S) (B S)
3
S
C
4
1
D
S
B
More about Trees
Issue: Starting at S and moving top to bottom,
will depth-first search ever reach G?
• Spanning trees
– Prim’s algorithm
– Kruskal’s algorithm
C
G
A
D
• Generic search algorithm
S
– Depth-first search example
– Handling cycles
– Breadth-first search example
B
Depth-First
How Do We Avoid Repeat Visits?
Effort can be wasted in more mild cases
Idea:
3
Q
1
(S)
2
(A S) (B S)
3
(C A S) (D A S) (B S)
4
(D A S) (B S)
5
(C D A S)(G D A S)
(B S)
6
(G D A S)(B S)
• Keep track of nodes already visited.
C
2
G
A
• Do not place visited nodes on Q.
4
1
D
S
B
• C visited multiple times
• Multiple paths to C, D & G
How much wasted effort can be incurred in the worst case?
Does this maintain correctness?
• Any goal reachable from a node that was visited a
second time would be reachable from that node the
first time.
Does it always improve efficiency?
• Guarantees each node appears at most once at
the head of a path in Q.
Simple Search Algorithm
More about Trees
Let Q be a list of partial paths,
Let S be the start node and
Let G be the Goal node.
• Spanning trees
1. Initialize Q with partial path (S) as only entry; set Visited = ( )
– Prim’s algorithm
– Kruskal’s algorithm
2. If Q is empty, fail. Else, pick some partial path N from Q
3. If head(N) = G, return N
(goal reached!)
4. Else
• Generic search algorithm
a) Remove N from Q
– Depth-first search example
– Handling cycles
– Breadth-first search example
b) Find all children of head(N) not in Visited and
create all the one-step extensions of N to each child.
c) Add to Q all the extended paths;
d) Add children of head(N) to Visited
e) Go to step 2.
Breadth-First
Breadth First Search (BFS)
Idea:
•Explore relatives at same level before their children
•Explore relatives left to right
S
A
4
D
8 C 9 G
5
1
Q
Visited
(S)
S
C
2
1
2
Pick first element of Q; Add path extensions to end of Q
3
3
B
7
6
C
D
10
C
G
G
11
Where do we place the children on the queue?
• Assume we pick first element of Q
• Add path extensions to ? of Q
4
5
6
G
A
1
D
S
B
1
Breadth-First
Breadth-First
Pick first element of Q; Add path extensions to end of Q
Pick first element of Q; Add path extensions to end of Q
Q
Visited
(S)
S
2
G
A
3
1
S
2
(A S) (B S)
A,B,S
C
G
A
1
D
S
B
6
Breadth-First
Breadth-First
Pick first element of Q; Add path extensions to end of Q
Pick first element of Q; Add path extensions to end of Q
Q
Visited
1
(S)
S
2
(A S) (B S)
A,B,S
6
(S)
5
6
5
1
4
B
5
4
Visited
3
D
S
4
3
Q
C
Q
Visited
1
(S)
S
2
(A S) (B S)
A,B,S
3
(B S) (C A S) (D A S)
C,D,B,A,S
C
2
G
A
1
D
S
4
B
5
6
C
2
G
A
1
D
S
B
1
Breadth-First
Breadth-First
Pick first element of Q; Add path extensions to end of Q
Pick first element of Q; Add path extensions to end of Q
Q
Visited
(S)
S
C
1
2
2
(A S) (B S)
A,B,S
3
(B S) (C A S) (D A S)
C,D,B,A,S
G
A
1
D
S
4
B
5
3
6
Depth First Search (DFS)
Visited
(S)
S
2
(A S) (B S)
A,B,S
3
(B S) (C A S) (D A S)
C,D,B,A,S
4
(C A S) (D A S) (G B S)*
G,C,D,B,A,S
5
(D A S) (G B S)
G,C,D,B,A,S
6
(G B S)
G,C,D,B,A,S
4
C
6
G
2
A
5
1
D
S
B
3
Summary
S
A
Q
Depth-first:
B
Add path extensions to front of Q
D
C
C
D
G
G
C
Pick first element of Q
G
Breadth First Search (BFS)
S
A
Breadth-first:
B
Add path extensions to back of Q
D
C
C
G
D
C
G
Pick first element of Q
G
Test_ordered_binary.adb
• Most problem solving tasks may be
formulated as state space search.
• Mathematical representations for search
are graphs and search trees.
• Depth-first and breadth-first search
may be framed, among others, as
instances of a generic search strategy.
• Cycle detection is required to achieve
efficiency and completeness.
• Document code
– What it is doing
– How it is doing it
– What it is not doing (detailed status)
• Test run code
• Zip code