Dave Reed
heuristics & informed search
 hill-climbing
bold & informed, heuristics, potential dangers, simulated annealing
 best first search
tentative & uninformed, BFS + hill-climbing
 Algorithm A
optimal cost solutions, admissibility, A*
Search strategies so far…
bold & uninformed: STUPID
tentative & uninformed: DFS (and variants), BFS
bold & informed: hill-climbing
 essentially, DFS utilizing a "measure of closeness" to the goal
(a.k.a. a heuristic function)
 from a given state, pick the best successor state
i.e., the state with highest heuristic value
 stop if no successor is better than the current state
EXAMPLE: travel problem (loc(omaha)  loc(los_angeles))
heuristic(State) = -(crow-flies distance from L.A.)
Hill-climbing example
move(loc(omaha), loc(chicago)).
move(loc(omaha), loc(denver)).
move(loc(chicago), loc(denver)).
move(loc(chicago), loc(los_angeles)).
move(loc(chicago), loc(omaha)).
move(loc(denver), loc(los_angeles)).
move(loc(denver), loc(omaha)).
move(loc(los_angeles), loc(chicago)).
move(loc(los_angeles), loc(denver)).
omaha
chicago
hval= -2200
chicago
heuristic(loc(omaha)) = -1700
heuristic(loc(chicago)) = -2200
heuristic(loc(denver)) = -1400
heuristic(loc(los_angeles)) = 0
heuristic value = -1700
denver
hval= -2200
the heuristic guides the
search, always moving
closer to the goal
hval= -1400
los_angeles
hval= 0
what if the flight from Denver to L.A. were cancelled?
8-puzzle example
heuristic(State) = # of tiles in place, including the space
1
2
8
6
7
5
3
start state has 5 tiles in place
4
hval = 5
1
2
1
8
6
3
8
7
5
4
7
hval = 4
1
3
8
2
6
7
5
4
hval = 4
2
5
3
1
2
3
6
8
6
4
4
7
5
hval = 6
1
7
hval = 6
2
3
1
2
3
8
6
8
5
6
5
4
7
hval = 4
of the 3 possible moves, 2
improve the situation
4
hval = 5
if assume left-to-right
preference, move leads to a
dead-end (no successor state
improves the situation) so
STOP!
Intuition behind hill-climbing
if you think of the state space as a topographical map (heuristic value = elevation),
then hill-climbing selects the steepest gradient to move up towards the goal
potential dangers
plateau: successor states have same
values, no way to choose
foothill: local maximum, can get
stuck on minor peak
ridge: foothill where N-step
lookahead might help
Hill-climbing variants
could generalize hill-climbing to continue even if the successor states
look worse
 always choose best successor
 don't stop unless reach the goal or no successors
dead-ends are still possible (and likely if the heuristic is not perfect)
simulated annealing
 allow moves in the wrong direction on a probabilistic basis
 decrease the probability of a backward move as the search continues
idea: early in the search, when far from the goal, heuristic may not be good
heuristic should improve as you get closer to the goal
approach is based on a metallurgical technique
Best first search
hill-climbing is dependent on a near-perfect heuristic since bold
tentative & informed: best first search
 like breadth first search, keep track of all paths searched
 like hill-climbing, use heuristics to guide the search
 always expand the "most promising" path
i.e., the path ending in a state with highest heuristic value
S1
S2
S4
hval = 5
hval = 0
hval = 20
S5
hval = 8
S7
hval = infinity
GOAL
S3
hval =10
S6
hval = 3
Best first example
1
2
8
6
7
5
3
4
hval = 5
1
2
1
8
6
3
8
7
5
4
7
hval = 4
2
5
3
1
2
3
6
8
6
4
4
7
5
hval = 6
1
3
8
2
6
7
5
4
hval = 4
1
7
hval = 6
2
3
1
2
3
1
2
8
6
8
5
6
8
6
5
4
7
4
7
5
hval = 4
hval = 5
3
4
3
8
6
4
5
hval = 7
2
8
7
2
7
hval = 5
1
1
6
3
1
2
3
4
8
6
4
7
5
5
hval = infinity
hval = 4
Breadth first search vs. best first search
procedural description of breadth first search
 must keep track of all current paths, initially: [ [Start] ]
 look at first path in the list of current paths
if the path ends in a goal, then DONE
otherwise,
 remove the first path
 generate all paths by extending to each possible move
 add the newly extended paths to the end of the list
 recurse
procedural description of best first search
 must keep track of all current paths w/ hvals, initially: [ Hval:[Start] ]
 look at first path in the list of current paths
if the path ends in a goal, then DONE
otherwise,
 remove the first path
 generate all paths by extending to each possible move
 add the newly extended paths to the list, sorted by (decreasing) hval
 recurse
Best first implementation
%%% best.pro
Dave Reed
3/15/02
%%% Best first search with cycle checking
%%%
best(Current, Goal, Path): Path is a list of states that
%%%
lead from Current to Goal with no duplicate states.
%%%
%%%
best_help(ListOfPaths, Goal, Path): Path is a list
%%%
of states (with associated hval) that lead from one of
%%%
the paths in ListOfPaths (a list of lists) to Goal with
%%%
no duplicate states.
%%%
%%%
extend(Path, Goal, ListOfPaths): ListOfPaths is the list
%%%
of all possible paths (with associated hvals) obtainable
%%%
by extending Path (at the head) with no duplicate states.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
best(State, Goal, Path) :best_help([0:[State]], Goal, RevPath), reverse(RevPath, Path).
best_help([_:[Goal|Path]|_], Goal, [Goal|Path]).
best_help([_:Path|RestPaths], Goal, SolnPath) :extend(Path, Goal, NewPaths),
append(RestPaths, NewPaths, TotalPaths),
sort(TotalPaths, SortedPaths), reverse(SortedPaths,RevPaths),
best_help(RevPaths, Goal, SolnPath).
extend([State|Path], Goal, NewPaths) :bagof(H:[NextState,State|Path],
(move(State, NextState),
not(member(NextState, [State|Path])),
heuristic(NextState,Goal,H)),
NewPaths), !.
extend(_, _, []).
differences from BFS
associate heuristic
value with each path
H:Path
since extend needs
to know heuristic
values for paths, must
pass Goal to extend
once the new paths
have been appended,
they must be sorted
(in decreasing order
by heuristic value)
Travel example
%%% heuristic(Loc, Goal, Value) : Value is -distance from Goal
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
heuristic(loc(omaha),
heuristic(loc(chicago),
heuristic(loc(denver),
heuristic(loc(los_angeles),
loc(los_angeles), -1700).
loc(los_angeles), -2200).
loc(los_angeles), -1400).
loc(los_angeles),
0).
must define the
heuristic function for
this problem
?- best(loc(omaha), loc(los_angeles), Path).
Path = [loc(omaha), loc(denver), loc(los_angeles)]
Yes
best first search is
able to find a path
8-puzzle example
%%% heuristic(Board, Goal, Value) : Value is the # of tiles in place
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
heuristic(tiles(Row1,Row2,Row3),tiles(Goal1,Goal2,Goal3), Value) :same_count(Row1, Goal1, V1),
same_count(Row2, Goal2, V2),
same_count(Row3, Goal3, V3),
Value is V1 + V2 + V3.
must define the
heuristic function for
this problem
same_count([], [], 0).
same_count([H1|T1], [H2|T2], Count) :same_count(T1, T2, TCount),
(H1 = H2, Count is TCount + 1 ; H1 \= H2, Count is TCount).
?- best(tiles([1,2,3],[8,6,space],[7,5,4]),
tiles([1,2,3],[8,space,4],[7,6,5]), Path).
Path = [tiles([1, 2, 3], [8, 6, space], [7, 5, 4]), tiles([1, 2, 3],
[8, 6, 4], [7, 5, space]), tiles([1, 2, 3], [8, 6, 4], [7, space,
5]), tiles([1, 2, 3], [8, space, 4], [7, 6, 5])]
Yes
?- best(tiles([2,3,4],[1,8,space],[7,6,5]),
tiles([1,2,3],[8,space,4],[7,6,5]), Path).
Path = [tiles([2, 3, 4], [1, 8, space], [7, 6, 5]), tiles([2, 3,
space], [1, 8, 4], [7, 6, 5]), tiles([2, space, 3], [1, 8, 4], [7,
6, 5]), tiles([space, 2, 3], [1, 8, 4], [7, 6, 5]), tiles([1, 2, 3],
[space, 8, 4], [7, 6, 5]), tiles([1, 2|...], [8, space|...], [7,
6|...])]
Yes
best first search is
able to find a path
Other examples
heuristic function for the Water Jug Problem?
move(jugs(J1,J2),
move(jugs(J1,J2),
move(jugs(J1,J2),
move(jugs(J1,J2),
move(jugs(J1,J2),
move(jugs(J1,J2),
move(jugs(J1,J2),
move(jugs(J1,J2),
jugs(4,J2))
jugs(J1,3))
jugs(0,J2))
jugs(J1,0))
jugs(4,N))
jugs(N,3))
jugs(N,0))
jugs(0,N))
::::::::-
J1
J2
J1
J2
J2
J1
J2
J1
<
<
>
>
>
>
>
>
4.
3.
0.
0.
0,
0,
0,
0,
J1
J1
J1
J1
+
+
+
+
J2
J2
J2
J2
>= 4, N is J2 - (4 - J1).
>= 3, N is J1 - (3 - J2).
< 4, N is J1 + J2.
< 3, N is J1 + J2.
heuristic function for Missionaries & Cannibals?
Comparing best first search
unlike hill-climbing, best first search can handle "dips" in the search
 not so dependent on a perfect heuristic
depth first search and breadth first search may be seen as special
cases of best first search
DFS: heuristic value is distance (number of moves) from start state
BFS: heuristic value is inverse of distance (number of moves) from start state
or, procedurally:
DFS: assign all states the same heuristic value
when adding new paths, add equal heuristic values at the front
BFS: assign all states the same heuristic value
when adding new paths, add equal heuristic values at the end
Optimizing costs
consider a related problem:
 instead of finding the shortest path (i.e., fewest moves) to a solution,
suppose we want to minimize some cost
EXAMPLE: airline travel problem
• could associate costs with each flight, try to find the cheapest route
• could associate distances with each flight, try to find the shortest route
we could use a strategy similar to breadth first search
 repeatedly extend the minimal cost path
search is guided by the cost of the path so far
but such a strategy ignores heuristic information
 would like to utilize a best first approach, but not directly applicable
 search is guided by the remaining cost of the path
IDEAL: combine the intelligence of both strategies
 cost-so-far component of breadth first search (to optimize actual cost)
 cost-remaining component of best first search (to make use of heuristics)
Travel problem revisited
%%% travelcost.pro
Dave Reed
3/15/02
%%%
%%% This file contains the state space definition
%%% for air travel, with costs (distances) associated
%%% with each flight and heuristics providing
%%% as-the-crow-flies distance.
%%%
%%% loc(City): defines the state where the person
%%%
is in the specified City.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
move(loc(omaha),
move(loc(omaha),
move(loc(chicago),
move(loc(chicago),
move(loc(chicago),
move(loc(denver),
move(loc(denver),
move(loc(los_angeles),
move(loc(los_angeles),
loc(chicago),
loc(denver),
loc(denver),
loc(los_angeles),
loc(omaha),
loc(los_angeles),
loc(omaha),
loc(chicago),
loc(denver),
heuristic(loc(omaha),
heuristic(loc(chicago),
heuristic(loc(denver),
heuristic(loc(los_angeles),
500).
600).
1000).
2200).
500).
1400).
600).
2200).
1400).
add the actual cost (number of
miles per flight) to each move
heuristic is crow-flies distance
(note: no longer negative, since
want estimate of remaining cost)
want to minimize the total flight
distance from Omaha to L.A.
loc(los_angeles), 1700).
loc(los_angeles), 2200).
loc(los_angeles), 1400).
loc(los_angeles),
0).
chicago
1000
500
denver
omaha
1400
600
2200
los_angeles
Algorithm A
associate 2 costs with a path
g
h
f=g+h
actual cost of the path so far
heuristic estimate of the remaining cost to the goal
combined heuristic cost estimate
Algorithm A (for trees – note that usual formulation is for graphs)
 best first search using f as the heuristic
S1
h = 24
20
10
S2
h = 19
f = 29
S3
2
1
h=5
f = 27
S4
h=4
f = 24
S5
6
S7
h=5
f = 26
10
h = 0 GOAL
f = 28
S6
h=2
f = 33
Algorithm A implementation
%%% Algorithm A (for trees)
%%%
atree(Current, Goal, Path): Path is a list of states
%%%
that lead from Current to Goal with no duplicate states.
%%%
%%%
atree_help(ListOfPaths, Goal, Path): Path is a list of
%%%
states (with associated F and G values) that lead from one
%%%
of the paths in ListOfPaths (a list of lists) to Goal with
%%%
no duplicate states.
%%%
%%%
extend(G:Path, Goal, ListOfPaths): ListOfPaths is the list
%%%
of all possible paths (with associated F and G values) obtainable
%%%
by extending Path (at the head) with no duplicate states.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
differences from best
associate two values
with each path (F is
total estimated cost,
G is actual cost so far)
F:G:Path
atree(State, Goal, G:Path) :atree_help([0:0:[State]], Goal, G:RevPath), reverse(RevPath, Path).
atree_help([_:G:[Goal|Path]|_], Goal, G:[Goal|Path]).
atree_help([_:G:Path|RestPaths], Goal, SolnPath) :extend(G:Path, Goal, NewPaths),
append(RestPaths, NewPaths, TotalPaths),
sort(TotalPaths, SortedPaths),
atree_help(SortedPaths, Goal, SolnPath).
extend(G:[State|Path], Goal, NewPaths) :bagof(NewF:NewG:[NextState,State|Path],
Cost^H^(move(State, NextState, Cost),
not(member(NextState, [State|Path])),
heuristic(NextState,Goal,H),
NewG is G+Cost, NewF is NewG+H),
NewPaths), !.
extend(_, _, []).
since extend needs to
know of current path,
must pass G
new feature of bagof:
if a variable appears
only in the 2nd arg,
must identify it as
backtrackable
Travel example
can use semi-colon to see alternative paths in order
?- atree(loc(omaha), loc(los_angeles), Path).
Path = 2000:[loc(omaha), loc(denver), loc(los_angeles)] ;
Path = 2700:[loc(omaha), loc(chicago), loc(los_angeles)] ;
Path = 2900:[loc(omaha), loc(chicago), loc(denver), loc(los_angeles)] ;
No
note: Algorithm A finds the path with least cost (here, distance)
not necessarily the path with fewest steps
 suppose the flight from Chicago to L.A. was 2500 miles (instead of 2200)
 then OmahaChicagoDenverL.A. is shorter than OmahaChicagoL.A.
8-puzzle example
could apply Alg. A to the 8-puzzle problem


actual cost of each move (g) is 1
remaining cost estimate (h) is # of tiles out of place
?- atree(tiles([1,2,3],[8,6,space],[7,5,4]),
tiles([1,2,3],[8,space,4],[7,6,5]), Path).
Path = 3:[tiles([1, 2, 3], [8, 6, space], [7, 5, 4]), tiles([1, 2, 3],
[8, 6, 4], [7, 5, space]), tiles([1, 2, 3], [8, 6, 4], [7, space, 5]),
tiles([1, 2, 3], [8, space, 4], [7, 6, 5])]
Yes
?- atree(tiles([2,3,4],[1,8,space],[7,6,5]),
tiles([1,2,3],[8,space,4],[7,6,5]), Path).
Path = 5:[tiles([2, 3, 4], [1, 8, space], [7, 6, 5]), tiles([2, 3,
space], [1, 8, 4], [7, 6, 5]), tiles([2, space, 3], [1, 8, 4], [7, 6,
5]), tiles([space, 2, 3], [1, 8, 4], [7, 6, 5]), tiles([1, 2|...],
[space, 8|...], [7, 6|...]), tiles([1|...], [8|...], [7|...])]
Yes
here, Algorithm A finds the same paths as best first search
 not surprising since actual cost component (g) is trivial
 still, not guaranteed to be the case (recall: best only cares about the future)
Algorithm A vs. hill-climbing
if the cost estimate function h is perfect, then f is a perfect heuristic
 Algorithm A is deterministic
S1
h = 24
20
10
S2
h = 20
f = 30
20
X
h = 0 GOAL
f = 30
S3
2
h=8
f = 28
1
h=6
f = 28
S4
S5
6
S5
h = 12
f = 33
10
h = 0 GOAL
f = 28
X
h=2
f = 33
2
X
h = 0 GOAL
f = 33
if know actual costs for each state, Alg. A reduces to hill-climbing
Admissibility
in general, actual costs are unknown at start – must rely on heuristics
if the heuristic is imperfect, Alg. A is NOT guaranteed to find an
optimal solution
S1
1
h=4
f=5
S2
h=2
f=2
3
1
X
h = 0 GOAL
f=2
h=1
f=4
S3
1
S4
h = 0 GOAL
f=4
if a control strategy is guaranteed to find an optimal solution (when a
solution exists), we say it is admissible
if cost estimate h is never overestimates actual cost, then Alg. A is admissible
(when admissible, Alg. A is commonly referred to as Alg. A*)
Admissible examples
is our heuristic for the travel problem admissible?
h (State) = crow-flies distance from L.A.
is our heuristic for the 8-puzzle admissible?
h (State) = number of tiles out of place, including the space
is our heuristic for the Missionaries & Cannibals admissible?
Cost of the search
the closer h is to the actual cost, the fewer states considered
 however, the cost of computing h tends to go up as it improves
cost of solving problem
computation
cost
cost of computing h
the best algorithm is one
that minimizes the total cost
of the solution
cost of search using h
closeness of h to
actual cost
also, admissibility is not always needed or desired
Graceful Decay of Admissibility:
If h rarely overestimates the actual cost by more than D, then Alg. A will rarely
find a solution whose cost is more than D greater than optimal.