Cost Based Satisficing Search
Considered Harmful
William Cushing
J. Benton
Subbarao Kambhampati
Performance Bug: ε-Cost `Trap’
 High cost variance: ε = $0.01 / $100.00




Board/Fly
Load/Drive
Labor/Precious Material
Mode Switch/Machine Operation
 Search depth:
 0-1(heuristic-error)=∞
 ε-1(heuristic-error)=huge




Optimal: cost=$1000.00, size=100,000
Runner-up: cost=$1000.10, size=20
Trillions of nodes expanded:
When does depth 20 get exhausted?
Outline
 Inevitability of e-cost Traps
 Cycle Trap
 Branching Trap
 Travel Domain
 If Cost is Bad, then what?
 Surrogate Search
 Simple First: Size
 Then: Cost-Sensitive Size-Based Search
Cycle Trap
 Effective search graph
1
1




1
1
g’ = f = g + h
Edge weights = changes in f
0 = ideal
- = over-estimated earlier
 Or under-estimating now
 + = under-estimated earlier
 Or over-estimating now
 Simple subgraph
 Heuristic plateau
 1 choice: Which way?
Cycle Trap
 Even providing a heuristic
perfect for all but 1 edge…
 Cost-based search fails
2
0
2
 Reversible operators are one way in
which heuristic penalty can end up
being bounded from above
 “Unbounded f along unbounded
paths”, to have completeness, also
forces a heuristic upper bound
 Fantastically over-estimating
(weighting) could help, but:
(2 )  1
 Suppose the right edge actually costs
1–ε
 Then both directions would have
identical heuristic value
 Weighting would be fruitless
Branching Trap
x = # of 1 cost children
y = # of ε cost children
d/2 + dε/2 = C
d = 2C/(1+ε)
x+y1/ε = ways to spend 1
(x+y1/ε)C = ways to spend C
(x+y)d = # of paths at same depth
(x+y)2C/(1+ε) << (x+y1/ε)C
Travel
1
2
R
A
B




Straight Fly = 10,000 cents
Diag. Fly = 7,000 cents
Board/Debark = 1 cent
Various Solutions:
 Cheapest Plan
 Fastest Plan
 Smallest Plan
Travel – Cheapest Plan
1
2
R
A
B
Travel – Cheapest Plan
1
2
R
A
B
Travel – Decent Start
1
2
R
A
g = 1 fly
+ 4 board
+ 1 debark
h = 2 fly
+ 4 debark
+ 1 board
B
f ~ 3 fly
Travel – Begin Backtracking
1
2
h = 2 fly
+ 4 debark
+ 1 board
R
A
g = 2 fly
+ 4 board
+ 1 debark
B
f ~ 4 fly
Travel – Backtracking
1
2
R
A
B
g = 1 fly
+ 4 board
+ 2 debark
h = 2 fly
+ 4 debark
+ 2 board
Travel – Backtracking
1
2
R
A
B
g = 1 fly
+ 4 board
+ 2 debark
h = 2 fly
+ 4 debark
+ 2 board
Travel – Backtracking
1
2
R
A
g = 1 fly
+ 4 board
+ 3 debark
h = 2 fly
+ 3 debark
+ 2 board
B
Fly 1-2-B
Then teleport passengers
Travel – Backtracking
1
2
g = 1 fly
+ 6 board
+ 3 debark
h = 2 fly
+ 4 debark
+ 1 board
R
A
B
 8 people: 3 2 = 1296
4
4
 1, 256, 6561, 390625
 (1+0)8, (1+1)8, (1+2)38, (1+4)8
Travel Calculations
 4 planes located in 5 cities
 54 = 625 plane assignments
 4k passengers, located in 9 places
 94k passenger assignments globally
 Cheap subspace
 Product over each city
 (1 + city-local planes) (city-local passengers)
 e.g., (1+2)4(1+1)4 = 1296
 Stop exploring
 Large evaluation
 Exhaustion of possibilities
 Cost-based search exhausts cheap subspaces
 Eventually
 Assuming an upper bound on the heuristic
Outline
 Inevitability of e-cost Traps
 Cycle Trap
 Branching Trap
 Travel Domain
 If Cost is Bad, then what?
 Surrogate Search
 Simple First: Size
 Then: Cost-Sensitive Size-Based Search
Surrogate Search
 Replace ill-behaved Objective with a well-behaved Evaluation
 Tradeoff: Trap Defense versus Quality Focus
 Evaluation Function: “Go no further”
 Force ε ~ 1
 Make g and f grow fast enough: in o(size)
 Normalize costs for hybrid methods
 Heuristic: “Go this way”
 Calculate h in the same units as g
 Retain true Objective
 branch-and-bound
 duplicates elimination + re-expansion
 Re-expansion of duplicates should be done carefully
 Can wait till future iterations, cache heuristics, use path-max, …
Size-based Search
 Replace ill-behaved Objective with a well-behaved Evaluation
 Pure Size
 Evaluation Function: “Go no further”
 Force ε = 1
 Heuristic: “Go this way”
 Replace cost metric with size metric in relaxed problem
 Retain true Objective, for pruning
 Resolve heuristic with real objective
 branch-and-bound: gcost+hcost >= best-known-cost
 duplicates: new.gcost >= old.gcost
 Re-expand better cost paths discovered
Cost-sensitive Size-Based Heuristic
 Replace ill-behaved Objective with a well-behaved Evaluation
 Evaluation Function: “Go no further”
 Heuristic: “Go this way”
 Estimate cheapest/best, but, calculate size
 sum/max/… propagation of real objective for heuristic
 make minimization choices with respect to real objective
 Last minute change:
 Recalculate value of minimization choices by surrogate
 Retain true Objective, for pruning
 Calculate relaxed solution’s cost, also
 Faster than totally resolving heuristic
 branch-and-bound: gcost+hcost >= best-known-cost
 If heuristic is inadmissible, force it to be admissible eventually
Results – LAMA
 LAMA
 Greedy best-first: bad plans
 (iterative) WA*: no plan, time out
 LAMA-size
 Greedy best-first: same bad plans
 (iterative) WA*: direct plans, time out
 Better cost! … but no rendezvous
Expected Result:
Only one kind of object
Costs not widely varying
Portfolio approach possible
Results – SapaReplan
 WA*-cost
 Weight 5: one bad plan, time out
 Weight 2: no plan, memory out
 WA*-size
 Weight 1-2: better plans, memory out
Quality-sensitive evaluation function: cost+size
Conclusion
 ε-cost traps are inevitable





Typical:
Large variation in cost
Large cheap subspaces
Upper-bounded heuristics
Large plateaus in objective
 Cost-based systematic
approaches are susceptible
 Even with all kinds of search
enhancements: LAMA
 Because search depth is
“unbounded” by cost-based
evaluation function
 ε-1(h-error) ~ 0-1(h-error)
 That is, search depth is bounded
only by duplicate checking
 Force good behavior:
 Evaluation ≠ Objective
 Force ε~1
 Quality Focus versus Trap Defense
 Simplest surrogate:
 Size-based Search
 Force ε=1
 Performs surprisingly well
 Despite total lack of Quality Focus
 Easy variation:
 Cost-sensitive Size-based
Heuristic
 Still force ε=1
 Recalculate heuristic by surrogate
 Performs yet better
Conclusion (Polemic)
 Lessons best learnt and then forgotten:
 goto is how computers
work efficiently
Go enthusiasts: joseki
 A* is how search works efficiently
 Both are indispensible
 Both are best-possible
 In just the right context
 Both are fragile
 If the context changes
If size doesn’t work…















Speed Everything Up
Reduce All Memory Consumption
Improve anytime approach: Iterated, Portfolio, Multi-Queue
Guess (search over) upper bounds
Decrease weights
Delay duplicate detection
Delay re-expansion
Delay heuristic computation
Exploit external memory
Use symbolic methods
Learn better heuristics: from search, from inference
Precompute/Memoize anything slow: the heuristic
Impose hierarchy (state/task abstraction)
Accept knowledge (LTL)
Use more hardware: (multi-)core/processor/computer, GPU
Related Work: The Best Approach?
 The Best Surrogate? The Best Approach Over All?




Improve Exploitation




(Dynamic) Heuristic Weighting (Pohl, Thayer+Ruml)
Real-time A* (Korf)
Beam search (Zhou)
Quality-sensitive probing/lookahead (Benton et al, PROBE)





Path-max, A** (Dechter+Pearl)
Multi-queue approaches (Thayer+Ruml, Richter+Westphal, Helmert)
Iterated search (Richter+Westphal)
Portfolio methods (Rintanen, Streeter)
Breadth-first search [as a serious contender] (Edelkamp)




h_cea, h_ff, h_lama, h_vhpop, h_lpg, h_crikey, h_sapa, …
Pattern Databases (Culbertson+Schaeffer, Edelkamp)
Limited Discrepancy Search (Ginsberg)
Negative Result: “How Good is Almost Perfect?” (Helmert+Röger)



Factored Planning (Brafman+Domshalak)
Direct Symmetry Reductions (Korf, Long+Fox)
Symbolic Methods, Indirect Symmetry Reduction (Edelkamp)
Improve Exploration
Directly Address Heuristic Error
`See’ the Structure (remove the traps)
Related Fields
 Reinforcement Learning: Exploration/Exploitation
 Markov Decision Processes: Off-policy/On-policy
 Reward Shaping, Potential Field Methods (Path-search)
 Prioritized Value Iteration
 Decision Theory: Heuristic Errors
 “Decision-Theoretic Search” (?)
 k-armed Bandit Problems (UCB)
 Game-tree Search: Traps, Huge Spaces




Without traps, game-tree pathology (Pearl)
Upper Confidence Bounds on Trees (UCT)
Quiescent Search
Proof-number search (Allis?)
 Machine Learning: Really Huge Spaces
 Surrogate Loss Functions
 Continuous/Differentiable relaxations of 0/1
 Probabilistic Reasoning: Extreme Values are Dangerous
 that 0/1 is bad is well known
 but also ε is numerically unstable
What isn’t closely related?









Typical Puzzles: Rubik’s Cube, Sliding Tiles, …
Prove Optimality/Small Problems
Tightly Bounded Memory: IDDFS, IDA*, SMA*
Unbounded Memory, but:


Delayed/Relaxed Duplicate Detection (Zhou, Korf)
External Memory (Edelkamp, Korf)



D*, D*-Lite, Lifelong Planning A* (Koenig)
Case-based planning
Learned heuristics




Bidirectional/Perimeter Search
Randomly expanding trees for continuous path planning in low dimensions
Waypoint/abstraction methods
Any-angle path planning (Koenig)



Explanation Based Learning
Theorem Proving (Clause/Constraint Learning)
Forward Checking (Unit Propagation)



Subroutine speedup via Precomputation/Memoization
Python vs C
Priority Queue implementation (bucket heaps!)
More than one problem:
State-space isn’t a blackbox:
State-space is far from a blackbox:
Planning isn’t (only) State-space search (Kambhampati)
Engineering:
Quotes
 “… if in some problem instance we were to allow B to skip even
one node that is expanded by A, one could immediately present an
infinite set of instances when B grossly outperforms A. (This is
normally done by appending to the node skipped a variety of trees
with negligible costs and very low h.)”
 Rina Dechter, Judea Pearl
 “I strongly advise that you do not make road movement free (zerocost). This confuses pathfinding algorithms such as A*, …”
 Amit Patel
 “Then we could choose an ĥ somewhat larger than the one defined
by (3). The algorithm would no longer be admissible, but it might
be more desirable, from a heuristic point of view, than any
admissible algorithm.”
 Peter Hart, Nils Nilsson, Bertram Raphael
 Roughly: `… inordinate amount of time selecting among equally
meritorious options’ – Ira Pohl