Quantified Boolean Formula (QBF) Reasoning
Bart Selman, Carla Gomes, Ashish Sabharwal
Cornell University
Feb 13, 2007
Tutorial for
Dr. Charles Holland and Dr. Tom Wagner
Tutorial Roadmap
1. Automated reasoning
– The complexity challenge
– State of the art in Boolean reasoning
2. SAT-based reasoning
–
–
–
–
Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
•
Example domain: planning
3. QBF reasoning (extends SAT)
– A new range of applications
– Two motivating examples
•
–
–
–
–
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
network planning, logistics planning
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
2
Tutorial Roadmap
1. Automated reasoning
 The complexity challenge
– State of the art in Boolean reasoning
2. SAT-based reasoning
–
–
–
–
Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
•
Example domain: planning
3. QBF reasoning (extends SAT)
– A new range of applications
– Two motivating examples
•
–
–
–
–
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
network planning, logistics planning
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
3
The Quest for Machine Reasoning
Objective:
Develop foundations and technology to enable
effective, practical, large-scale automated reasoning.
Machine Reasoning (1960-90s)
Computational complexity of reasoning
appears to severely limit real-world
applications
Current reasoning technology
Revisiting the challenge:
Significant progress with new
ideas / tools for dealing with
complexity (scale-up),
uncertainty, and multi-agent
reasoning
4
General Automated Reasoning
Problem
instance
Domain-specific
e.g. logistics, war games,
chess, space missions,
planning, scheduling, ...
Model
Generator
(Encoder)
General
Inference
Engine
Solution
Generic
applicable to all domains
within range of modeling language
Research objective
Impact
Better reasoning and
modeling technology
Faster solutions
in several domains
5
Reasoning Complexity
• EXPONENTIAL COMPLEXITY: INHERENT
AN worst case
N= No. of Variables/Objects A= Object states
Simple Example:
Knowledge Base
Variables (binary)
X1 = email_ received
X2 = in_ meeting
X3 = urgent
X4 = respond_to_email
X5 = near_deadline
X6 = postpone
X7 = air_ticket_info_request
X8 = travel_ request
X9 = info_request
Rules:
1. X1 & (not X2) & X3  X4
2. X2  not X4
3.
4.
5.
6.
7.
X5  X3 or X6
X7  X8
X8  X9
X8  X5
X6  not X9
• TIME/SPACE
Granularity   Object states
• Current implementations trade
time with soundness
Question: Given: X1= true; X2 = false; X7=true.
What is X4 = ?
Answer Development: Inference Chain
Step 1: X7  X8 (rule 4)
Step 2: X8  X5 (rule 6)
Step 3: X5  X3 or X6 (rule 3)
Search for rules to apply
M
For N variables: 2N cases drive
complexity!
Case A: X6 = true
Step 4: X6  not X9
Step 5: X9  not X8
Step 6: Contradiction
Backtrack to M
Case B: X3 = true
X1 & (not X2) & X3  X4
Step 7: X4 = true (Rule 1)
Check Contradictions
6
Exponential Complexity Growth:
The Challenge of Complex Domains
Case complexity
Note: rough estimates, for propositional reasoning
1M War Gaming
5M
10301,020
0.5M VLSI
1M Verification
10150,500
100K
450K
106020
Military Logistics
20K Chess (20 steps deep)
100K
No. of atoms
on the earth
103010
Seconds until heat
death of sun
1047
Protein folding
Calculation
(petaflop-year)
10K
50K
Deep space mission control
100 Car repair diagnosis
200
1030
100
10K
20K
100K
1M
Variables
Rules (Constraints)
[Credit: Kumar, DARPA; Cited in Computer World magazine]
7
Tutorial Roadmap
1. Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
2. SAT-based reasoning
–
–
–
–
Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
•
Example domain: planning
3. QBF reasoning (extends SAT)
– A new range of applications
– Two motivating examples
•
–
–
–
–
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
network planning, logistics planning
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
8
Progress in Last 15 Years
Focus: Combinatorial Search Spaces
Specifically, the Boolean satisfiability problem, SAT
Significant progress since the 1990’s.
How much?
•
Problem size: We went from 100 variables, 200 constraints (early
90’s) to 1,000,000 vars. and 5,000,000 constraints in 15 years.
Search space: from 10^15 to 10^300,000.
[Aside: “one can encode quite a bit in 1M variables.”]
•
Tools: 50+ competitive SAT solvers available
Overview of state of the art:
Plenary talk at IJCAI-05 (Selman); Gomes and Selman, Nature ’05
9
How Large are the Problems?
A bounded model checking problem:
10
SAT Encoding
(automatically generated from problem specification)
i.e., ((not x1) or x7)
((not x1) or x6)
etc.
x1, x2, x3, etc. are our Boolean variables
(to be set to True or False)
Should x1 be set to False??
11
10 Pages Later:
…
i.e., (x177 or x169 or x161 or x153 …
x33 or x25 or x17 or x9 or x1 or (not x185))
clauses / constraints are getting more interesting…
Note x1 …
12
4,000 Pages Later:
…
13
Finally, 15,000 Pages Later:
Search space of truth assignments:
Current SAT solvers solve this instance in
under 30 seconds!
14
SAT Solver Progress
Solvers have continually improved over time
Instance
Posit' 94 Grasp' 96 Sato' 98 Chaff' 01
40.66s
1.20s
0.95s
0.02s
bf1355-638
1805.21s
0.11s
0.04s
0.01s
pret150_25
>3000s
0.21s
0.09s
0.01s
dubois100
>3000s
11.85s
0.08s
0.01s
aim200-2_0-no-1
>3000s
0.01s
< 0.01s
< 0.01s
2dlx_..._bug005
>3000s
>3000s
>3000s
2.90s
c6288
>3000s
>3000s
>3000s
>3000s
ssa2670-136
Source: Marques-Silva 2002
15
How do SAT Solvers Keep Improving?
From academically interesting to practically relevant.
We now have regular SAT solver competitions.
(Germany ’89, Dimacs ’93, China ’96, SAT-02, SAT-03, SAT-04, SAT-05, SAT-06)
E.g. at SAT-2006 (Seattle, Aug ’06):
• 35+ solvers submitted, most of them open source
• 500+ industrial benchmarks
•
50,000+ benchmark instances available on the www
This constant improvement in SAT solvers is the key to making, e.g.,
SAT-based planning very successful.
16
Current Automated Reasoning Tools
Most-successful fully automated methods:
based on Boolean Satisfiability (SAT) / Propositional Reasoning
– Problems modeled as rules / constraints over Boolean variables
– “SAT solver” used as the inference engine
Applications: single-agent search
• AI planning
SATPLAN-06, fastest optimal planner;
ICAPS-06 competition (Kautz & Selman ’06)
•
Verification – hardware and software
Major groups at Intel, IBM, Microsoft, and universities
such as CMU, Cornell, and Princeton.
SAT has become the dominant technology.
•
Many other domains: Test pattern generation, Scheduling,
Optimal Control, Protocol Design, Routers, Multi-agent systems,
E-Commerce (E-auctions and electronic trading agents), etc.
17
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
2. SAT-based reasoning

–
–
–
Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
•
Example domain: planning
3. QBF reasoning (extends SAT)
– A new range of applications
– Two motivating examples
•
–
–
–
–
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
network planning, logistics planning
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
18
Boolean Logic
Defined over Boolean (binary) variables a, b, c, …
Each of these can be True (1) or False (0)
Variables connected together with logic operators: and, or, not (denoted )
E.g. (a or b) is True iff at least one of a and b is True
((c and d)) or f) is True iff
either c is True and d is False, or f is True
Fact: All other Boolean logic operators can be expressed with and, or, not
E.g. (a  b) same as (a or b)
Boolean formula, e.g. F = (a or b) and (a and (b or c))
(Truth) Assignment: any setting of the variables to True or False
Satisfying assignment: assignment where the formula evaluates to True
19
Boolean Logic: Example
F = (a or b) and (a and (b or c))
Note: True often written as 1, False as 0
• There are 23 = 8 possible truth assignments to a, b, c
– (a=0,b=1,c=0)
– (a=0,b=0,c=1)
– …
representing (a=False, b=True, c=False)
• Exactly 3 truth assignments satisfy F
– (a=0,b=1,c=0)
– (a=0,b=1,c=1)
– (a=1,b=0,c=0)
Truth Table for F
a
b
c
F
0
0
0
0
0
0
1
0
0
1
0
1
0
1
1
1
1
0
0
1
1
0
1
0
1
1
0
0
1
1
1
0
20
Boolean Logic: Expressivity
All discrete single-agent search problems can be
cast as a Boolean formula!
Variables a, b, c, … often represent “states” of the
system, “events”, “actions”, etc.
(more on this later, using Planning as a example)
Very general encoding language. E.g. can handle
• Numbers (k-bit binary representation)
• Floating-point numbers
• Arithmetic operators like +, x, exp(), log()
• …
SAT encodings (generated automatically from high
level languages) routinely used in domains like
planning, scheduling, verification, e-commerce,
network design, …
Recall Example:
Variables
X1 = email_ received
X2 = in_ meeting
X3 = urgent
X4 = respond_to_email
“event”
“state”
“action”
X5 = near_deadline
X6 = postpone
X7 = air_ticket_info_request
X8 = travel_ request
X9 = info_request
Rules:
1. X1 & (not X2) & X3  X4
2. X2  not X4
constraint
3.
4.
5.
6.
7.
X5  X3 or X6
X7  X8
X8  X9
X8  X5
X6  not X9
21
Boolean Logic: Standard Representations
Each problem constraint typically specified as (a set of) clauses:
E.g. (a or b), (c or d or f), (a or c or d), …
clauses (only “or”, “not”)
Formula in conjunctive normal form, or CNF: a conjunction of clauses
E.g.
F = (a or b) and (a and (b or c)) changes to
FCNF = (a or b) and (a or b) and (b or c)
Alternative [useful for QBF]: specify each constraint as a term (only “and”, “not”):
E.g. (a and d), (b and a and f), (b and d and e), …
Formula in disjunctive normal form, or DNF: a disjunction of terms
E.g. FDNF = (a and b) or (a and b and c)
22
Boolean Satisfiability Testing
The Boolean Satisfiability Problem, or SAT:
Given a Boolean formula F,
• find a satisfying assignment for F
• or prove that no such assignment exists.
•
•
•
A wide range of applications
Relatively easy to test for small formulas (e.g. with a Truth Table)
However, very quickly becomes hard to solve
–
Search space grows exponentially with formula size
(more on this next)
SAT technology has been very successful in taming this exponential blow up!
23
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
2. SAT-based reasoning


–
–
Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
•
Example domain: planning
3. QBF reasoning (extends SAT)
– A new range of applications
– Two motivating examples
•
–
–
–
–
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
network planning, logistics planning
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
24
SAT Search Space
All vars free
Fix one variable to True or False
Fix another var
Fix a 3rd var
Fix a 4th var
True
False
False
False
True
SAT Problem: Find a path to a True leaf node.
For N Boolean variables, the raw search space is of size 2N
• Grows very quickly with N
• Brute-force exhaustive search unrealistic without efficient heuristics, etc.
25
Worst-Case Complexity
SAT is an NP-complete problem
•
Worst-case believed to be exponential
(roughly 2N for N variables)
•
10,000+ problems in CS are NPcomplete (e.g. planning, scheduling,
protein folding, reasoning)
•
P vs. NP --- $1M Clay Prize
However, real-world instances are usually
not pathological and can often be solved
very quickly with the latest technology!
Typical-case complexity provides a more
detailed understanding and a more
positive picture.
exponential
polynomial
27
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
2. SAT-based reasoning



–
Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
•
Example domain: planning
3. QBF reasoning (extends SAT)
– A new range of applications
– Two motivating examples
•
–
–
–
–
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
network planning, logistics planning
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
28
Typical-Case Complexity
A key hardness parameter for k-SAT: the ratio of clauses to variables
Delete Constraints
Add Constraints
Problems that are not critically constrained tend to be much easier in practice
than the relatively few critically constrained ones
[Mitchell, Selman, and Levesque ’92; Kirkpatrick and Selman – Science ’94]
29
Typical-Case Complexity
Random 3-SAT as of 2004
Phase
transition
Linear time algs.
Random Walk
DP
DP
’
GSAT
Walksat
SP
SAT solvers continually getting close to tackling problems in the hardest region!
SP (survey propagation) now handles 1,000,000 variables
very near the phase transition region
30
Tractable Sub-Structure Can Dominate and
Drastically Reduce Solution Cost!
Median runtime
2+p-SAT model: mix 2-SAT (tractable) and 3-SAT (intractable) clauses
> 40% 3-SAT: exponential scaling
 40% 3-SAT: linear scaling!
Number of variables
(Monasson, Selman et al. – Nature ’99; Achlioptas ’00)
31
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
2. SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
•
Example domain: planning
3. QBF reasoning (extends SAT)
– A new range of applications
– Two motivating examples
•
–
–
–
–
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
network planning, logistics planning
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
32
SAT Encoding Example: Planning Domain
Planning Problem  Propositional CNF formula
by axiom schemas
Discrete time, modeled by integers
•
state predicates: indexed by time at which they hold
E.g. at_location(x,,loc,i), free(x,i+1), route(cityA,cityB,i)
•
action predicates: indexed by time at which action begins
E.g. fly(cityA,cityB,i), pickup(x,loc,i), drive_truck(loc1,loc2,i)
– each action takes 1 time step
– many actions may occur at the same step
33
Encoding Rules
•
Actions imply preconditions and effects
fly(x,y,i)
•

at(x,i) and route(x,y,i) and at(y,i+1)
Conflicting actions cannot occur at same time (A deletes a precondition of B)
fly(x,y,i) and yz
•

If something changes, an action must have caused it
(Explanatory Frame Axioms)
at(x,i) and not at(x,i+1)
•
not fly(x,z,i)

y . route(x,y) and fly(x,y,i)
Initial and final states hold
at(NY,0) and ... and at(LA,9) and ...
34
Using SAT Solvers for Planning
Modeling and Solving a Planning Problem
Problem description in
high level language
axiom
schemas
instantiate
instantiated
propositional
clauses
(manual)
length
mapping
plan
interpret
satisfying
model
(fully automatic)
SAT
engine(s)
36
Planning Benchmark Complexity
Logistics domain – a complex, highly-parallel transportation domain
E.g. logistics.d problem:
o 2,165 possible actions per time slot
o optimal solution contains 74 distinct actions over 14 time slots
(out of 5 x 10^46 possible sequential plans of length 14)
Satplan [Selman et al.] approach is currently fastest optimal planning approach.
Winner ICAPS-05 & ICAPS-06 international planning competitions.
39
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
3. QBF reasoning (extends SAT)
 A new range of applications
– Two motivating examples
•
–
–
–
–
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
network planning, logistics planning
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
40
The Next Challenge in Reasoning Technology
Multi-Agent Reasoning:
Quantified Boolean Formulae (QBF)
– Allow use of Forall and Exists quantifiers over Boolean variables
– QBF significantly more expressive than SAT:
from single-person puzzles to competitive games!
New application domains:
•
•
•
•
Unbounded length planning and verification
Multi-agent scenarios, strategic decision making
Adversarial settings, contingency situations
Incomplete / probabilistic information
war gaming
But, computationally *much* harder (formally PSPACE-complete rather than NP-complete)
Key challenge: Can we do for QBF what was done for SAT solving in the last decade?
Would open up a tremendous range of advanced automated reasoning capabilities!
41
Multi-Agent Reasoning
A Wide Range of Applications
• Logistics planning and scheduling under adversarial / uncertain conditions
– Multi-agent contingency planning with performance guarantees
•
Analysis and validation of distributed agent strategies and coordination
– Event logic
•
Reasoning in rich multi-player (adversarial) settings
– War gaming, designing secure data and communication networks
QBF Technology for Multi-Agent Reasoning offers:
• Performance guarantee / optimality / “worst-case” scenario analysis
– NOTE: Worst-case may be rare but can have catastrophic consequences!
The Challenge: Overcoming the high computational complexity
42
SAT Reasoning vs. QBF Reasoning
SAT Reasoning
Scope of
technology
Worst-case
complexity
Application
areas
Research
status




Combinatorial search
for optimal and nearoptimal solutions
NP-complete
(hard)
planning, scheduling,
verification, model
checking, …
From 200 vars in early
’90s to 1M vars. Now a
commercially viable
technology.
QBF Reasoning




Combinatorial search
for optimal and nearoptimal solutions in
multi-agent, uncertain, or
hostile environments
PSPACE-complete
(harder)
adversarial planning,
gaming, security protocols,
contingency planning, …
From 200 vars in late 90’s
to 100K vars currently. Still
rapidly moving.
43
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
3. QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
•
–
–
–
–
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
network planning, logistics planning
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
44
The Need for QBF Technology
SAT technology, while very successful for single-agent search,
is not suitable for adversarial reasoning.
Must model the adversary and incorporate his actions into reasoning
• SAT does not provide a framework for this
• In fact, it cannot (more on this later)
Two examples next:
1. Network planning: create a data/communication network between N
nodes which is robust under failures during and after network creation
2. Logistics planning: achieve a transportation goal in uncertain
environments
45
Adversarial Planning: Motivating Example
Network Planning Problem:
–
–
–
–
Input: 5 nodes, 9 available edges that can be placed between any two nodes
Goal: all nodes finally connected to each other (directly or indirectly)
Requirement (A): final network must be robust against 2 node failures
Requirement (B): network creation process must be robust against 1 node failure
E.g. a sample robust
final configuration:
(uses only 8 edges)
Side note: Mathematical structure of the problem:
1. (A) implies every node must have degree ≥ 3
(otherwise it can easily be “isolated”)
2. At least one node must have degree ≥ 4
(follows from 1. and that not all 5 nodes can have odd degree in any
graph)
3. Need at least 8 edges total (follows from 1. and 2.)
4. If one node fails during creation, the remaining 4 must be
connected with 6 edges to satisfy (A)
5. Actually need 9 edges to guarantee construction (follows
from 4. because a node may fail as soon as its degree becomes 3)
46
Example: A SAT-Based Sequential Plan
Create edge
Next move if no failures
Final network robust against
2 failures
The plan goes smoothly and we end up with the target
network, which is robust against any 2 node failures
47
Example: A SAT-Based Sequential Plan
Node failures may render the
original plan ineffective, but
re-planning could help make
the remaining network robust.
What if the left
node fails?
Create edge
Node failure during network creation
Next move if a particular node fails
Next move if no failures
Final network robust against
2 more failures
Can still make the remaining 4 nodes
robust using 2 more edges (total 8 used)
• Feasible, but must re-plan to find
a different final configuration
48
Example: A SAT-Based Sequential Plan
Trouble! Can get stuck if
• Resources are limited
What if the top
node fails?
(only 9 edges)
• Adversary is smart
(takes out node with degree 4)
• Poor decisions were made early
on in the network plan
Need to create 4 more edges to
make the remaining 4 nodes robust
• Stuck! Have already used up 6
of the 9 available edges!
49
Example: A QBF-Based Contingency Plan
A QBF solver will return a robust contingency plan (a tree)
• Will consider all relevant failure modes and responses
(only some “interesting” parts of the plan tree are shown here)
….
….
….
….
….
….
….
….
only 8
edges
used
Create edge
Node failure during network creation
Next move if a particular node fails
Next move if no failures
9 edges
needed
Final networks robust against
2 more failures
only 8
edges
used
9 edges
needed
50
Another Example: Logistics Planning
• Blue nodes are cities, green nodes are military bases
(1)
(2) SatPlan
QbPlan
City-1
*At any step commercial
player can transport up to
80 civilians
(s1)

(s1)
City-2
(s1)
City-4
(s2)
60p
(s1)
(s2)
City-3
•
• operator: “transport” t(who, amount, from, to, step)
• parallel actions can be taken at each step
• Goal: Send 60 personal from Base-1 to Base-2 in at most 3
steps
(20p)
+ (up
80c)
<= 100
(60p)
+ (80c)
> to
100
(civilian
transport capacity)
Re-planning needed !!!
One player: military player, deterministic classic planning, SatPlan
•
•
• Blue edges are commercial transports, green edges are
military
• Green edges (transports) have a capacity of 60 people, blue
edges have a capacity of 100 people
(s3)
(s3)
(s2)
Base 1
20p 20p
60p 20p
Base 2
(s2)
(1) Sat-Plan: t(m, 60, base-1, city-3, 1), t(m, 60, city-3, city-4, 2), t(m, 60, city-4, base-2, 3)
Two players: deterministic adversarial planning QB Plan
•
Military Player (m) is “white player”, Commercial Player (c) is “black player” (Chess analogy).
Commercial player can move up to 80 civilians between cities. Commercial moves can not be
invalidated. Goal can be read as: “send 60 personal from Base-1 to Base-2 in at most 3 steps
whatever commercial needs (moves) are”
•
If commercial player decides to move 80 civilians from city-3 to city-4 at the second step, we should
replan (1). Indeed, the goal can not be achieved if we have already taken the first action of (1)
•
(2) QB-Plan: t(m, 20, base-1, city-1, 1), t(m, 20, base-1, city-2, 1), t(m, 20, base-1, city-3, 1), t(m, 20, city-1, city-4, 2),
t(m, 20, city-2, city-4, 2), t(m, 20, city-3, city-4, 2), t(m, 60, city-4, base-2, 3)
51
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
3. QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
 network planning, logistics planning

–
–
–
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
52
Quantified Boolean Logic
Boolean logic extended with “quantifiers” on the variables
– “there exists a value of x in {True,False}”, represented by x
– “for every value of y in {True,False}”, represented by y
– The rest of the Boolean formula structure similar to SAT,
usually specified in CNF form
E.g. QBF formula F(v,w,x,y) =
v w x y : (v or w or x) and (v or w) and (v or y)
Quantified Boolean variables
constraints (as before)
53
Quantified Boolean Logic: Semantics
F(v,w,x,y,z) = v w x y : (v or w or x) and (v or w) and (v or y)
What does this QBF formula mean?
Semantic interpretation:
F is True iff
“There exists a value of v s.t.
for both values of w
there exists a value of x s.t.
for both values of y
(v or w or x) and (v or w) and (v or y) is True”
54
Quantified Boolean Logic: Example
F(v,w,x,y,z) = v w x y : (v or w or x) and (v or w) and (v or y)
Truth Table for F as a SAT formula
v
w
x
y
F
0
0
0
0
0
0
0
0
1
1
0
0
1
0
0
0
0
1
1
1
0
1
0
0
0
0
1
0
1
0
0
1
1
0
0
0
1
1
1
0
1
0
0
0
0
1
0
0
1
0
1
0
1
0
1
1
0
1
1
1
1
1
0
0
1
1
1
0
1
1
1
1
1
0
1
1
1
1
1
1
Is F True as a QBF formula?
Without quantifiers (as SAT):
have many satisfying assignments
e.g. (v=0, w=0, x=0, y=1)
With quantifiers (as QBF):
many of these don’t work
e.g. no solution with v=0
F does have a QBF solution
with v=1 and x set depending on w
55
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
3. QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
 network planning, logistics planning


–
–
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
56
QBF Modeling: Building Blocks Similar to SAT
•
•
Boolean Variables: states + actions
w-king_at_(2,1)_3
=
“white king is at cell (1,2) after time step 3”
move_b-rook_(1,1)_(2,1)_2
=
“move b-rook from (1,1) to (2,1) at step 2”
truck1_at_cityA_2
load_box20_truck1_4
=
=
“truck #1 is at city A after time step 2”
“load box #20 onto truck #1 at time step 4”
Constraints: initial state + goal / query + rules of the domain
w-king_at_(4,1)_start
=
“w-king starts out at cell (4,1)”
(move_b-rook_(1,1)_(3,1)_2 implies (b-rook_at_(1,1)_2 and empty_(2,1)_2))
= “if b-rook moves from (1,1) to (3,1) at time step 2,
then it must be at cell (1,1) then, and cell (2,1) must be empty ”
box12_at_cityC_end
=
“box12 must reach cityC at the end”
(drive_truck1_cityA_cityB_1 implies truck1_at_cityA_1)
57
QBF Modeling: The Semantics
Example 1: a 4-move chess game
There exists a move of the white s.t.
for every move of the black
there exists a move of the white s.t.
for every move of the black
the white player wins
Example 2: contingency planning
for disaster relief
There exist preparatory steps s.t.
for every disaster scenario within limits
there exists a sequence of actions s.t.
necessary food and shelter can
be guaranteed within two days
58
QBF and Uncertainty
Adversarial planning requires reasoning about
uncertainty in the actions of the adversary
(A) Uncertainty may be captured as probabilities over adversarial actions
: probabilistic planning
results good in expectation or with high probability
(B) Uncertainty may alternatively be modeled as a set of contingencies
: discrete contingency planning
guaranteed results robust against all modeled modes of failure
QBF in the basic form takes approach (B)
Can be extended to weighted QBF in order to capture (A)
59
Adversarial Uncertainty Modeled as QBF
•
•
•
•
Two agents: self and adversary
Both have their own set of actions, rules, etc.
Self performs actions at time steps 1, 3, 5, …, T
Adversary performs actions at time steps 2, 4, 6, …, T-1
The following QBF formulation is True if and only if
self can achieve the goal no matter what actions adversary takes
There exists a self action at step 1 s.t.
for every adversary action at step 2
there exists a self action at step 3 s.t.
for every adversary action at step 4
…
there exists a self action at step T s.t.
(
(initialState(time=1) and
self-respects-modeled-behavior(1,3,5,…,T) and goal(T))
OR (NOT adversary-respects-modeled-behavior(2,4,…,T-1)) )
60
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
3. QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
 network planning, logistics planning



–
Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
61
QBF Search Space
Initial state
 : self
Self action
 : adversary
Adversary action
Self action
Adversary
action
No goal
Goal
No goal
Goal
Initial state
Self action
Recall traditional SAT-type search space
Self action
Goal
No goal
Goal
62
QBF Solution: A Policy or Strategy
Initial state
Self action
Adversary action
Self action
Adversary
action
Goal
No goal
Goal
Goal
Goal
Initial state
Contingency plan
• A policy / strategy of actions for self
• A subtree of the QBF search tree (contrast with
a linear sequence of actions in SAT-based planning)
Self action
Self action
No goal
Goal
63
Exponential Complexity Growth
Planning (single-agent):
find the right sequence of actions
HARD: 10 actions, 10! = 3 x 106 possible plans
Contingency planning (multi-agent):
actions may or may not
produce the desired effect!
…
1 out
of 10
4 out
2 out of 8
of 9
REALLY HARD: 10 x 92 x 84 x 78 x … x 2256 =
10224 possible contingency plans!
exponential
polynomial
64
Exponential Complexity Growth:
Chess Problems (benchmark domain for REAL)
Board Parameters
No. of vars.
QBF model
size (no. of
constraints)
Boolean
search
space
RAW SPACE
(legal moves)
3
600
~ 4K
10180
107
5
4
1.3K
~ 10K
10390
1012
5x5
5
5
3K
~ 25K
10900
1014
6x6
8
6
10K
~ 100K
103,000
1019
7x7
10
7
24K
~ 200K
107,200
1025
8x8
16
11
90K
~ 500K
1027,000
1043
bxb
p
m
O(pb3m)
[highly board
dependent]
O(2pb3m)
O(pmb3m)
Board
size
No. of
pieces
No. of
moves
4x4
3
4x4
(such problems solved in under 20 seconds by Qbf-Cornell)
65
Computational Complexity Hierarchy
Hard
EXP-complete:
games like Go, …
EXP
PSPACE-complete:
QBF, adversarial
planning, …
NP-complete:
PSPACE
SAT, scheduling,
graph coloring, …
NP
P-complete:
circuit-SAT, …
P
In P:
sorting, shortest path…
Easy
Note: widely believed hierarchy; know P≠EXP for sure
66
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
3. QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
4. High-Performance QBF reasoning
– Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
 network planning, logistics planning




Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
67
QBF Scaling Behavior in Practice
Although hard in the worst-case, QBF technology can scale well in practice.
•
Real-world problems often have built-in structure and contain hidden tractable
sub-problems that make QBF solvers much faster in practice.
(Note: This has helped make SAT-based single-agent reasoning scale to large
problems with 1,000,000+ variables. We are pushing QBF toward this goal.)
•
Qbf-Cornell and Qbf-CornellD can automatically exploit many hidden tractable
sub-structures and regularities in the problem (scaling plots on the next few slides)
On many chess instances, we observe
– either low-polynomial (or even near-linear) scaling
– or exponential scaling but with a much lower exponent than raw search
68
Scaling: Number of Pieces
Qbf-CornellD compared with the raw space of legal chess moves
searched at 100,000,000 moves per second:
Polynomial scaling with number of pieces (log-log plot)
8K vars,
150K constraints
Runtime in seconds (logscale)
1e+08
Raw Search at 100M moves/sec
Cornell QBF Solver
(1 year)
1e+07
1e+06
100000
(1 day)
10000
1000
100
(1 minute)
10
17K vars,
400K constraints
1
8
9
10
11
12
13
14 15 16 17 18 19
Number of pieces (logscale)
69
Scaling: Number of Steps
The worst-case scaling is exponential in the number of steps. However, in practice,
the exponent is much smaller for the Qbf-CornellD than for raw search.
Exponential speed-up over
the raw search time
Runtime in seconds (logscale)
Scaling with number of steps (log-plot)
Raw Search at 100M moves/sec
Cornell QBF Solver
1e+14
1e+12
 Problems remain feasible
much longer
1e+10
5-18K vars,
100-370K constraints
1e+08
(1 year)
1e+06
(1 day)
10000
100
(1 minute)
1
3
5
7
9
11
Number of steps
70
Scaling: Number of Steps
Many real-world problems contain tractable (but hidden) sub-problems.
Qbf-CornellD can often automatically learn and exploit these.
Reachability-based boards:
Scaling with no. of pieces, Hidden tractable sub-structure (log-plot)
Runtime in seconds (logscale)
10000
Near-linear scaling
with number of steps!
Raw Search at 100M moves/sec
Cornell QBF Solver
y = x, for comparison
(1 hour)
1000
77K vars,
2.2M constraints
100
(1 minute)
10
1
3
5
7
9
11
13
15
Number of steps
71
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
 QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
4. High-Performance QBF reasoning
 Key research advances
– The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
 network planning, logistics planning




Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
72
Key Technical Advances
1. Good scaling with increasing number of quantifier alternations
2. New problem modeling techniques
• natural, generic, automated
• exploited by new solver techniques (Qbf-Cornell [’05], Qbf-CornellD [’06])
3. Effective constraint propagation across quantifiers
• essential for SAT solver success; challenging for QBF
4. Addressing efficiency drop caused by auxiliary variables
• the “illegal search space issue” specific to QBF
[more on these next]
73
Key Advance #1
•
Most QBF benchmarks have only 2-3 quantifier levels
–
–
–
•
Might as well translate into SAT (it often works!)
Early QBF solvers focused on such instances
Benchmarks with many quantifier levels are often the hardest
Practical issues in both modeling and solving become much more
apparent with many quantifier levels
QBF solvers can now scale well with
10+ quantifier alternations
Achieved through better modeling and learning-while-reasoning techniques.
[details to follow]
74
Key Advance #2
QBF solvers are extremely sensitive to encoding!
–
Especially with many quantifier levels,
e.g., evader-pursuer chess instances
[Madhusudan et al. 2003]
Instance
(N, steps)
Model X
[Madhusudan et al. 2003]
Model A
[Cornell 2005]
QuBEJ
[Italy]
Semprop
[Germany]
Quaffle
[Princeton]
Best other
solver
Model B
[Cornell 2005]
QbfCornell
Best other
solver
QbfCornell
4
7
2030
>2030
>2030
7497
3
0.03
0.03
4
9
--
--
--
--
28
0.06
0.04
8
7
--
--
--
--
800
5
5
We now have generic QBF modeling techniques
that are simple and efficient for solvers
[details to follow]
75
Key Advance #3
For QBF, traditional encodings hinder unit propagation
–
–
–
E.g. unsatisfiable “reachability” queries
A SAT solver would have simply unit propagated
Most QBF solvers need 1000’s of backtracks and relatively complex
mechanisms like learning to achieve simple propagation
Best solver
with only unit
propagation
Best solver
(Qbf-Cornell)
with learning
conf-r1
2.5
0.2
conf-r5
8603
5.4
conf-r6
>21600
7.1
?
q-unsat: too few steps for White
QBF solvers now achieve effective propagation across quantifiers
New solver (Qbf-CornellD) exploiting new modeling techniques [details to follow]
76
Example: Lack of Effective Propagation
(in Traditional QBF Solvers)
Question:
Can White reach the
pink square without
being captured?
Impossible! White has one too
few available moves
[ click image
for video ]
This instance should ideally be easy even with many additional (irrelevant) pieces!
Unfortunately, all CNF-based QBF solvers scale exponentially 
Good news: Qbf-CornellD based on dual encoding technology resolves this issue!77
Key Advance #4
QBF solvers suffer from the “illegal search space issue”
[Cornell 2005]
–
–
Auxiliary variables needed for conversion into Boolean form
Can push solver into large irrelevant parts of search space
–
–
Note: negligible impact on SAT solvers due to effective propagation
Our first solution for QBF: Qbf-Cornell [2005]
•
Pass “flags” to the solver, which detect this event and trigger backtracking
Dual encoding based QBF solvers can
completely avoid the illegal search space issue
Achieved through better modeling and solving techniques (Qbf-CornellD).
[details to follow]
78
Intuition for Illegal Search Space:
Search Space for SAT Approaches
Search Space
SAT Encoding
2N+M
Space
Searched
Original
by SAT Solvers
Search
Space
2N/C ; Nlog(N)
2N; Poly(N)
Original
2N
In practice, for
many real-world
applications,
polytime scaling.
79
Search Space of QBF
Search Space
QBF Encoding
2N+M’
Search Space
Standard QBF Encoding
2N+M’’
Space
Searched
Original
by
Qbf-Cornell
Search
Space
with Streamlining
2N
Original
2N
80
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
 QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
4. High-Performance QBF reasoning
 Key research advances
 The technology behind QBF
–
A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
 network planning, logistics planning




Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
81
Progress in QBF Reasoning
QBF technology has grown rapidly and can now scale to 100,000
variables, compared to 200 variables 5 years ago. (Advances largely
due to research supported by Darpa’s REAL program.)
What were the key techniques that made these advances
possible?
82
High-Performance Multi-Agent Reasoning
#2. Learning
while
Reasoning
#1. New QBF
Modeling
Techniques
gain factor
> 102
gain factor > 104
(w.r.t. best QBF solver, 2004)
#3. Structure
Exploitation: gain factor
> 103
Streamlining
Technology behind
Qbf-Cornell [’05] and Qbf-CornellD [’06]
Goal: solve 80% chess boards in < 5 secs
[12 pieces, 5 moves, 10K vars, 40K rules]
Result: achieved 88.5% success!
[10-12 pieces, 7 moves, 12K vars, 100K rules]
Overall performance gain
over state of the art in 2004
= 109x
Reveals:
• Power of QBF for developing and analyzing
strategies in multi-agent settings
• As tool for augmented cognition, e.g. in
military adversarial analysis
83
Qbf-Cornell on a Chess Instance
16 pieces
11 steps
[ click image
for video of
a sample run ]
Raw search
space size
~ 1043
Solved in
~ 20 seconds
QBF solution gives a strategy with which
White king can provably reach the goal square
– no matter what Black tries to do!
84
Generality of the QBF Approach
General reasoning very different from specialized programs.
1. Generality: Improved general reasoning  faster solutions for all domains
–
If you can model a domain (logistics, security protocols, network configuration, etc.),
you can solve it with general reasoning
2. Flexibility: End-user can ask different (sub)-queries
–
“Can two black pieces get within two squares from white king in 4 moves?”
–
“Can my troops capture that bridge in one day without using heavy artillery?
–
Provides strategic analysis capabilities
QBF Reasoning Technology offers even more:
3. Optimality: Guarantees optimality in any multi-agent scenario
–
Can analyze worst-case
e.g. “Can my troops capture that bridge no matter what the enemy does?”
4. Complex internal reasoning:
–
–
Exploits interplay between representations (e.g. move-based & location-based)
Can go beyond a human programmer
85
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
 QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
4. High-Performance QBF reasoning
 Key research advances
– The technology behind QBF
 A. New modeling techniques
–
B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
 network planning, logistics planning




Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
86
A. New QBF Modeling Techniques
a)
b)
Logarithmic encoding
•
Succinctly represents collective behavior of objects
e.g. “in chess, only one move can be taken at each time step”
•
Technique extended to QBF and our setting
Both moves and locations used as building blocks
•
•
c)
Trigger-based pruning of search space
•
•
d)
Allows interplay between move-based and location-based reasoning / representation
Most heuristic chess programs use only move-based search
Automatically generated Indicator variables
flag illegal actions of the universal agent
e.g. in chess, illegal moves of the black player when white plays first
QBF-Cornell implements flag-based pruning
Dual-encoding, combining conjunctive and disjunctive rules (CNF and DNF)
•
•
Exploit a dichotomy between the existential and universal agents
Qbf-CornellD exploits this dual format to further boost QBF-Cornell
RESULT: Over six orders of magnitude improvement
87
Encoding: The Traditional Approach
Problem
of interest
CNF-based
QBF encoding
QBF Solver
e.g. circuit minimization
Any discrete
adversarial task
Solution!
88
Encoding: Our Game-Based Approach
Game G:
Adversarial
Task
players E & U,
states, actions,
rules, goal
“Planning as Satisfiability”
framework
[SelmanKautz ’96]
e.g. circuit minimization
Create CNF encoding
separately for E and U:
Solution!
QBF Solver
Qbf-Cornell
[2005]
Flag-based
CNF encoding
QBF Solver
Qbf-CornellD
[2006]
Dual (split)
CNF-DNF encoding
initial state axioms,
action implies precondition,
fact implies achieving action,
frame axioms,
goal condition
Solution!
Negate
CNF part for U
(creates DNF)
89
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
 QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
4. High-Performance QBF reasoning
 Key research advances
– The technology behind QBF

A. New modeling techniques

B. Learning while reasoning
–
C. Structure discovery
– Experimental Results
5. Summary
 network planning, logistics planning




Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
96
B. Learning while Reasoning
A key component of the QBF solver is the ability to
learn new constraints during the search.
•
The learned information is captured by new clauses that are added to the
formula. These clauses can dramatically prune the remaining search.
Recall:
Computational cost (secs)
•
4000
50 var
40 var
20 var
3000
2000
Delete Rules
Add Rules
1000
0
Ratio of clauses to variables
Hardness complexity curves
97
Encoding: Before Learning
Before learning, constraints in the encoding specify the rules of the game
(i.e. what it means for a move to be legal) and the query as Boolean logic. E.g.
Rule
: “if White Rook moves from a2 to a5 at step 7, then a4 must be empty at 7.”
Encoding : (move_wRook_a2_a5_7  empty_a4_7)
True/False variables
Rule
: “Only one piece can move at each step t.”
Encoding : Several constraints of the form:
(move_pieceA_cell1_cell2_t
 NOT move_pieceB_cell3_cell4_t)
a
b
c
d
e
f
6
: “Can White King reach cell e2 in
at most 6 steps?”
Encoding : (location_wKing_e2_1 OR location_wKing_e2_2
OR … OR location_wKing_e2_6)
Query
5
4
3
2
1
98
Encoding: After Learning
After learning, new constraints specify 1000’s of more complex observations
logically deduced from the board.
These observations typically combine spatial and temporal knowledge, often in
terms of “if-then” facts. Some of these are used subsequently in the search,
and the rest are gradually discarded. E.g.
Knowledge : “if White King is at c5 and White Pawn is at d3 initially, then White King
cannot be at d3 after making 2 moves (its own piece is blocking d3).”
Encoding
: ((location_wKing_c5_1 AND location_wPawn_d3_1)
 NOT location_wKing_d3_5)
Note: White moves on odd steps, Black on even steps.
a
b
c
d
e
f
6
Knowledge : “if White moves the bottom Rook at step 1,
then wKing cannot be in row 2 at step 7.”
5
Encoding
3
: For each cell X in {a2,b2,c2,d2,e2,f2},
a constraint: (NOT location_wRook_f2_2)
 (NOT location_wKing_X_7)
4
2
1
99
Learning while Reasoning
QBF reasoning engine has no hard-coded knowledge of chess
or distance functions!
– still learns automatically, e.g., that a square is not reachable!
– continuously adds learned/inferred constraints from failure, simplifying future search
NOTE: Such knowledge must be hand-programmed into conventional solvers
E.g. can w-king reach top-left cell in 5 moves?
Without Learning
explores > 20K
possibilities
With Learning
explores < 5
possibilities
100
Example: Learning “Reachability”
Query: Can the white king reach the top-left cell in 5 moves?
Without Learning
(> 20K possibilities explored)
[ click images
for video
]
With Learning
(< 5 possibilities explored)
RECALL: No hard-coded knowledge.
Learns automatically about reachability
101
Learning while Reasoning:
A Visualization
Solver learns, e.g., to focus the search
to the most relevant areas of the board
without explicitly being told anything
about it!
RESULT: Over two orders of magnitude improvement
102
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
 QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
4. High-Performance QBF reasoning
 Key research advances
– The technology behind QBF

A. New modeling techniques

B. Learning while reasoning

C. Structure discovery
– Experimental Results
5. Summary
 network planning, logistics planning




Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
103
C. Structure Exploitation: Streamlining
Search space
R2
R1
Exploit domain and problem structure
to narrow down search to
small regions very likely to have a solution!
E.g in chess,
solutions
(white player wins)
Try region R1: white never moves a piece
away from the goal
: no solution found
Try region R2: white doesn’t use the two
of its far away pawns
: solution found!
Key invariant: if an agent’s actions are restricted and it still succeeds,
it will succeed in the original setting as well
The art lies in choosing “good” restrictions – not too strong, not too weak!
RESULT: Over three orders of magnitude improvement
104
Structure Exploitation:
Structure Discovery
Constraint graph of a
reasoning problem
One node per variable:
edge between two variables
if they share a constraint.
Planning problem encoding
843 vars, 7301 constraints, approx. min backdoor 16
(“backdoor” set = reasoning shortcut)
(Gomes et al. ’03, ’04)
105
Structure Exploitation:
Structure Discovery
After setting 5 backdoor vars
(out of 800)
After setting just 12 backdoor vars
– problem almost solved!
106
Structure Exploitation:
Structure Discovery
MAP-6-7.cnf infeasible planning instances. Strong backdoor of size 3.
392 vars, 2,578 clauses.
Structure Exploitation:
Structure Discovery
After setting 2 (out of 392) backdoor vars -- complexity almost gone!
Dynamic view: Running SAT solver
(no backdoor detection)
[ click image
for video ]
109
SAT solver detects backdoor set
[ click image
for video ]
110
Tractable Sub-Structure Can Lead to
Near-Linear Scaling in QBF as Well!
Many real-world problems contain tractable (but hidden) sub-problems.
Qbf-CornellD can often automatically learn and exploit these.
Reachability-based boards:
Scaling with no. of pieces, Hidden tractable sub-structure (log-plot)
Runtime in seconds (logscale)
10000
Near-linear scaling
with number of steps!
Raw Search at 100M moves/sec
Cornell QBF Solver
y = x, for comparison
(1 hour)
1000
77K vars,
2.2M constraints
100
(1 minute)
10
1
3
5
7
9
11
13
15
Number of steps
111
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
 QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
4. High-Performance QBF reasoning
 Key research advances
 The technology behind QBF

A. New modeling techniques

B. Learning while reasoning

C. Structure discovery
 Experimental Results
5. Summary
 network planning, logistics planning




Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
112
Experimental Results
5-15
quantifier
levels
(reachability)
7-9
quantifier
levels
Dual
Encoding
Pure CNF Encoding
xChess instance
name
T/F
Semprop
[Germany]
sKizzo
[France]
Quaffle
[Princeton]
Qbf-Cornell
[2005]
Qbf-CornellD
[2006]
conf-r1
F
12
4.0
15
1.3
0.01
conf-r2
F
25
5.86
33
2.5
0.02
conf-r3
F
55
9.3
62
4.1
0.03
conf-r4
F
85
26
124
6.4
0.04
conf-r5
F
985
84
676
34
0.08
conf-r6
F
2042
73
713
49
0.10
conf01
F
1225
492
--
539
6.4
conf02
F
93
30
6.0
1.0
0.0
conf03
T
--
1532
--
83
1.4
conf04
T
--
--
2352
100
3.5
conf05
F
3290
448
510
196
0.1
conf06
F
--
memout
--
633
30.6
conf07
F
261
42
78
3.5
0.0
conf08
T
--
1509
--
1088
31.2
113
Experimental Results, contd.
7-9
quantifier
levels
Dual
Encoding
Pure CNF Encoding
xChess instance
name
T/F
Semprop
[Germany]
sKizzo
[France]
Quaffle
[Princeton]
Qbf-Cornell
[2005]
Qbf-CornellD
[2006]
conf1a
T
627
83
--
161
1.8
conf1b
F
682
176
2939
124
1.3
conf1c
T
659
804
--
156
2.1
conf1d
F
706
1930
1473
148
2.2
conf2a
T
--
--
--
438
65.9
conf2b
F
--
--
--
275
56.9
conf3a
T
--
memout
--
653
5.2
conf3b
F
--
--
2128
327
2.2
conf4
F
--
--
--
274
32.0
conf5
F
1018
427
142
11
0.1
Qbf-Cornell and Qbf-CornellD dramatically outperform
all leading QBF solvers on these challenging instances
114
Tutorial Roadmap
 Automated reasoning
 The complexity challenge
 State of the art in Boolean reasoning
 SAT-based reasoning




Boolean logic
Search space, worst-case complexity
Hardness profiles, scaling in practice
Modeling problems as SAT
 Example domain: planning
 QBF reasoning (extends SAT)
 A new range of applications
 Two motivating examples
 High-Performance QBF reasoning
 Key research advances
 The technology behind QBF

A. New modeling techniques

B. Learning while reasoning

C. Structure discovery
 Experimental results
5. Summary
 network planning, logistics planning




Quantified Boolean logic
Modeling problems as QBF
Search space, worst-case complexity
Scaling in practice
115
Where Does QBF Reasoning Stand?
We have come a long way since the first QBF solvers 5 years ago!
(thanks to the Darpa REAL program)
• From 200 variable problems to 100,000 variable problems
(Qbf-Cornell and Qbf-CornellD)
• From 2-3 quantifier alternations to 10+ quantifiers
• New techniques for modeling and solving
• A better understanding of issues like
propagation across quantifiers and illegal search space
• Many more benchmarks and test suites
116
Summary
QBF Reasoning: a promising new automated reasoning technology!
On the road to a whole new range of applications:
•
Strategic decision making
•
Performance guarantees in complex multi-agent scenarios
•
Secure communication and data networks in hostile environments
•
Robust logistics planning in adversarial settings
•
Large scale contingency planning
•
Provably robust and secure software and hardware
117
Creating the Next Generation Reasoning
Technology at the Intelligent Information Systems
Institute (IISI), Cornell University
Director: Dr. Carla Gomes
Connections and Collaborations
Branching Processes
K. Athreya (Cornell)
Power laws vs. Small-world
S. Strogatz (Cornell)
T. Walsh (U. New South Wales)
Approximations and Randomization
Lucian Leahu (Cornell)
David Shmoys (Cornell)
Random CSP Models
C. Fernandez, M. Valls (U. Lleida)
C. Bessiere (LIRMM-CNRS)
C. Moore (U. New Mexico)
Learning Dynamic
Restart Strategies
Formal Models. Problem
structure, Backdoors
Robustness vs.
Fragility
John Doyle (Caltech)
Walter Willinger (AT&T Labs)
E. Horvitz (Micrsoft Research)
H. Kautz and Y. Ruan (U. Washington)
Nudelman and Shoham (Stanford)
H. Chen (Cornell)
John Hopcroft (Cornell)
Jon Kleinberg (Cornell)
R. Williams (CMU)
Joerg Hoffman (Max-Planck Inst.)
Information Theory
S. Wicker (Cornell)
118
Computer Science
Engineering
Mathematics
Operations Research
Physics
Cross-fertilization
of ideas for the study
and design of
Intelligent Systems
Economics
Cognitive Science
Research part of Cornell’s Intelligent Information Systems Institute.
Director: Carla Gomes.
Additional Material
Ease of Use / Portability
Q: Can someone else easily use our techniques for problems other than chess?
A: Yes, it is straightforward once they understand the concept of a QBF encoding.
– Solver: they can use the Qbf-Cornell and Qbf-CornellD off-the-shelf as a black box.
– Encoder: A general encoder can be written for each application domain.
Our chess encoder will serve as a useful starting point.
– Our technique is described in research publications [AAAI-05, SAT-06] and has
already been explored and extended by others in various contexts [e.g. Zhang AAAI06, Tang-Malik SAT-06, Bessiere-Verger CP-06]. (Our related work received best
paper awards at the AAAI-06 and CP-06 conferences.)
– We have an initial proposal for a generic automated QBF encoder for a higher-level
adversarial planning language. This language will be similar to commonly used
single-agent planning specification languages like STRIPS.
Q: Are we creating a general solution?
A: Yes, the approach provides a fully general platform for adversarial reasoning.
– Solver: generic; no chess-based or other specific knowledge embedded.
– Encoder technology: the core is applicable to any discrete adversarial reasoning task.
– Chess encoder: can be adapted manually for other tasks.
121
Encoding: Generation
•
The encoding process is automatic and very efficient.
–
–
–
–
–
The “encoding” is a re-statement of the problem in the language of the solver.
We use a fully automated chess-to-QBF encoder.
Each of the 600 go/no-go boards required < 0.3 sec to encode.
The hardest instances we have tried required < 2 sec to encode.
For chess and most future applications of interest, encoder time will be polynomial in
the problem parameters. The challenge is always the solver time.
The encoding process takes a fraction of the time compared to the solver.
122
Encoding: Effect of Number of Steps
The number of steps is simply a parameter to the encoder.
If a different number of steps is desired, the new encoding can be generated within
seconds by re-running the encoder.
Additional information:
– Decreasing the number of steps (and possibly also changing the query itself) :
Requires changing only the few constraints that encode the new query
(< 1% of the encoding). The rest of the encoding (> 99%) remains untouched.
– Increasing the number of steps in the query:
Requires additional variables and constraints for the extra steps,
which are simply appended to the original encoding.
– Transfer learning technology can be exploited to re-use knowledge gained from
the original encoding when the query is changed; we are exploring this.
123
From Adversarial Tasks To Games
Example #2:
The Chromatic Number Problem: Given a graph G and a positive
number k, does G have chromatic number k?
–
Chromatic number: minimum number of colors needed to color G so
that every two adjacent vertices get different colors
–
A game with 2 turns
–
•
Moves : First, E produces a coloring S of G; second, U produces
a coloring T of G
•
Rules : S must be a legal k-coloring of G; T must be a legal
(k-1)-coloring of G
•
Goal
: E wins if S is valid and T is not
“E wins” iff graph G has chromatic number k
124