Satisfiability and Unsatisfiability:
Proof Complexity and Algorithms
Paul Beame
University of Washington
1
Outline
Intro to proof complexity
Proof complexity and complete SAT algs
 Review Resolution/DPLL connection
 Application to random k-SAT
 Proof complexity and DPLL extensions
Clause learning
Caching
Proof system survey
2
Proof Complexity
Study of the number of symbols required
for proofs of unsatisfiability (or tautology) in
propositional logic
Does not address algorithmic issue
 How would you find short proofs if they existed?
Existence of short proofs for every
unsatisfiable formula is equivalent to
NP = co-NP (and is implied by P=NP)
 Generally believed that such proofs don’t exist
3
Propositional proof systems
A propositional proof system is a polynomial time
algorithm V s.t. for all formulas F
F is unsatisfiable  there exists a string P
s.t. V accepts input (P,F)
Note:
  direction is usually called soundness
  direction is usually called completeness
Proof complexity is size of P as a function of the
size of F
4
Sample propositional proof
systems
Truth tables
 proof is a fully filled out truth table
easy to verify that it is filled out correctly and all truth
assignments yield T
Frege systems
 inference rules: e.g. modus ponens
A, (A  B) | B
 axioms: e.g. excluded middle | (A  A)
 axioms & inference rules are schemas
can make consistent substitution of arbitrary formulas for
variables in schema
e.g. excluded middle yields ((xy)  (xy))
5
The graph of a proof
F1
F5
F2
Axioms/inputs
are sources
F3
F10
F8
F4
F12
F9
F13
F7
F6
F11
Inference rule
associated with
each node
Sink labelled by tautology
(or L for refutation)
6
Resolution
Start with clauses of CNF formula F
Resolution rule
 Given (A  x), (B  x) derive
(A  B)
The empty clause is derivable
 F is unsatisfiable
Proof size = # of clauses used
7
Restricted forms of Resolution
Resolution
 In general, graph of inferences is a directed
acyclic graph
Tree Resolution
 Graph of inferences forms a binary tree
i.e., if you want to use a clause more than once you
need to re-derive it
Regular Resolution
 Once a variable is resolved out of a clause you
can’t add it back later
 More general than Tree Resolution
8
Satisfiability Algorithms
Incomplete Algorithms
 Local search
GSAT
Walksat
[Selman,Levesque,Mitchell 92]
[Kautz,Selman 96]
 Belief Propagation
SP
[Braunstein, Mezard, Zecchina 02]
Complete Algorithms
 Backtracking search
DPLL
[Davis,Putnam 60]
[Davis,Logeman,Loveland 62]
DPLL + clause learning
 etc
GRASP, SATO, zchaff, berkmin
9
Proof Complexity and
SAT Algorithms
Proof Complexity
 Unsatisfiable formulas
 Nondeterministic algorithms
SAT Algorithms
 Satisfiable formulas
 Deterministic or randomized algorithms
10
Proof complexity and complete
satisfiability algorithms
In general
 The transcript of the run of a complete
satisfiability algorithm on an unsatisfiable
formula is a proof of its unsatisfiability
For backtracking style algorithms
 Even runs on satisfiable formulas yield
proofs of unsatisfiability of related
formulas
11
Backtracking search/DPLL
DPLL(F)
while F contains a 1-clause x
Residual
F  F|x=1
formula
if F is empty return 1
if F has an empty clause return 0
else select a literal x to branch on
return DPLL(F|x=1)  DPLL(F|x=0)
12
Resolution and DPLL
DPLL tree with any variable selection
rule on an unsatisfiable formula F
generates a Resolution refutation of F
 # of clauses  running time
Note: Regular resolution corresponds
to a similar DAG (read-once branching
program) rather than a tree but
general resolution has no such
analogue
13
DPLL Refutation
a
Clauses
a
1. a b c
2. ab
3. c
4. a d
5. d b
a
b
b
b
c
c
b
b
d
3
c
b
b
d
1
2
4
a b c
ac
a d
3
d
b
5
d b
14
DPLL Refutation
= Tree-like Resolution Proof
a:L
Clauses
1. a b c
2. ac
3. b
4. a d
5. a b
b: a
c: a b
b: a
d:a b
3
3
b
b
1
2
4
a b c
ac
a d
5
d b
15
Random k-CNF formulas
Randomly choose m clauses over n
variables independently each of size k
 Each size k clause is equally likely
Key parameter r=m/n, the ratio of
clauses to variables:
Threshold value rk* of r
Below, almost certainly satisfiable
Above, almost certainly unsatisfiable
Hardest problems near threshold
16
DPLL on random 3-CNF*
Proof complexity
# of DPLL
backtracks
shows 2Q(n/r) time
is required for
unsatisfiable formulas
for r  r3*
[B,Karp,Saks,Pitassi 98]
[Ben-Sasson 02]
1
0
What about satisfiable
formulas below threshold?
4.267
ratio of clauses to variables
r
[Mitchell,Selman,Levesque 92]
* n = 50 variables
17
Some simple select choices
for DPLL algorithms
UC: Unit Clause/Ordered DLL
 Choose variables in a fixed order
 Always set True first
UCwm: Unit Clause with majority
 Choose variables in a fixed order
 Apply a majority vote among 3-clauses for
assigning each value
GUC: Generalized Unit Clause
 Choose a variable v in a shortest clause C
 Set v to satisfy C
18
Parameters of residual formulas
follow trajectories
1
2-clause ratio
[Chao,Franco 88]
0
[Frieze,Suen 95]
[Achlioptas 00]
[Achlioptas,Sorkin 00]
UC
GUC
2/3
[Kaporis et al 03]
8/3
3-clause ratio
3.52
19
Satisfiability for mixed random
formulas: proven properties
[Achlioptas et al 96]
1
[Kaporis et al 03]
2-clause ratio
?
?
? ? ?
?
[Dubois 01]
UNSAT
? ?
?
?
SAT
?
2/3
2.28
3.52
3-clause ratio
?
4.501
20
Resolution proof complexity of
mixed random formulas
Theorem: [Achlioptas, B, Molloy 01] For
any constants r21 and r3 0 random
formulas with r2n 2-clauses and r3n
3-clauses need exponential-size
resolution proofs almost surely
Extends [Chvatal-Szemeredi 88]
21
Long-running DPLL Executions and
Proof Complexity
Residual formula at
each node is a mix of
2- and 3-clauses
Residual formula at
is unsatisfiable
2
rn
Every
resolution
Algorithm’s
proof of unsatisfiability
is exponentially long
22
Trajectory on 3-CNF
2-clause ratio
1
UC Algorithm Trajectory
0
Provably
UNSAT
& Hard
Provably
SAT & Easy
3.52 3.81 4.267 4.51
3-clause ratio
23
Exponential lower bounds for 3-CNF
formulas below ratio 4.267
Corollary For almost all 3-CNF formulas,
above ratio
 3.81 UC takes exponential time
 3.83 UCwm takes exponential time
 4.01 GUC takes exponential time
Exponential lower bound below the
proven k-CNF threshold
[Achlioptas, Peres 02]
rk*  2k ln 2 – (k+4)/2
[Achlioptas, B, Molloy 04] For k  4, for
almost all k-CNF formulas UC takes
exponential time above ratio c 2k/k
Note These formulas have huge numbers of
satisfying assignments (more than 2 (1-) n
out of a possible 2n) but still are hard
25
Open Problem
Closing gap for unsatisfiability of mixed formulas
would yield sharp threshold behavior for each
algorithm A
 Below rA algorithm runs
in linear time
 Above rA algorithm
requires exponential time
Conjecture: no polynomial time selection heuristic
will work up to the threshold
 rA  r3* for all algorithms A
26
Outline
Intro to proof complexity
Proof complexity and complete SAT algs
 Review Resolution/DPLL connection
 Application to random k-SAT
 Proof Complexity and DPLL extensions
Clause learning
Caching
Proof system survey
27
Clause Learning
At each backtrack point in DPLL search
add new clauses to the input formula
that correspond to known causes of
failure of the search
 Critical to the good behavior of the best
current algorithms
What is the power of clause learning?
 Intuitively: it makes DPLL trees DAG-like
  regular resolution?  general resolution?
28
Conflict Graph
Known Clauses
(p  q  a)
( a   b   t)
(t  x1)
(t  x2)
(t  x3)
(x1  x2  x3  y)
(x2  y)
Current decisions
p  false
q  false
b  true
1-UIP scheme
(t)
x1
p
y
q
a
t
b
x2
false
y
x3
Decision scheme
(p  q   b)
29
Proof complexity
and clause learning
Clause learning is no stronger than
Resolution
 Implication graph yields resolution derivation of
each learned clause
There are examples where clause learning
works in polynomial time but any Regular
Resolution proof is exponential
[Sabharwal, B, Kautz 04]
 Clause learning can sometimes be much stronger
than Regular Resolution
30
Proof complexity
and clause learning
Separation is true for any proper subsystem
R of Resolution for which refuting F|x is no
harder than F
 Clause learning does not have this property
unless it is as powerful as resolution
Idea: Take formula F easy for Resolution but
hard for R and create new formula F’ that is
F plus clauses for (CpC) for each C in
Resolution proof of F where each pC is a new
propositional variable
 Setting each pC to true yields original F but
branching on sequence of pC allows sequence of
clauses C to be learned
31
Still Open
Can clause learning do everything regular
resolution can?*
Is clause learning as powerful as general
resolution?
* Note: Alan Van Gelder has pointed out that if one returns
learned clauses up the tree and resolves them while
backtracking then clause learning does efficiently simulate
regular resolution
32
Proof Complexity and Formula
Caching
Formula Caching: Memoization
 Cache residual formulas that have been
already found to be unsatisfiable
 Do not make recursive call if the current
residual formula is already cached
33
DPLL
DPLL(F)
if F is empty report satisfiable and halt
if F contains the empty clause L
backtrack
else select a literal x
DPLL(F|x)
DPLL(F|x)
34
Basic Formula Caching
FC(F,L)
if F is empty report satisfiable and halt
if F contains the empty clause L or FL
return
else choose a literal x
FC(F|x,L)
FC(F|x,L)
Start with FC(F,)
add F to L
35
Proof Complexity of Formula
Caching [B, Impagliazzo, Pitassi, Segerlind 03]
Basic Formula Caching can exponentially
improve DPLL
Basic Formula Caching is not as powerful as
regular resolution
 but some of its natural extensions are
These natural extensions can be
exponentially more powerful than general
resolution or even Res(k) for any k !
 Res(k) is like resolution but uses k-DNFs instead
of clauses
36
Extensions of Formula Caching
Don’t just check membership in the
cache
 F is trivially unsatisfiable given G
iff every clause of G contains some clause
of F
Record the reasons why residual
formulas are known to be unsatisfiable
and check the reasons rather than the
formulas
37
More General Formula Caching
FC*(F,L)
if F is empty report satisfiable and halt
if F is trivially unsatisfiable given L
add F to L
return
else choose a literal x
FC*(F|x,L)
FC*(F|x,L)
add F to L
38
Adding Reasons
FCreason(F,L)
if F is empty report satisfiable and halt
if F is trivially unsatisfiable based on GL
add F to L
return(G)
else choose a literal x
GFCreason(F|x,L)
HFCreason(F|x,L)
J (xG) (xH)
add F,J to L
return(J)
39
Combining Clause Learning and
Formula Caching
Cachet system used to count # of
satisfying assignments
[Sang, Bacchus, B., Kautz, Pitassi 04]
 Built on zchaff’s clause learning
 Uses decomposition of residual formula
into components
 Caches components rather than the whole
formula
 Subtle interactions
40
Outline
Intro to proof complexity
Proof complexity and complete SAT
algs
 Resolution/DPLL review
 Application to random k-SAT
 DPLL extensions
Clause learning
Caching
Proof system survey
41
Proof Systems Hierarchy
ZFC
Ext-Frege
Frege
LS
AC0-Frege
Cutting Planes
PCR
Res(k)
Resolution
DPLL
Polynomial Calculus
Nullstellensatz
Truth Tables
42
AC0-Frege
Constant-depth Frege (AC0-Frege)
 Like Frege but formulas in proof only have
constant # of alternations between
 and 
 Resolution is depth-1 Frege
43
Some Known Hard Problems for
Resolution
Exact/Modular Counting
 Pigeonhole principle: n+1 pigeons can’t fit into n
pigeonholes without getting cosy with each other
 Parity Principle: e.g. can’t pair up an odd number
of people
 Tseitin Tautologies
 These problems are also hard for AC0-Frege
Random Unsatisfiable
 k-CNF
 Graph k-colorability
 Existence of k-cliques in graphs
44
Cutting Planes/PseudoBoolean
Introduced to relate integer and linear
programming [Gomory 59, Chvatal 73]:
 Objects are linear integer inequalities
 Clause (x1 x2  x3) becomes inequality
x1+(1-x2)+x3  1
 Add inequalities xi  0 and 1-xi  0
Goal: derive 0  1
45
Cutting Planes rules
addition:
a1x1 + ... + anxn  A
b1x1 + ... + bnxn  B
(a1+b1)x1+...+(an+bn)xn  A+B
multiplication by positive integer:
a1x1 + ... + anxn  A
ca1x1 + ... + canxn  cA
Division by positive integer:
ca1x + ... + canxn  B
a1x1 + ... + anxn  B/c
46
Limitations of Cutting Planes
At least as good as resolution
Can also handle
 pigeonhole principle
 parity principle
But can’t handle counting when it is a
derived property
 Clique-Coloring
A graph can’t have a k-clique and be k-1-colorable
Tseitin tautologies still open
47
Proof Systems Hierarchy
ZFC
Ext-Frege
Frege
LS
AC0-Frege
Cutting Planes
PCR
Res(k)
Resolution
DPLL
Polynomial Calculus
Nullstellensatz
Truth Tables
48
Hilbert’s Nullstellensatz
System of polynomials
Q1(x1,…,xn)=0,…,Qm(x1,…,xn)=0
over field K has no solution in any extension
field of K

there exist polynomials
P1(x1,…,xn),…,Pm(x1,…,xn) in K[x1,…,xn] s.t.
m
 PQ
i
i
1
i=1
49
Nullstellensatz proof system
C
Clause (x1 x2  x3)
becomes equation (1-x1)x2(1-x3)=0
QC
Add equations xi2-xi =0 for each variable
 Guarantees only 0-1 solutions
A proof is polynomials P1,…, Pm+n proving
unsatisfiability: i.e. such that
m

j=1
n
2
+
(x
P jQ Cj  Pm+i - x)  1
i=1
50
Polynomial Calculus
Similar to Nullstellensatz except:
 Begin with Q1,…,Qm+n as before
 Given polynomials R and S can infer
aR + bS for any a, b in K
xiR
 Derive constant polynomial 1
 Degree = maximum degree of polynomial
appearing in the proof
 Can find proof of degree d in time nO(d) using
Groebner basis-like algorithm (linear
algebra)
51
PCR = PC + Resolution
Two variables x and x’ for each proposition x
 x’ stands for x
 include equations x+x’-1=0, x2-x=0, and
(x’)2-x’=0
Translate (x1 x2  x3) as x’1 x2 x’3=0
Same proof rules and proof search as
polynomial calculus
 Efficiently simulates resolution: clausemonomial
 Degree-size relationship for PCR is same as widthsize relationship of resolution
52
Lovasz-Schrijver (LS) Proofs
Linear inequalities
Steps:
 Lift to get degree 2 inequalities
E.g. multiply inequalities
 Project
Combine degree 2 inequalities plus x2=x to
cancel out degree 2 terms
Captures properties of semi-definite
programming
53
Proof Systems Hierarchy
ZFC
Ext-Frege
Frege
LS
AC0-Frege
Cutting Planes
PCR
Res(k)
Resolution
DPLL
Polynomial Calculus
Nullstellensatz
Truth Tables
54
Conclusions
Exchange of ideas between proof complexity
and SAT algorithms can be fruitful
 Better understanding of existing algorithms
 Source of ideas for new algorithms and proof
systems
Potential to branch out beyond DPLL
derivatives
 Wide range of proof systems to explore
55
Thank You
Thank you
to
Toby Walsh
Fahiem Bacchus
Ian Miguel
56