Satisfiability and Unsatisfiability: Proof Complexity and Algorithms Paul Beame University of Washington 1 Outline Intro to proof complexity Proof complexity and complete SAT algs Review Resolution/DPLL connection Application to random k-SAT Proof complexity and DPLL extensions Clause learning Caching Proof system survey 2 Proof Complexity Study of the number of symbols required for proofs of unsatisfiability (or tautology) in propositional logic Does not address algorithmic issue How would you find short proofs if they existed? Existence of short proofs for every unsatisfiable formula is equivalent to NP = co-NP (and is implied by P=NP) Generally believed that such proofs don’t exist 3 Propositional proof systems A propositional proof system is a polynomial time algorithm V s.t. for all formulas F F is unsatisfiable there exists a string P s.t. V accepts input (P,F) Note: direction is usually called soundness direction is usually called completeness Proof complexity is size of P as a function of the size of F 4 Sample propositional proof systems Truth tables proof is a fully filled out truth table easy to verify that it is filled out correctly and all truth assignments yield T Frege systems inference rules: e.g. modus ponens A, (A B) | B axioms: e.g. excluded middle | (A A) axioms & inference rules are schemas can make consistent substitution of arbitrary formulas for variables in schema e.g. excluded middle yields ((xy) (xy)) 5 The graph of a proof F1 F5 F2 Axioms/inputs are sources F3 F10 F8 F4 F12 F9 F13 F7 F6 F11 Inference rule associated with each node Sink labelled by tautology (or L for refutation) 6 Resolution Start with clauses of CNF formula F Resolution rule Given (A x), (B x) derive (A B) The empty clause is derivable F is unsatisfiable Proof size = # of clauses used 7 Restricted forms of Resolution Resolution In general, graph of inferences is a directed acyclic graph Tree Resolution Graph of inferences forms a binary tree i.e., if you want to use a clause more than once you need to re-derive it Regular Resolution Once a variable is resolved out of a clause you can’t add it back later More general than Tree Resolution 8 Satisfiability Algorithms Incomplete Algorithms Local search GSAT Walksat [Selman,Levesque,Mitchell 92] [Kautz,Selman 96] Belief Propagation SP [Braunstein, Mezard, Zecchina 02] Complete Algorithms Backtracking search DPLL [Davis,Putnam 60] [Davis,Logeman,Loveland 62] DPLL + clause learning etc GRASP, SATO, zchaff, berkmin 9 Proof Complexity and SAT Algorithms Proof Complexity Unsatisfiable formulas Nondeterministic algorithms SAT Algorithms Satisfiable formulas Deterministic or randomized algorithms 10 Proof complexity and complete satisfiability algorithms In general The transcript of the run of a complete satisfiability algorithm on an unsatisfiable formula is a proof of its unsatisfiability For backtracking style algorithms Even runs on satisfiable formulas yield proofs of unsatisfiability of related formulas 11 Backtracking search/DPLL DPLL(F) while F contains a 1-clause x Residual F F|x=1 formula if F is empty return 1 if F has an empty clause return 0 else select a literal x to branch on return DPLL(F|x=1) DPLL(F|x=0) 12 Resolution and DPLL DPLL tree with any variable selection rule on an unsatisfiable formula F generates a Resolution refutation of F # of clauses running time Note: Regular resolution corresponds to a similar DAG (read-once branching program) rather than a tree but general resolution has no such analogue 13 DPLL Refutation a Clauses a 1. a b c 2. ab 3. c 4. a d 5. d b a b b b c c b b d 3 c b b d 1 2 4 a b c ac a d 3 d b 5 d b 14 DPLL Refutation = Tree-like Resolution Proof a:L Clauses 1. a b c 2. ac 3. b 4. a d 5. a b b: a c: a b b: a d:a b 3 3 b b 1 2 4 a b c ac a d 5 d b 15 Random k-CNF formulas Randomly choose m clauses over n variables independently each of size k Each size k clause is equally likely Key parameter r=m/n, the ratio of clauses to variables: Threshold value rk* of r Below, almost certainly satisfiable Above, almost certainly unsatisfiable Hardest problems near threshold 16 DPLL on random 3-CNF* Proof complexity # of DPLL backtracks shows 2Q(n/r) time is required for unsatisfiable formulas for r r3* [B,Karp,Saks,Pitassi 98] [Ben-Sasson 02] 1 0 What about satisfiable formulas below threshold? 4.267 ratio of clauses to variables r [Mitchell,Selman,Levesque 92] * n = 50 variables 17 Some simple select choices for DPLL algorithms UC: Unit Clause/Ordered DLL Choose variables in a fixed order Always set True first UCwm: Unit Clause with majority Choose variables in a fixed order Apply a majority vote among 3-clauses for assigning each value GUC: Generalized Unit Clause Choose a variable v in a shortest clause C Set v to satisfy C 18 Parameters of residual formulas follow trajectories 1 2-clause ratio [Chao,Franco 88] 0 [Frieze,Suen 95] [Achlioptas 00] [Achlioptas,Sorkin 00] UC GUC 2/3 [Kaporis et al 03] 8/3 3-clause ratio 3.52 19 Satisfiability for mixed random formulas: proven properties [Achlioptas et al 96] 1 [Kaporis et al 03] 2-clause ratio ? ? ? ? ? ? [Dubois 01] UNSAT ? ? ? ? SAT ? 2/3 2.28 3.52 3-clause ratio ? 4.501 20 Resolution proof complexity of mixed random formulas Theorem: [Achlioptas, B, Molloy 01] For any constants r21 and r3 0 random formulas with r2n 2-clauses and r3n 3-clauses need exponential-size resolution proofs almost surely Extends [Chvatal-Szemeredi 88] 21 Long-running DPLL Executions and Proof Complexity Residual formula at each node is a mix of 2- and 3-clauses Residual formula at is unsatisfiable 2 rn Every resolution Algorithm’s proof of unsatisfiability is exponentially long 22 Trajectory on 3-CNF 2-clause ratio 1 UC Algorithm Trajectory 0 Provably UNSAT & Hard Provably SAT & Easy 3.52 3.81 4.267 4.51 3-clause ratio 23 Exponential lower bounds for 3-CNF formulas below ratio 4.267 Corollary For almost all 3-CNF formulas, above ratio 3.81 UC takes exponential time 3.83 UCwm takes exponential time 4.01 GUC takes exponential time Exponential lower bound below the proven k-CNF threshold [Achlioptas, Peres 02] rk* 2k ln 2 – (k+4)/2 [Achlioptas, B, Molloy 04] For k 4, for almost all k-CNF formulas UC takes exponential time above ratio c 2k/k Note These formulas have huge numbers of satisfying assignments (more than 2 (1-) n out of a possible 2n) but still are hard 25 Open Problem Closing gap for unsatisfiability of mixed formulas would yield sharp threshold behavior for each algorithm A Below rA algorithm runs in linear time Above rA algorithm requires exponential time Conjecture: no polynomial time selection heuristic will work up to the threshold rA r3* for all algorithms A 26 Outline Intro to proof complexity Proof complexity and complete SAT algs Review Resolution/DPLL connection Application to random k-SAT Proof Complexity and DPLL extensions Clause learning Caching Proof system survey 27 Clause Learning At each backtrack point in DPLL search add new clauses to the input formula that correspond to known causes of failure of the search Critical to the good behavior of the best current algorithms What is the power of clause learning? Intuitively: it makes DPLL trees DAG-like regular resolution? general resolution? 28 Conflict Graph Known Clauses (p q a) ( a b t) (t x1) (t x2) (t x3) (x1 x2 x3 y) (x2 y) Current decisions p false q false b true 1-UIP scheme (t) x1 p y q a t b x2 false y x3 Decision scheme (p q b) 29 Proof complexity and clause learning Clause learning is no stronger than Resolution Implication graph yields resolution derivation of each learned clause There are examples where clause learning works in polynomial time but any Regular Resolution proof is exponential [Sabharwal, B, Kautz 04] Clause learning can sometimes be much stronger than Regular Resolution 30 Proof complexity and clause learning Separation is true for any proper subsystem R of Resolution for which refuting F|x is no harder than F Clause learning does not have this property unless it is as powerful as resolution Idea: Take formula F easy for Resolution but hard for R and create new formula F’ that is F plus clauses for (CpC) for each C in Resolution proof of F where each pC is a new propositional variable Setting each pC to true yields original F but branching on sequence of pC allows sequence of clauses C to be learned 31 Still Open Can clause learning do everything regular resolution can?* Is clause learning as powerful as general resolution? * Note: Alan Van Gelder has pointed out that if one returns learned clauses up the tree and resolves them while backtracking then clause learning does efficiently simulate regular resolution 32 Proof Complexity and Formula Caching Formula Caching: Memoization Cache residual formulas that have been already found to be unsatisfiable Do not make recursive call if the current residual formula is already cached 33 DPLL DPLL(F) if F is empty report satisfiable and halt if F contains the empty clause L backtrack else select a literal x DPLL(F|x) DPLL(F|x) 34 Basic Formula Caching FC(F,L) if F is empty report satisfiable and halt if F contains the empty clause L or FL return else choose a literal x FC(F|x,L) FC(F|x,L) Start with FC(F,) add F to L 35 Proof Complexity of Formula Caching [B, Impagliazzo, Pitassi, Segerlind 03] Basic Formula Caching can exponentially improve DPLL Basic Formula Caching is not as powerful as regular resolution but some of its natural extensions are These natural extensions can be exponentially more powerful than general resolution or even Res(k) for any k ! Res(k) is like resolution but uses k-DNFs instead of clauses 36 Extensions of Formula Caching Don’t just check membership in the cache F is trivially unsatisfiable given G iff every clause of G contains some clause of F Record the reasons why residual formulas are known to be unsatisfiable and check the reasons rather than the formulas 37 More General Formula Caching FC*(F,L) if F is empty report satisfiable and halt if F is trivially unsatisfiable given L add F to L return else choose a literal x FC*(F|x,L) FC*(F|x,L) add F to L 38 Adding Reasons FCreason(F,L) if F is empty report satisfiable and halt if F is trivially unsatisfiable based on GL add F to L return(G) else choose a literal x GFCreason(F|x,L) HFCreason(F|x,L) J (xG) (xH) add F,J to L return(J) 39 Combining Clause Learning and Formula Caching Cachet system used to count # of satisfying assignments [Sang, Bacchus, B., Kautz, Pitassi 04] Built on zchaff’s clause learning Uses decomposition of residual formula into components Caches components rather than the whole formula Subtle interactions 40 Outline Intro to proof complexity Proof complexity and complete SAT algs Resolution/DPLL review Application to random k-SAT DPLL extensions Clause learning Caching Proof system survey 41 Proof Systems Hierarchy ZFC Ext-Frege Frege LS AC0-Frege Cutting Planes PCR Res(k) Resolution DPLL Polynomial Calculus Nullstellensatz Truth Tables 42 AC0-Frege Constant-depth Frege (AC0-Frege) Like Frege but formulas in proof only have constant # of alternations between and Resolution is depth-1 Frege 43 Some Known Hard Problems for Resolution Exact/Modular Counting Pigeonhole principle: n+1 pigeons can’t fit into n pigeonholes without getting cosy with each other Parity Principle: e.g. can’t pair up an odd number of people Tseitin Tautologies These problems are also hard for AC0-Frege Random Unsatisfiable k-CNF Graph k-colorability Existence of k-cliques in graphs 44 Cutting Planes/PseudoBoolean Introduced to relate integer and linear programming [Gomory 59, Chvatal 73]: Objects are linear integer inequalities Clause (x1 x2 x3) becomes inequality x1+(1-x2)+x3 1 Add inequalities xi 0 and 1-xi 0 Goal: derive 0 1 45 Cutting Planes rules addition: a1x1 + ... + anxn A b1x1 + ... + bnxn B (a1+b1)x1+...+(an+bn)xn A+B multiplication by positive integer: a1x1 + ... + anxn A ca1x1 + ... + canxn cA Division by positive integer: ca1x + ... + canxn B a1x1 + ... + anxn B/c 46 Limitations of Cutting Planes At least as good as resolution Can also handle pigeonhole principle parity principle But can’t handle counting when it is a derived property Clique-Coloring A graph can’t have a k-clique and be k-1-colorable Tseitin tautologies still open 47 Proof Systems Hierarchy ZFC Ext-Frege Frege LS AC0-Frege Cutting Planes PCR Res(k) Resolution DPLL Polynomial Calculus Nullstellensatz Truth Tables 48 Hilbert’s Nullstellensatz System of polynomials Q1(x1,…,xn)=0,…,Qm(x1,…,xn)=0 over field K has no solution in any extension field of K there exist polynomials P1(x1,…,xn),…,Pm(x1,…,xn) in K[x1,…,xn] s.t. m PQ i i 1 i=1 49 Nullstellensatz proof system C Clause (x1 x2 x3) becomes equation (1-x1)x2(1-x3)=0 QC Add equations xi2-xi =0 for each variable Guarantees only 0-1 solutions A proof is polynomials P1,…, Pm+n proving unsatisfiability: i.e. such that m j=1 n 2 + (x P jQ Cj Pm+i - x) 1 i=1 50 Polynomial Calculus Similar to Nullstellensatz except: Begin with Q1,…,Qm+n as before Given polynomials R and S can infer aR + bS for any a, b in K xiR Derive constant polynomial 1 Degree = maximum degree of polynomial appearing in the proof Can find proof of degree d in time nO(d) using Groebner basis-like algorithm (linear algebra) 51 PCR = PC + Resolution Two variables x and x’ for each proposition x x’ stands for x include equations x+x’-1=0, x2-x=0, and (x’)2-x’=0 Translate (x1 x2 x3) as x’1 x2 x’3=0 Same proof rules and proof search as polynomial calculus Efficiently simulates resolution: clausemonomial Degree-size relationship for PCR is same as widthsize relationship of resolution 52 Lovasz-Schrijver (LS) Proofs Linear inequalities Steps: Lift to get degree 2 inequalities E.g. multiply inequalities Project Combine degree 2 inequalities plus x2=x to cancel out degree 2 terms Captures properties of semi-definite programming 53 Proof Systems Hierarchy ZFC Ext-Frege Frege LS AC0-Frege Cutting Planes PCR Res(k) Resolution DPLL Polynomial Calculus Nullstellensatz Truth Tables 54 Conclusions Exchange of ideas between proof complexity and SAT algorithms can be fruitful Better understanding of existing algorithms Source of ideas for new algorithms and proof systems Potential to branch out beyond DPLL derivatives Wide range of proof systems to explore 55 Thank You Thank you to Toby Walsh Fahiem Bacchus Ian Miguel 56