Practical SAT Solving: Achievements, Problems, Opportunities •! Part 1. Modern resolution based SATsolvers. What are they “made of”? Why do they work so well? •! Part 2. Linear size resolution proofs that can not be found efficiently (equivalence checking of combinational circuits) •! Part 3. Testing the satisfiability of a formula by computing a stable set of points. Two Kinds of Complexity Given a proof system S, proving unsatisfiability of CNFs of a class K may be hard for two reasons •! The shortest proofs for formulas of K in S have exponential length (pigeon-hole formulas for RPS) •! Formulas of K have short proofs in S, but there is no algorithm to find them (S is not automatizable) In this part, we consider the second kind of complexity on the example of equivalence checking (EC). We introduce an important class of EC formulas that have short resolution proofs that most probably can not be found efficiently by a deterministic algorithm. Summary (we are here !) •! Part 1. •! Part 2. •!Introduction •!Equivalence Checking (EC) •!Circuits with a Common Specification (CS) •!Toggle Equivalence •!EC of Circuits with CS •!Conclusions •! Part 3. Seperation between Deterministic and Non-Deteriministic SAT-Algorithms “Separation” of deterministic and non-deterministic algorithms for satisfiable formulas is obvious. There is a linear complexity non-deterministic algorithm while no efficient deterministic algorithm exists (unless P = NP). For unsatisfiable formulas such separation is not “trivial”. Many classes of formulas having short resolution proofs of unsatisfiability can also be easily solved by a deterministic algorithm. This applies for example to 2-CNF and Horn formulas. Summary (we are here !) A Very Generic Reason to Study EC EC is “simpler” than SAT •! Part 1. •! Part 2. •!Introduction •!Equivalence Checking (EC) •!Circuits with a Common Specification (CS) •!Toggle Equivalence •!EC of Circuits with CS •!Conclusions •! Part 3. NF N" ….. NF N" …. ….. Two Practical Reasons for Studying EC A CNF formula F can be represented as a circuit NF . EC of and NF and N" (the latter being a constant 0) is hard because they are very different “structurally”. 0 0 ….. ….. Miter xk # xm xm xk “External” Equivalence Checking. EC of combinational circuits is a very important part of the design flow for electronic circuits. (For example, one may need to check if the original circuit is equivalent to one modified by a designer manually or optimized by a second party logic synthesis tool. “Internal” Equivalence Checking. When a circuit N is optimized, for every logic synthesis transformation of N there has to be a “proof” that the circuits before and after this transformation are functionally equivalent. z=y1 # y2 0 1 0 0 1 1 1 0 The circuit on the left (called a miter) is satisfiable if and only if N1 and N2 are inequi-valent. Summary (we are here !) Circuits with a Common Specification (first definition) Circuit N2 Circuit N1 •! Part 1. •! Part 2. •!Introduction •!Equivalence Checking (EC) •!Circuits with a Common Specification (CS) •!Toggle Equivalence •!EC of Circuits with CS •!Conclusions •! Part 3. G3 G1 x1 x2 G2 x3 x4 x5 Circuit M of multi-valued gates Specification and Implementation vi are “coding variables” Circuits with a Common Specification (another definition equivalent to first) Circuit N1 N3 N1 Z=Gi(X,Y) q(Z)=I(Gi)(q(X),q(Y)) q(X),q(Y),q(Z) are encodings Circuits N1 and N2 have a common specification (CS) if there is a circuit M of multi-valued gates such that N1 and N2 are just different implementations of M. Subcircuits N11,N12, N13 and N21,N22,N23 above are two different implementations of gates G1,G2,G3 (on the left). N2 x1 x2 x3 x4 x5 Comminication graph (acyclic) Circuit N2 Circuits N1 and N2 have a CS if •! Subcircuits N11,N12, N13 and N21,N22,N23 are connected “in the same way” (i.e. they have the same communication graph). •!N1i, N2i,i=1,2,3 are toggle equivalent Toggle Equivalence Summary (we are here !) Multi-output circuits N1 and N2 are toggle equivalent if N1(x)$N1(x*) % N2(x)$N2(x*) •! Part 1. •! Part 2. •!Introduction •!Equivalence Checking (EC) •!Circuits with a Common Specification (CS) •!Toggle Equivalence •!EC of Circuits with CS •!Conclusions •! Part 3. y1 N1 ym z1 … N1 N2 … x1 zk xn y*1,..,y*m y!1,..,y!m x1 … xn z*1,..,z*k z!1,..,z!k y"1,..,y"m Let P1 be the partition of the set of input assignments of N 1 into subsets for which N1 produces the same output assignment. Let P2 be the same partition for N2. z"1,..,z"k Input space partitions P1 and P2 If N1 and N2 are toggle equivalent, then P1 and P2 have to be identical. If P1$ P2, then there are two points (i.e. input assignments) x! and x" that are in the same subset of P1 and in two different subsets of P2 or vice versa. So either N2 toggles while N1 does not or vice versa. zk … If N1 and N2 are toggle equivalent, then there is a one-to-one mapping K between outputs assignments of N1 and N2. N2 … xn x1 x1 … xn That is N1(x) = K(N2(x)) and N2(x) = K-1(N1(x)) If N1 and N2 are toggle equivalent and have only one output, they are either functionally equivalent or complements of each other. Sketch of Proof y1 … z1 ym … Finding Output Relation y1 … ym z1 … N1 x1 zk N2 f*(x1,..,xi-1,,xi+1,..,xn) = f(x1,..,xi=0,..,xn) & f(x1,..,xi=1,..,xn) … … xn x1 xn Let H be represented as a CNF formula. Then a CNF formula H* representing the relation of outputs of N1 and N2 can be obtained from H by resolving out all the variables of X,I1,I2. Denote by H(X,I1,I2,Y,Z) the function sat(N1) ' sat(N2). Here sat(Ni) is the CNF formula specifying Ni. Let H*(Y,Z) be obtained from H by existential quantification of the variables of X,I1,I2. Then H*(y,z)=1 iff there is an input vector x such that N1(x)=y and N2(x)=z. Toggle Equivalence of Circuits with Different Inputs Bijective Function Relation Let X and Y be sets of Boolean variables and F(X,Y) be a Boolean function. F(X,Y) is a bijective function relation iff there is a subset Ax ( {0,1}|X| and BY ( {0,1}|Y| such that •! |AX| = |BY| •! F(X,Y) specifies a bijective mapping M: AX ) BY. Namely, F(x,y)=1 iff x * AX, y * BY and y=M(x). Testing if F(X,Y) is a bijective function relation needs two SAT checks: R1 = F(X,Y) ' F(X*,Y*) ' EQ(X,X*) ' NEQ(Y,Y*) R2 = F(X,Y) ' F(X*,Y*) ' NEQ(X,X*) ' EQ(Y,Y*) If both R1 and R2 are unsatisfiable, then F(X,Y) is a bijective function relation Summary (we are here !) y1 … ym z1 Suppose N1 and N2 have different input variables that are related by a bijective function relation Bf(X,P). N2 N1 x1 … zk … p1 xn … pd Bf(x,p) = 1 iff M(x) = p where M is a bijective function Bf(x,p)=1 We assume that only pairs of input vectors x,p for which Bf(x,p)=1 are “legal”. Then one can extend definition of toggle equivalence. Multi-output circuits N1 and N2 are toggle equivalent if for any pair of vectors x,x* it is true that N1(x)$N1(x*) % N2(p)$N2(p*) where p=M(x) and p*=M(x*). An Example of EC of Circuits with a known CS N1 •! Part 1. •! Part 2. •!Introduction •!Equivalence Checking (EC) •!Circuits with a Common Specification (CS) •!Toggle Equivalence •!EC of Circuits with CS •!Conclusions •! Part 3. N2 H 1* N 23 N 13 N11 N 12 N 21 N 22 …. …. …. …. H1=Sat(N11) ' Sat(N21) ) H1*(out(N11),out(N21)) ) Is H1* a bijective function relation? H2=Sat(N12) ' Sat(N22) ) H2*(out(N12),out(N22)) ) Is H2* a bijective function relation? H3=Sat(N11) ' Sat(N21) ' H1* ' H2* ) H3*(out(N11),out(N21)) ) Is H3* a bijective function relation? EC of Circuits with a Known CS (pseudocode) Spec(N1) = {N11,..,N1k}, Spec(N2)={N21,…,N2k} 1 2 3 4 5 6 7 8 EC(N1, N2, Spec(N1),Spec(N2)) { // check if Spec(Ni) is topological (i.e. that its comm. graph is acyclic) if (topol_spec(N1,Spec(N1)) == ‘no’) || topol_spec(N2,Spec(N2)) == ‘no’)) return(‘CS_check_failure’); // check isormophism of communication graphs if (graph_isomorphism (N1,N2,Spec(N1),Spec(N2)) == ‘no’) return(‘CS_check_failure’); // main loop for (i=1; i <= k ; i++) { Dinp(N1i, N2i )= constraint_function(N1,N2,i); Dout(N1i, N2i) = exist_quantify(N1i,N2i, Dinp); if ( bijective_function_rel(Dout)) == ‘no’) return(‘CS_check_failure’);} // last subcircuits are functionally equivalent? 10 if (Dout(N1k, N2k) is ‘equivalence_function’) return(‘equivalent’); 11 else return(‘inequivalent’);} 9 EC by Deterministic Algorithm If CS Is Unknown A formula F of K(p) can be solved by the following trivial algorithm. This algorithm is to generate resolvents of length no more than 2*p literals. (If a resolvent becomes longer than 2*p it is discarded). The complexity of this algorithm is |F|2p where |F| is the length of formula F. Although, this algorithm is polynomial for a fixed p, its complexity is huge. This complexity is due to the fact that if Spec(N1) and Spec(N2) are unknown, this algorithm just resolves all clauses of F that can be resolved. Complexity in Resolution Let Spec(N1) = {N11,..,N1k}, Spec(N2)={N21,…,N2k} Let pa be the size of the largest subcircuit Nai, a=1,2,i=1,..,k. We will call pa the granularity of specification Spec(Na). Denote by p max(p1,p2). We will call p the granularity of CS of N1 and N2. The procedure we described is linear in the number of subcircuits k and exponential in granularity p of CS. The exponentiality in p is because one needs to compute the relation between outputs of N1i, N2i and check if it specifies a bijective funciton. Let us K(p) be the class of CNF formulas describing equivalence checking of circuits with a CS of granularity bounded by a constant p. Formulas of K(p) have linear size resolution proofs. Can We Do Better? It is very unlikely that there is an efficient algorithm for solving formulas of K(p). This claim is based on the following two assumptions. a)! Let F be a formula of K(p) describing equivalence checking of circuits N1 and N2 with specifications Spec(N1),Spec(N2). Finding a short proof that F is unsatisfiable would mean finding Spec(N1),Spec(N2) b)! Given N1 and N2, finding their CS of granularity +p is hard. (Ideally, one should show that the problem of finding a CS of granularity + p for N1 and N2 is NP-complete or give some other formal estimate of the complexity of this problem.) Summary (we are here !) •! Part 1. •! Part 2. •!Introduction •!Equivalence Checking (EC) •!Circuits with a Common Specification (CS) •!Toggle Equivalence •!EC of Circuits with CS •!Conclusions •! Part 3. Making SAT-solver More “Intelligent” In the case of EC of circuits N1 and N2 with a CS, giving a SAT-solver a little “tip” in the form of partitions of N1 and N2 into subcircuits makes huge difference. Practitioners usually try to build an efficient blackbox SAT-solver (that should not exist unless P = NP). A more reasonable approach is to explore various possibilities to make a SAT-solver more receptive to information supplied by the user. This should make a SAT-solver more intelligent. Power of SAT-Solver Is Very Limited We introduced a class of formulas where there is a sharp separation between the complexity of non-deterministic algorithm (that knows a CS of circuits N1 and N2) and a deterministic algorithm (that does know CS). Sadly, this class of formulas is not a mathemtical curiousity. On the contrary, it is very important from a practical point of view. This result emphasizes how limited the power of a “black-box” SAT solver is. Practical SAT Solving: Achievements, Problems, Opportunities •! Part 1. Modern resolution based SATsolvers. What are they “made of”? Why do they work so well? •! Part 2. Linear size resolution proofs that can not be found efficiently (equivalence checking of combinational circuits) •! Part 3. Testing the satisfiability of a formula by computing a stable set of points. Summary (we are here !) •! Part 1. •! Part 2. •! Part 3. Testing the satisfiability of a formula by computing a stable set of points. •!Stable Set of Points (SSP) •!Computing SSPs in points •!Computing SSPs in clusters •!Application to Symmetric Formulas Taking into Account Formula Specifics Current SAT-solvers are not very good at taking into account formula specifics. For example, if a formula has symmery, a resolution based SAT-algorithm taking into account this symmetry looks ackward. The notion of a Stable Set of Points (SSP) is a promising way to make use of particular properties of a CNF formula. SSPs have the following interesting features: •!An SSP of a formula is an algorithm-independent “characteristic” of this formula. •! SSPs can take into account formula specifics through clustering of points •!Building an SSP of a formula can be viewed as proving unsatisfiability by induction. Random Walk Papadimitriou 1991, Selman, Kautz 1992 flip flip flip flip p1 ) p2 ) … ) pk ) s C1 ) C2 ) … ) Ck ) , Any solution the closest to p1 is reachable from p1 in n steps at most Any clause falsifying pi can be used as a “stepping stone” Let F(x1,…,xn) be a satisfiable CNF formula. Let C= x1& x5 & x7 be a clause of F. Let p=(x1=0,…., x5=0, …,x7=0…) be a complete assignment falsifying C. Since F is satisfiable, there is a satisfying assignment s. Since s satisfies C, then either x1=1 or x5=1 or x7=1 in s. So by flipping x1,x5 or x7 point p becomes closer to s. 1-Neighborhood Let C be the clause ~x1& x10 & x50 of a CNF F(x1,…,xn). Let p=(x1=1,..,x10=0,..,x50=0) be a point that falsifies C. Let the following three points p! =(x1=0,..,x10=0,..,x50=0), p " =(x1=0,..,x10=1,..,x50=0), p!!! =(x1=1,..,x10=1,..,x50=1) be obtained from p by flipping the value of x1, x10 or x50. The set {p!,p", p!!! } is called the 1-neighborhood of p with respect to clause C. It is denoted by Nbhd(p,C). In general, given a point p falsifying clause C, Nbhd(p,C) is the set of the points that are at Hamming distance 1 from p and that satisfy C. Stable Set of Points (trivial example) Let F be a CNF formula of, say, a million variables that contains the following three clauses: Stable Set of Points (example cont.) x1=0, p1,C1 x100=0 Denote by P the set {p1,p2,p3,p4}. Note that C1(p1)=0, C2(p2)=0, C3(p3)=0, C3(p4)=0. Note that Nbhd(p1,C1), Nbhd(p2,C2), Nbhd(p3,C3),Nbhd(p4,C3) are subsets of P. For example, Nbhd(p1,C1) = {p2,p3}. Stable Set of Points (definition) Denote by Z(F) the set of all points p * {0,1}n for which F(p)=0. Let P be a subset of Z(F). We will call a mapping G: P ) F a transport function if for any p* P, C(p) = 0 if C=G(p). Set P ( Z(F) is stable with respect to F and mapping G:P ) F if for any pi* P, Nbhd(pi,C) ( P where C = G(pi) F is unsatisfiable if and only if there is a set of points P and transport function G such that P is a stable set of points (SSP) with respect to F and G. p2,C2 x1=1, x100=0 x100 C1 = x1 & x100, C2 = ~x1 & x100, C3= ~x100 , Let p1=(x1=0,..,x100=0,..) be a point falsifying C1. Let p2 = (x1=1,.., x100=0,..), p3 = (x1=0,..,x100=1,..), p4 = (x1=1,..,x100=1, ..) be points obtained from p1 by flipping the value of x1 and/or x100. x1 x1=0, x100=1 p3,C3 p4,C3 x1=1, x100=1 Suppose that we try to reach a solution by random walk starting from a point of P and using the clause G(pi) as a “stepping stone” when we reach pi. That is next point is picked from the set Nbhd(pi,G(pi)) C1 = x1 & x100, C2 = ~x1 & x100, C3= ~x100 , Denote by G the following function mapping P={p1,p2,p3,p4} to {C1,C2,C3}: G(p1) = C1, G(p2)=C2, G(p3) = C3, G(p4)=C3 Since Nbhd(pi,G(pi))( P, such a random walk never reaches a point other than those in P. Then the formula is unsatisfiable. Summary (we are here !) •! Part 1. •! Part 2. •! Part 3. Testing the satisfiability of a formula by computing a stable set of points. •!Stable Set of Points (SSP) •!Computing SSPs in points •!Computing SSPs in clusters •!Application to Symmetric Formulas A Trivial Algorithm for Computing SSP 1. Generate a starting point p*, Boundary={p*}, Body=, 2. Pick a point p * Boundary, Boundary=Boundary \ {p} 3. Find the set X of clauses falsified by p. If X=, then p is a solution. Stop. 4. Pick a clause C * X . 5. Nbhd(p,C) = Nbhd(p,C) \ (Boundary - Body) Boundary=Boundary - Nbhd(p,C) Body=Body - {p} 6. If Boundary = , then Body is an SSP. Stop. 7. Go to step 2. Summary (we are here !) SSPs for Random CNF Formulas Number of variables n |SSP| |SSP| / 2n % 10 430 41.97 11 827 40.39 12 1,491 36.41 17 30,381 23.18 22 736,329 17.55 23 1,370,890 16.34 Computing SSPs more efficiently Computing an SSP of a CNF formula F “point by point” is too inefficient. •! Part 1. •! Part 2. •! Part 3. Testing the satisfiability of a formula by computing a stable set of points. •!Stable Set of Points (SSP) •!Computing SSPs in points •!Computing SSPs in clusters •!Application to Symmetric Formulas •!Conclusions There are a few ways to “speed up” the computation of SSPs dramatically. One of them is to compute an SSP in clusters of points. Clusters may be represented implicitly. So instead of an SSP one builds a stable set of clusters. This approach has the following three advantages. •!Many points are processed at once. Since a cluster may be represented implicitly, it may be of exponential size. •!By picking the “shape” of clusters one can input information about subtle structural properties of formula F. •! One can build complex hierarchies of stable objects like a stable set of clusters of clusters and so on. A Stable Set of Clusters Let F be a CNF formula. Let D ( Z(F) and G be a transport funciton G: D ) F. Denote by Nbhd(D,G) the union of 1neighborhoods Nbhd(p,G(p)) over all the points of D. Let D1,.., Dm be subsets of Z(F). (We will call subset Di a cluster). Let Gi be a transport function Di ) F. We will call D1,..,Dm a stable set of clusters (SSC) with respect to F and transport functions G1,..,Gm if for every cluster Di, i=1,..,m Nbhd(Di,Gi) ( D1 - .. - Dm If P is an SSP of F then any set of subsets D1,..,Dm such that D1 - .. - Dm = P is an SSC. And vice versa if D1,..,Dn is an SSC , then the set D1 - ..-Dm is an SSP Summary (we are here !) •! Part 1. •! Part 2. •! Part 3. Testing the satisfiability of a formula by computing a stable set of points. •!Stable Set of Points (SSP) •!Computing SSPs in points •!Computing SSPs in clusters •!Application to Symmetric Formulas Building SSCs efficiently The definition of an SSC we gave above is too “loose”. To compute SSCs efficiently we impose three more informal requirements on clusters D1,..,Dm forming an SSC. •!Each cluster Di has to have a “compact representation” that can be handled efficiently. •!Computation of 1-neighborhood Nbhd(Di,Gi) should be “easy” for every cluster Di •!There should exist an efficient procedure for testing that Nbhd(Di,Gi) ( D1 - .. - Dm. To facilitate efficiency one may require, for example, that Nbhd(Di,Gi) = Dk or Nbhd(Di,Gi) ( Dk. Symmetric CNF Formulas Let F(x1,..,xn) be a CNF formula. Denote by . a permutation preserving F i.e. .(F)=F. Denote by Gr(F) the set of all permutations under which F is symmetric. This set forms a group. Let F=(x1 & x2 & ~x3) ' (~x1 & x2 & x3) is symmetric under the permutation swapping x1 and x3. Point p * {0,1}n is symmetric to p* if .(p*)= p for some . * Gr(F). Denote by D(p) the orbit of p i.e. the set of all points that can be obtained from p by permutations of Gr(F). From definition of a group it follows that if p1, p2 * {0,1}n are symmetric with respect to Gr(F), then D(p1)=D(p2). If p1 and p2 are not symmetric, then D(p1) / D(p2)=,. So the set {0,1}n can be partitioned into subsets of D1,..,Dk, each subset Di being an orbit of a point p. 1-Neighborhood of an Orbit Let Di be an orbit such that Di ( Z(F). (If p1,p2 * Di, then F(p1)=F(p2). So checking whether Di is a subset of Z(F) is trivial. If F(p)=0 for a point p of Di then Di ( Z(F).) Let us define a transport function Gi:Di)F in the following way. Let us pick a point p of Di and select a clause C of F such that C(p) = 0. Let p* be a point of Di such that p*= .(p). Then we pick the clause C* equal to .(C) as the value Gi(p*). Since .(F)=F then the clause .(C) is in F. Besides, it is not hard to show that C(p) = .(C)(.(p))= C*(p*)=0. Let p* Di and C=Gi(p). Let Nbhd(p,C) be equal to {p1,..,pk}. It is not hard to show that Nbhd(Di,Gi) = D(p1) - .. - D(pk). Comments to The Procedure For Building SSC The procedure above can be viewed as building an SSC. Each cluster Di is represented by a point p that is currently in Boundary - Body. (That is Di is an orbit of a point p from Boundary or Body.) When we check if a point of Nbhd(p,C) is symmetric to some other point of Boundary - Body (two modified lines of paragraph 5) we want to make sure that we do not add a cluster (i.e. an orbit) that has been already visited. Let Body be equal to {p1,…,pm} when the set Boundary is empty. Then D(p1),..,D(pm) is an SSC for F and D(p1)-…-D(pm) is an SSP for F. Building SSC for Symmetric CNF 1. Generate a starting point p*, Boundary={p*}, Body=, 2. Pick a point p * Boundary, Boundary=Boundary \ {p} 3. Find the set X of clauses falsified by p. If X=, then p is a solution. Stop. 4. Pick a clause C * X . 5. Remove from Nbhd(p,C) every point p* that is // Modif. line symmetric to a point of Boundary - Body // Modif. line Boundary=Boundary - Nbhd(p,C) Body=Body - {p} 6. If Boundary = , then Body specifies an SSC. Stop // Modified line 7. Go to step 2. Are Orbits “Simple” Clusters? Orbits satisfy the three “simplicity” requirements to clusters we mentioned above modulo complexity of checking if two points are symmetric. •! Each cluster D(pi) is easy to represent because it is specified by one point (i.e. point pi). The rest of the points of D(pi) are symmetric to pi. So each cluster D(pi) is represented implicitly. •! The set Nbhd(D(pi),Gi) is easily computable. One just needs to compute the set Nbhd(pi,Gi(pi)). The k points of Nbhd(pi,Gi(pi)) represent the k orbits whose union is equal to Nbhd(D(pi),Gi). •! To check if Nbhd(D(pi),Gi) is in D(p1) - .. - D(pm) one needs to check if every point p* from Nbhd(pi,Gi(pi)) is symmetric to a point from {p1,…,pm}. In general, this operation may have non-trivial complexity. Pigeon-Hole Formulas CNF formula PH(m,n) describes the problem of allocating m objects (pigeons) in n containers (holes). No two pigeons may share the same hole. Let Boolean variable phij be equal to 1 if i-th pigeon is in j-th hole. Otherwise phij = 0. PH(m,n) consists of clauses phi1& .. & phin , i=1,..,m requiring that i-th pigeon has to be in at least one hole and clauses ~phrj& ~phsj (r$s) r=1,..,m,s=1,..,m,j=1,..,n requiring that two pigeons may not share the same hole. For m > n , PH(m,n) are unsatisfiable. Haken showed that formulas PH(n+1,n) have only exponential proofs in RPS. Formulas PH(m,n) are highly symmetric. Gr(PH(m,n)) consists of all permutations .1•.2 where .1 (or .2) is a permutation of pigeons (respectively holes). Conclusions to Parts 1,2,3 •!The potential of resolution based SAT-solvers has not been exausted. Search for practical methods for finding short proofs may still bring surprises. •!A SAT-solver (or, more generally, a computer) most likely can not recover high-rank relations (like toggle equvialence of circuits). So building SAT-solvers that are receptive to structural information provided by the user is very important. •!Stable sets of points may provide a framework for building such receptive SAT-solvers. SSC for Pigeon-Hole Formulas It can be shown that for PH(n+1,n) formula there is an SSC of 2*n + 1 clusters. Some clusters have exponential size (so the corresponding SSP is exponential). The seven points below specify the seven clusters for the case n = 3. pg1 pg2 pg3 pg4 h1 h2 h3