Logic & AR Summer School, 2002
Constraints and Search
Toby Walsh
Cork Constraint Computation Centre (4C)
tw@4c.ucc.ie
Constraint satisfaction
• Constraint satisfaction problem (CSP) is a triple
<V,D,C> where:
– V is set of variables
– Each X in V has set of values, D_X
• Usually assume finite domain
• {true,false}, {red,blue,green}, [0,10], …
– C is set of constraints
Goal: find assignment of values to variables to satisfy all the
constraints
Constraint solver
• Tree search
– Assign value to variable
– Deduce values that must be removed from future/unassigned
variables
• Constraint propagation
– If any future variable has no values, backtrack else repeat
• Number of choices
– Variable to assign next, value to assign
Some important refinements like nogood learning, non-chronological
backtracking, …
Constraint propagation
• Enfrocing arc-consistency (AC)
– A binary constraint r(X1,X2) is AC iff
for every value for X1, there is a consistent value (often called support)
for X2 and vice versa
E.g. With 0/1 domains and the constraint X1 =/= X2
Value 0 for X1 is supported by value 1 for X2
Value 1 for X1 is supported by value 0 for X2
…
– A problem is AC iff every constraint is AC
Tree search
•
•
•
•
•
Backtracking (BT)
Forward checking (FC)
Maintaining arc-consistency (MAC)
Limited discrepany search (LDS)
Non-chronological backtracking & learning
Modelling case study: orthogonal
Latin squares
Or constraint programming isn’t
purely declarative!
Modelling decisions
• Many different ways to model even simple
problems
– It’s not pure declarative programming!
• Combining models can be effective
– Channel between models
• Need additional constraints
– Symmetry breaking
– Implied (but logically) redundant
Orthogonal Latin squares
• Find a pair of Latin
squares
– Every cell has a different
pair of elements
• Generalized form:
– Find a set of m Latin
squares
– Each possible pair is
orthogonal
Orthogonal Latin squares
1234
2143
3412
4321
11
23
34
42
22
14
43
31
1234
3412
4321
2143
33
41
12
24
44
32
21
13
• Two 4 by 4 Latin
squares
• No pair is repeated
History of (orthogonal) Latin
squares
• Introduced by Euler in 1783
– Also called Graeco-Latin or Euler squares
• No orthogonal Latin square of order 2
– There are only 2 (non)-isomorphic Latin
squares of order 2 and they are not orthogonal
History of (orthogonal) Latin
squares
• Euler conjectured in 1783 that there are no orthogonal
Latin squares of order 4n+2
– Constructions exist for 4n and for 2n+1
– Took till 1900 to show conjecture for n=1
– Took till 1960 to show false for all n>1
• 6 by 6 problem also known as the 36 officer problem
“… Can a delegation of six regiments, each of which sends a colonel,
a lieutenant-colonel, a major, a captain, a lieutenant, and a sublieutenant be arranged in a regular 6 by 6 array such that no row or
column duplicates a rank or a regiment?”
More background
• Lam’s problem
– Existence of finite projective plane of order 10
– Equivalent to set of 9 mutually orthogonal Latin
squares of order 10
– In 1989, this was shown not to be possible after 2000
hours on a Cray (and some major maths)
• Orthogonal Latin squares are used in experimental
design
– To ensure no dependency between independent
variables
A simple 0/1 model
• Suitable for integer programming
– Xijkl = 1 if pair (i,j) is in row k column l, 0 otherwise
– Avoiding advice never to use more than 3 subscripts!
• Constraints
– Each row contains one number in each square
Sum_jl Xijkl = 1
Sum_il Xijkl = 1
– Each col contains one number in each square
Sum_jk Xijkl = 1
Sum_ik Xijkl = 1
A simple 0/1 model
• Additional constraints
– Every pair of numbers occurs exactly once
Sum_kl Xijkl = 1
– Every cell contains exactly one pair of numbers
Sum_ij Xijkl = 1
Is there any symmetry?
Symmetry removal
• Important for solving CSPs
– Especially for proofs of optimality?
• Orthogonal Latin square has lots of
symmetry
– Permute the rows
– Permute the cols
– Permute the numbers 1 to n in each square
• How can we eliminate such symmetry?
Symmetry removal
• Fix first row
11 22 33 …
• Fix first column
11
23
32
..
• Eliminates all this symmetry?
What about a CSP model?
• Exploit large finite domains possible in CSPs
– Reduce number of variables
– O(n^4) -> ?
• Exploit non-binary constraints
– Problem states that squares contain pairs that are all
different
– All-different is a non-binary constraint our solvers can
reason with efficiently
CSP model
• 2 sets of variables
– Skl = i if the 1st element in row k col l is i
– Tkl = j if the 2nd element in row k col l is j
• How do we specify all pairs are different?
– All distinct (k,l), (k’,l’)
if Skl = i and Tkl = j then Sk’l’=/ i or Tk’l’ =/ j
O(n^4) loose constraints, little constraint propagation!
What can we do?
CSP model
• Introduce auxiliary variables
– Fewer constraints, O(n^2)
– Tightens constraint graph => more propagation
– Pkl = i*n + j if row k col l contains the pair i,j
• Constraints
– 2n all-different constraints on Skl, and on Tkl
– All-different constraint on Pkl
– Channelling constraint to link Pkl to Skl and Tkl
CSP model v O/1 model
• CSP model
– 3n^2 variables
– Domains of size n, n
and n^2+n
– O(n^2) constraints
– Large and tight nonbinary constraints
• 0/1 model
–
–
–
–
n^4 variables
Domains of size 2
O(n^4) constraints
Loose but linear
constraints
• Use IP solver!
Solving choices for CSP model
• Variables to assign
– Skl and Tkl, or Pkl?
• Variable and value ordering
• How to treat all-different constraint
– GAC using Regin’s algorithm O(n^4)
– AC using the binary decomposition
Good choices for the CSP model
• Experience and small instances suggest:
– Assign the Skl and Tkl variables
– Choose variable to assign with Fail First
(smallest domain) heuristic
• Break ties by alternating between Skl and Tkl
– Use GAC on all-different constraints for Skl
and Tkl
– Use AC on binary decomposition of large alldifferent constraint on Pkl
Performance
n
4
0-1 model
CSP model
Fails t/sec AC
Fails t/sec
4
0.11 2
0.18
CSP model
GAC
Fails t/sec
2
0.38
5
1950
4.05 295
6
?
?
7*
20083 59.8 91687 51.1 57495 66.1
1.39 190
1.55
640235 657 442059 773
General methodology?
• Choose a basic model
• Consider auxiliary variables
– To reduce number of
constraints, improve
propagation
• Consider combined models
– Channel between views
• Break symmetries
• Add implied constraints
– To improve propagation
2ns case study: Langford’s
problem
Langford’s problem
• Prob024 @
www.csplib.org
• Find a sequence of 8
numbers
– Each number [1,4]
occurs twice
– Two occurrences of i
are i numbers apart
• Unique solution
– 41312432
Langford’s problem
• L(k,n) problem
– To find a sequence of k*n
numbers [1,n]
– Each of the k successive
occrrences of i are i apart
– We just saw L(2,4)
• Due to the mathematician
Dudley Langford
– Watched his son build a
tower which solved L(2,3)
Langford’s problem
• L(2,3) and L(2,4) have unique solutions
• L(2,4n) and L(2,4n-1) have solutions
– L(2,4n-2) and L(2,4n-3) do not
– Computing all solutions of L(2,19) took 2.5 years!
• L(3,n)
– No solutions: 0<n<8, 10<n<17, 20, ..
– Solutions: 9,10,17,18,19, ..
A014552
Sequence: 0,0,1,1,0,0,26,150,0,0,17792,108144,0,0,39809640,326721800,
0,0,256814891280,2636337861200
Basic model
• What are the variables?
Basic model
• What are the variables?
Variable for each occurrence of a number
X11 is 1st occurrence of 1
X21 is 1st occurrence of 2
..
X12 is 2nd occurrence of 1
X22 is 2nd occurrence of 2
..
• Value is position in the sequence
Basic model
• What are the constraints?
– Xij in [1,n*k]
– Xij+1 = i+Xij
– Alldifferent([X11,..Xn1,X12,..Xn2,..,X1k,..Xnk
])
Recipe
• Create a basic model
– Decide on the variables
• Introduce auxiliary variables
– For messy/loose constraints
• Consider dual, combined or
0/1 models
• Break symmetry
• Add implied constraints
• Customize solver
– Variable, value ordering
Break symmetry
• Does the problem have any symmetry?
Break symmetry
• Does the problem have any symmetry?
– Of course, we can invert any sequence!
Break symmetry
• How do we break this symmetry?
Break symmetry
• How do we break this symmetry?
– Many possible ways
– For example, for L(3,9)
• Either X92 < 14 (2nd occurrence of 9 is in 1st half)
• Or X92=14 and X82<14 (2nd occurrence of 8 is in
1st half)
Recipe
• Create a basic model
– Decide on the variables
• Introduce auxiliary variables
– For messy/loose constraints
• Consider dual, combined or
0/1 models
• Break symmetry
• Add implied constraints
• Customize solver
– Variable, value ordering
What about dual model?
• Can we take a dual view?
What about dual model?
• Can we take a dual view?
• Of course we can, it’s a permutation!
Dual model
• What are the (dual) variables?
– Variable for each position i
• What are the values?
Dual model
• What are the (dual) variables?
– Variable for each position i
• What are the values?
– If use the number at that position, we cannot
use an all-different constraint
– Each number occurs not once but k times
Dual model
• What are the (dual) variables?
– Variable for each position i
• What are the values?
– Solution 1: use values from [1,n*k] with the value i*n+j
standing for the ith occurrence of j
– Now want to find a permutation of these numbers
subject to the distance constraint
Dual model
• What are the (dual) variables?
– Variable for each position i
• What are the values?
– Solution 2: use as values the numbers [1,n]
– Each number occurs exactly k times
– Fortunately, there is a generalization of all-different
called the global cardinality constraint (gcc) for this
Global cardinality constraint
• Gcc([X1,..Xn],l,u) enforces values used by
Xi to occur between l and u times
– All-different([X1,..Xn]) = Gcc([X1,..Xn],1,1)
• Regin’s algorithm enforces GAC on Gcc in
O(n^2.d)
– Regin’s papers are tough to follow but this
seems to beat his algorithm for all-different!?
Dual model
• What are the constraints?
– Gcc([D1,…Dk*n],k,k)
– Distance constraints?
Dual model
• What are the constraints?
– Gcc([D1,…Dk*n],k,k)
– Distance constraints:
• Di=j then Di+j+1=j
Combined model
• Primal and dual variables
• Channelling to link them
– What do the channelling constraints look like?
Combined model
• Primal and dual variables
• Channelling to link them
– Xij=k implies Dk=i
Solving choices?
• Which variables to assign?
– Xij or Di
Solving choices?
• Which variables to assign?
– Xij or Di, doesn’t seem to matter
• Which variable ordering heuristic?
– Fail First or Lex?
Solving choices?
• Which variables to assign?
– Xij or Di, doesn’t seem to matter
• Which variable ordering heuristic?
– Fail First very marginally better than Lex
• How to deal with the permutation constraint?
– GAC on the all-different
– AC on the channelling
– AC on the decomposition
Solving choices?
• Which variables to assign?
– Xij or Di, doesn’t seem to matter
• Which variable ordering heuristic?
– Fail First very marginally better than Lex
• How to deal with the permutation
constraint?
– AC on the channelling is often best for time
Conclusions
• Modelling is an art but there are patterns
– Develop basic model
• Decide on the variables and their values
– Use auxiliary variables to represent constraints
compactly/efficiently
– Consider dual, combined and 0/1 models
– Break symmetry
– Add implied constraints
– Customize solver for your model