An Introduction to
Constraint Programming
J. Christopher Beck
Dept. of Mechanical & Industrial Engineering
University of Toronto
Canada
University of Toronto
Mechanical & Industrial Engineering
University of Toronto
Mechanical & Industrial Engineering
You have
5 minutes!
Crystal Maze
• Place the numbers 1 through 8 in the
nodes such that:
– Each number appears exactly once
– No connected
nodes have
consecutive
numbers
?
?
?
?
?
?
University of Toronto
Mechanical & Industrial Engineering
?
?
Constraint Satisfaction Problem
(CSP)
• Given:
– V, a set of variables {v0, v1, …, vn}
– D, a set of domains {D0, D1, …, Dn}
– C, a set of constraints {c0, c1, …, cm}
• Each constraint, ci, has a scope
ci(v0, v2, v4, v117, …), the variables that it
constrains
University of Toronto
Mechanical & Industrial Engineering
Constraint Satisfaction Problem
(CSP)
• A constraint, ci, is a mapping from the
elements of the Cartesian product of the
domains of the variables in its scope to
{T,F}
– ci(v0, v2, v4, v117, …) maps:
(D0 X D2 X D4 X D117 X … )  {T,F}
• A constraint is satisfied if the assignment
of the variables in its scope are mapped to
T
University of Toronto
Mechanical & Industrial Engineering
Constraint Satisfaction Problem
(CSP)
• In a solution to a CSP:
– each variable is assigned a value from its
domain: vi = di, di є Di
– each constraint is satisfied
University of Toronto
Mechanical & Industrial Engineering
Crystal Maze
• Place the numbers 1 through 8 in the
nodes such that:
– Each number appears exactly once
– No connected
nodes have
consecutive
numbers
?
?
?
?
?
?
University of Toronto
Mechanical & Industrial Engineering
?
?
Modeling Crystal Maze
• Each node  a variable
• {1, …, 8}  values in the domain of each
variable
• No consecutive numbers  a constraint
– (vi, vj)  |vi – vj| > 1
• All values used  all-different constraint
University of Toronto
Mechanical & Industrial Engineering
Heuristic Search
{1, 2, 3, 4, 5, 6, 7, 8}
?
?
?
1?
8?
?
?
?
University of Toronto
Mechanical & Industrial Engineering
Inference/Propagation
{1, 2, 3, 4, 5, 6, 7, 8}
?
?
?
1?
8?
?
?
?
{1, 2, 3, 4, 5, 6, 7, 8}
University of Toronto
Mechanical & Industrial Engineering
Inference/Propagation
{3, 4, 5, 6}
4?
{3, 4, 5, 6}
6?
{3, 4, 5, 6, 7}
7?
{2, 3, 4, 5, 6}
1?
8?
3?
5?
{3, 4, 5, 6}
2?
University of Toronto
{3, 4,Mechanical
5, 6}& Industrial
Engineering
Generic CP Algorithm
Start
Solution?
Success
Propagators
Dead-end?
Backtrack
Technique
Nothing to
retract?
Failure
Make
Heuristic
Decision
Assert
University of Toronto
Constraint
Mechanical & Industrial Engineering
The Core of CP
• Modeling
– How to represent the problem
• Heuristic search
– How to branch
– How much effort to find a good branch
• Backtracking
• Inference/propagation
University of Toronto
Mechanical & Industrial Engineering
Modeling
• How do you represent
the problem with your
tools?
• Like MIP, it’s an art
– Not as “advanced” on the theory side as MIP
modeling
• Different models of the same problem can
have radically different performance
University of Toronto
Mechanical & Industrial Engineering
Modeling
• Often the first model is easy but poor
– rich constraint language, “natural”
– slow, scales poorly
• Need to think about:
– symmetry, “what would inference do?”, search
heuristics, adding redundancy, multiple
viewpoints, …
• May have to build custom constraints
University of Toronto
Mechanical & Industrial Engineering
Heuristic Search
• Traditional view:
– variable-ordering &
value ordering
• More general view:
– add constraints that partition the space
• Domain independent vs. domain
dependent
– min domain/degree, adaptive weights, …
– e.g., heuristics for scheduling
University of Toronto
Mechanical & Industrial Engineering
The Importance of Heuristics
N = 30, |d| = 8, 45 constraints, 26 tuples/constraint
(64-26)/64 nogoods => constraint tightness ~ 0.6
Forward checking search
means for 10 problems
lexical
min-domain
Searchtree 353,599 nodes
Time
7.36s
44 nodes
0.01s
University of Toronto
Mechanical & Industrial Engineering
Backtracking
• Need to follow alternative branches
– dead-end or a possible sub-optimal solution
• Chronological backtracking (aka DFS)
– simplest, most popular, dumbest
• Constraint-directed backjumping
– keep track of info to provably jump over some
branches
• Randomized restart
– nice mathematical model
University of Toronto
Mechanical & Industrial Engineering
Inference/Propagation
• Probably the primary reason for the
problem solving power of CP
– main mechanism to reduce search
• Most mathematically/theoretically mature
sub-area of CP
• 300+ “global constraints” in current catalog
– domain-independent to specialized
– stamp collecting?
University of Toronto
Mechanical & Industrial Engineering
Heuristic Search or
Propagation?
There must be a 1 or 8
here. Why?
?
?
?
1?
8?
?
?
{1, 2, 3, 4, 5, 6, 7, 8}
?
University of Toronto
Mechanical & Industrial Engineering
Arc Consistency
• Fundamental notion in CP!
• Given: c1(v1,v2)
– a binary constraint
– e.g., v1 < v2
• Given: D1 = D2 = {0, 1, …, 5}
University of Toronto
Mechanical & Industrial Engineering
Arc Consistency
• c1 is arc consistent iff
– for all values d1 є D1 there exists a value
d2 є D2 such that c1(v1=d1,v2=d2)  T
– And similarly for all values d2 є D2
V1
<
{0,1,2,3,4,5}
V2
{0,1,2,3,4,5}
University of Toronto
Mechanical & Industrial Engineering
What Now?
{0,1,2,3,4,5}
V3
V1
<
{0,1,2,3,4,5}
V2
{0,1,2,3,4,5}
University of Toronto
Mechanical & Industrial Engineering
Generalized Arc Consistency
(GAC)
• Given: c1(v1,..., vm)
• C1 is GAC iff
– for all variables di, for all values di є Di there
exists a tuple of values [dj є Dj], j≠i such that
C1(vi=di,[vj=dj])  T
• E.g., c1(v1,v2,v3,v4)
– for every value in d1 є D1 there must be a
triple [d2 є D2, d3 є D3, d4 є D4] s.t. c1(v1=d1,
v2=d2, v3=d3, v4=d4)  T
University of Toronto
Mechanical & Industrial Engineering
All-different Constraint
• all-diff(v1, v2, …, vn)
– all vi’s must take on a different value
– logically equivalent to v
i j
i vj,
• Useful in many, many combinatorial
problems
– travelling salesman, timetabling, …
University of Toronto
Mechanical & Industrial Engineering
All-Diff vs. Clique of ≠
• Logically, all-diff(v1, v2, …, vn) =def
vi ≠ vj for 1 ≤ i < j ≤ n
• D1=D2=D3={1,3}
• Establish AC (or GAC) for
– v1 ≠ v2, v1 ≠ v3, v2 ≠ v3
– all-diff(v1,v2,v3)
{1,3}
V1
{1,3}
{1,3}
V3
≠
V1
V3
{1,3}
all-diff
≠
≠
V2
{1,3}
V2
{1,3}
University of Toronto
Mechanical & Industrial Engineering
All-Diff vs. Clique of ≠
• So GAC on all-diff is stronger than AC on
a clique of ≠
• But what about the computational
complexity?
• All-diff is a “global constraint”
– GAC on some global constraints is NPcomplete
– What about all-diff?
University of Toronto
Mechanical & Industrial Engineering
GAC Propagation on All-Diff
•
•
•
•
Value graph
Matchings & graph theory
Main Theorem
Algorithm Sketch
University of Toronto
Mechanical & Industrial Engineering
Value Graph
1
A
2
B
3
C
4
D
• Example
– D1 = {B,C,D,E},
D2 = {B,C},
D3 = {A,B,C,D},
D4 = {B,C}
– all-diff(v1,…,v4)
E
University of Toronto
Mechanical & Industrial Engineering
Definitions
• Given a graph G(V,E)
• A matching M  E is a set of disjoint
edges
– disjoint = no common end-points
• M covers a set S  Vif all vertices in S
are end-points of an edge in M
• A vertex is M-free if it isn’t covered by M
University of Toronto
Mechanical & Industrial Engineering
Value Graph
• In terms of
matchings, provide
a characterization
of an assignment
that satisfies the alldiff constraint
1
A
2
B
3
C
4
D
E
University of Toronto
Mechanical & Industrial Engineering
Theorem [Regin94]
• G is a value graph
• All-diff(v1,…,vn) is GAC iff every edge in G
belongs to a matching in G covering
{v1,…,vn}
University of Toronto
Mechanical & Industrial Engineering
1
A
2
B
3
C
4
D
E
University of Toronto
Mechanical & Industrial Engineering
1
A
2
B
3
C
4
D
E
University of Toronto
Mechanical & Industrial Engineering
1
A
2
B
3
C
4
D
E
University of Toronto
Mechanical & Industrial Engineering
Finding Initial Max-Matching
1
A
2
B
3
C
4
D
• Find a maximum matching
– O(n√m), n = |V|, m = |Di|
– well-known graph theory
algorithm
University of Toronto
Mechanical & Industrial Engineering
E
Finding More Max-Matchings 1
• Orient edges:
– In M – from variables to values
– Not in M – from values to variables
• Find strongly connected components
– depth-first search – if you re-enter a node it is
in a strongly connected component
• M-alternating cycle
University of Toronto
Mechanical & Industrial Engineering
1
A
2
B
3
C
4
D
E
2
B
C
4
University of Toronto
Mechanical & Industrial Engineering
Finding More Max-Matchings 2
• Find an M-alternating path starting from an
uncovered vertex
• Breadth first search
University of Toronto
Mechanical & Industrial Engineering
1
A
A
3
2
B
3
C
D
4
D
1
E
E
University of Toronto
Mechanical & Industrial Engineering
GAC Propagation
• All edges
– in M
– in an M-alternating circuit, or
– in an M-alternating path
• are consistent
• Remove the rest
University of Toronto
Mechanical & Industrial Engineering
1
A
1
A
2
B
2
B
3
C
3
C
4
D
4
D
E
E
University of Toronto
Mechanical & Industrial Engineering
Value Graph
• D1 = {B,C,D,E}, D2 = {B,C},
D3 = {A,B,C,D}, D4 = {B,C}
• All-different:
– D1 = {B,C,D,E}, D2 = {B,C},
D3 = {A,B,C,D}, D4 = {B,C}
University of Toronto
Mechanical & Industrial Engineering
GAC All-Diff Propagation
Summary
• Find a max-matching
– O(n√m), n = |V|, m = |Di|
• Find all M-alternating circuits
– DFS to find strongly connected components:
O(n+m)
• Find all M-alternating paths starting from
an M-free vertex
– Breadth-first search: O(m)
University of Toronto
Mechanical & Industrial Engineering
Global Constraints
• Any number of variables
• Accompanying efficient propagation
algorithm
– to enforce some level of consistency (not
necessarily GAC)
• A custom global constraint becomes both
an algorithmic and modeling object
– extending the language
– inference algorithm can be arbitraryUniversity of Toronto
Mechanical & Industrial Engineering
– compositionality
Differences with MIP
• Richer, extensible language
• Traditionally no explicit use of lower
bounds
– no optimization function
• constraint satisfaction problems
– less focus on relaxations
– less true now: LBs “inside” global constraints
University of Toronto
Mechanical & Industrial Engineering
CP Engineering Process
• Model, remodel, remodel, …
– if you are lucky, you’re done
– abstraction?
• Domain-specific heuristics
• Custom global constraints
• Hybrids
– CP/LP, CP/MIP, CP/Metaheuristics
University of Toronto
Mechanical & Industrial Engineering
When to Choose CP
• <Handwaving mode on>
• Finding a feasible solution is hard
– intricate set of complicated constraints
• No tight relaxations
• Existing global constraints
• Strong back-propagation
University of Toronto
Mechanical & Industrial Engineering