An Introduction to
Constraint Programming Part II:
Solving Scheduling Problems
J. Christopher Beck
Dept. of Mechanical & Industrial Engineering
University of Toronto
Canada
University of Toronto
Mechanical & Industrial Engineering
Where We Left Off …
• CP has been mostly developed to solve
combinatorial feasibility and optimization
problems
• General solution approach: tree search
with inference at each node
– no generic relaxation
• Key concept: global constraint propagation
University of Toronto
Mechanical & Industrial Engineering
Global Constraints
• A constraint over an arbitrary
number of variables that
embodies a frequently encountered subproblem
– all-different(v1, …, vk), global cardinality
– disjunctive/cumulative resource
– grammar
Global Constraint Catalog
www.emn.fr/z-info/sdemasse/gccat/index.html
–…
• [still some arguments about this within
CP]
University of Toronto
Mechanical & Industrial Engineering
Global Constraints
• A global constraint represents
a commonly occurring problem
substructure
• The focus of modeling and solving
techniques
– models are conjunctions of global constraints
– solving is typically tree search with domain
consistency enforced at every node
University of Toronto
Mechanical & Industrial Engineering
Differences with MIP
• Richer, extensible language
• Focus on global constraints
• Traditionally no explicit use of lower
bounds
– no optimization function
• constraint satisfaction problems
– less focus on relaxations
– less true now: LBs “inside” global constraints
University of Toronto
Mechanical & Industrial Engineering
When to Choose CP
• <Handwaving mode on>
• Finding a feasible solution is hard
– intricate set of complicated constraints
• No tight relaxations
• Existing global constraints model parts of
your problem
• Strong back-propagation
University of Toronto
Mechanical & Industrial Engineering
Questions?
University of Toronto
Mechanical & Industrial Engineering
Job Shop Scheduling
Operation/Task/
Activity
Job
Precedence Constraint
University of Toronto
Mechanical & Industrial Engineering
Job Shop Scheduling
makespan
University of Toronto
Mechanical & Industrial Engineering
Notation
estj
ectj
lstj
lctj
pj
pj – processing time of activity j (aka duration)
estj – earliest start time of activity j
lstj – latest start time of activity j
ectj – earliest completion time of activity j
lctj – latest completion time of activity j
University of Toronto
Mechanical & Industrial Engineering
Notation
[estj lstj]
[ectj lctj]
pj
Domain of start times
(represented by an interval)
University of Toronto
Mechanical & Industrial Engineering
Temporal Propagation
(Arc Consistency - AC)
The same as CPM
[0 40]
20
[20 60]
15
[35 75]
10
[45 85]
15
0
20
100
[20 40]
20
0
[40 60]
15
[55 75]
10
[65 85]
15
100
University of Toronto
Mechanical & Industrial Engineering
Simplified Edge-Finding (aka
Constraint-Based Analysis (CBA))
• What can you infer here?
15
20
35
50
80
90
Operations on the same unary capacity resource
University of Toronto
Mechanical & Industrial Engineering
Edge-Finding Exclusion
25
20
10
15
0
15
10
est(S)
80
S
100
lct(S)
University of Toronto
Mechanical & Industrial Engineering
Edge-Finding Exclusion
70
100
25
20
10
15
0
15
75
10
est(S)
S
100
lct(S)
University of Toronto
Mechanical & Industrial Engineering
Exclusion Rules
On the same
unary-capacity resource
For all non-empty subsets, S, and
activities AS:
(lft(S) - est(S) < durA + dur(S))
 (lft(S) - estA < durA + dur(S))
estA  est(S) + dur(S)
(lft(S) - est(S) < durA + dur(S))
 (lftA - est(S) < durA + dur(S))
lftA  lft(S) - dur(S)
University of Toronto
Mechanical & Industrial Engineering
Algorithm [Vilim et al 05]
• With a clever data structure (Θ-Λ-trees) a
set of activities can be updated in
O(n log n) time
• An easier-to-understand algorithm can do
it in O(n2) [Nuijten 1994]
University of Toronto
Mechanical & Industrial Engineering
CP vs. MIP for JSP
Mean
relative
error vs.
best known
solution
University of Toronto
Mechanical & Industrial Engineering
Resource Allocation &
Scheduling
Assign jobs
Schedule
Schedule
Schedule
University of Toronto
Mechanical & Industrial Engineering
[Hooker 2005] Constraints, 10, 385-401, 2005.
CP Model
minimize cost of assignment
all jobs assign to 1 resource
global resource constraint
bounds on start time var.
University of Toronto
Mechanical & Industrial Engineering
[Heinz & Beck 2012] CPAIOR 2011.
MIP Time-indexed Model
resource capacity at each time point
University of Toronto
Mechanical & Industrial Engineering
Logic-Based Benders Decomp.
Master Problem
Cut
Cut
Solution
Subproblem 1
Global Model
...
Solution
Subproblem n
University of Toronto
Mechanical & Industrial Engineering
[Hooker & Ottosson 2003] Mathematical Programming, 96, 33-60, 2003.
Assign jobs
The Approach
Assignment
• Assign jobs to
Cuts
resources subject
to sub-problem
relaxation and
Benders cuts
• Schedule independently
• Send Benders cuts back to master
• Repeat
University of Toronto
Mechanical & Industrial Engineering
LBBD Model
MP
SP relaxation
Benders cuts
global resource constraint
SP
University of Toronto
Mechanical & Industrial Engineering
[Hooker 2005] Constraints, 10, 385-401, 2005.
Results
Number problems (out of 195)
for which each method found:
University of Toronto
Mechanical & Industrial Engineering
[Heinz & Beck 2012] CPAIOR 2012.
But wait, there’s more …
University of Toronto
Mechanical & Industrial Engineering
Better LBBD
tighter SP relaxation
University of Toronto
Mechanical & Industrial Engineering
[Hooker 2007] Operations Research 55 2013.
New MIP Model
redundant variables
use tighter SP relaxation from LBBS as a
redundant resource capacity constraint
University of Toronto
Mechanical & Industrial Engineering
[Heinz, Ku, & Beck 2013] CPAIOR 2013.
Results
University of Toronto
Mechanical & Industrial Engineering
[Heinz, Ku, & Beck 2013] CPAIOR 2013.
Confused?
University of Toronto
Mechanical & Industrial Engineering
Conclusions from Experiments
• CP is worst?
– well, it finds very high quality solutions is a
very short time
• MIP is still a very strong technology even
for scheduling problems
• Hybrids!
– CP is a key part of LBBD
– I didn’t show you the main point of the paper
…
University of Toronto
Mechanical & Industrial Engineering
Full Results
CIP = constraint integer programming
University of Toronto
Mechanical & Industrial Engineering
[Heinz, Ku, & Beck 2013] CPAIOR 2013.
CP Hybrids
University of Toronto
Mechanical & Industrial Engineering
CP/OR Integration 1
OR
• OR inside
– For years CP has been using OR algorithms
“inside” global constraints
• edge-finding, all-diff, etc.
Cost-based pruning
in TSPTW
University of Toronto
Mechanical & Industrial Engineering
CP/OR Integration 2
• Cooperating solvers, solving the same
problem
– Model the problem in CP, model the problem
in MP, solvers communicate through the
variables
• MP is “master” and CP supplies
new implied constraints
• CP is “master” and MP supplies lower
ETSP
bounds, relaxed optimal solutions, etc.
• Higher level control: dynamic “master-slave”
University of Toronto
relationship
Mechanical & Industrial Engineering
CP/OR Integration 3
• Decomposition
– Overall problem is divided between CP and
MP solvers
• Benders decomposition (MP master, CP slave)
• Column generation
Column
generation
Benders
decomposition
University of Toronto
Mechanical & Industrial Engineering
CP/OR Integration 4
• Deep integration
– incorporation global constraint inference into a
MIP search
• Constraint Integer Programming: SCIP
• SIMPL
University of Toronto
Mechanical & Industrial Engineering
MIE1619:
Constraint Programming &
Local Search
J. Christopher Beck
Dept. of Mechanical & Industrial Engineering
University of Toronto
Canada
University of Toronto
Mechanical & Industrial Engineering
Advanced Grad Course
• Course is for “doctoral stream” students
• This will be a challenging course
– 1-2 papers to read per week
– 10-15 hours/week outside lectures
– I expect a high-level of performance both in
class participation and in the project
• Expected level: publishable!
University of Toronto
Mechanical & Industrial Engineering
Goals
• Apply Constraint
Programming (CP) and/or Local Search
(LS) to your research interests
• Teach “grad school” skills
– Literature research, algorithmic research,
writing research papers, writing peer reviews,
giving presentations, being an active member
of the research community
University of Toronto
Mechanical & Industrial Engineering
Evaluation
Project:
Research Paper
Problem
Presentation
Modeling
Assignments
10-questions
60
5% checkpoint
5% initial submission
10% peer reviews
25% final paper
15% presentation
15
1 of them
20 minutes + questions
15
number: enrollment - 1
10
Once or twice (in pairs)
University of Toronto
Mechanical & Industrial Engineering
Research Paper Goal
• Apply CP and/or LS to your research topic
– I have some ideas for people who are stuck
• Publish!
– The goal (but not a requirement)
• The paper you hand in for this course should be
publishable
• And usable as part of your thesis
University of Toronto
Mechanical & Industrial Engineering
Lecture Goal
• Doing the readings is essential
• You are expected to participate each week
– I will ask people to explain particular points
– Starting in Week 3, you will begin preparing
questions based on the readings
University of Toronto
Mechanical & Industrial Engineering
Outline
•
•
•
•
•
•
Intro: 1 week
CP: 3 weeks
LS: 2 weeks
Writing Peer Reviews: 0.5 weeks
OR/CP/LS Hybrids: 4.5 weeks
Project Presentations: 1 or 2 weeks
University of Toronto
Mechanical & Industrial Engineering
Questions?
University of Toronto
Mechanical & Industrial Engineering
Search: Texture Measurements
• Algorithms for the analysis of the
constraint graph representation of a
search state
• Heuristic search idea:
– Use texture measurement to reveal problem
structure
– Formulate heuristic commitment based on the
structure
University of Toronto
Mechanical & Industrial Engineering
The SumHeight Heuristic
Find the resource, R*, and time point, t*,
with highest competition
Find the 2 activities not sequenced with
each other with the highest individual
demand for R* at t*
Heuristically, post a sequencing constraint
between them
University of Toronto
Mechanical & Industrial Engineering
The SumHeight Texture:
Individual Demand
time
A1
est
ect
lst
lct
University of Toronto
Mechanical & Industrial Engineering
The SumHeight Texture:
Aggregate Demand
Dynamic Focus of Attention
R2
R2
R2
time
University of Toronto
Mechanical & Industrial Engineering
Search
• At each node (after propagation)
– recalculate texture measurements
– identify peaks
– identify activity pair, A, B
– branch on (A  B) OR (B  A)
• See [Beck 1999] for more than you want to
know about this
University of Toronto
Mechanical & Industrial Engineering
What If We Want to
Minimize Makespan?
Assign jobs
Assignments
Cuts
Schedule
Schedule
Schedule
University of Toronto
Mechanical & Industrial Engineering
Minimizing Makespan:
What Changes?
M = maxj{tj+pxjj}
different Benders cuts
University of Toronto
Mechanical & Industrial Engineering
The Challenge:
Benders Cuts for Makespan
• Apparently this was the 12th or 13th
Benders’ cut formulation that the author
tried
• For details see [Hooker 2005]
University of Toronto
Mechanical & Industrial Engineering