Proceedings of the Tenth Symposium on Abstraction, Reformulation, and Approximation
Modeling, Global Constraints, and Decomposition∗
J. Christopher Beck
Department of Mechanical and Industrial Engineering
University of Toronto
jcb@mie.utoronto.ca
Extended Abstract
Constraint Programming and Global
Constraints
recognition of sub-structure in problem for which there already exists a corresponding global constraint or, failing
that, for which a novel global constraint can be formulated.
Similarly, search is seen as the interplay between global
heuristic decisions (e.g., branching on a variable) and local
inferences as the propagation algorithms of each global constraint work to compute the local implications of the branching decisions and inference of other constraints.2
Our primary thesis here is that this perspective on modeling and search should be pursued more vigorously to expand
the modeling and solving approaches that are commonly
viewed as mainstream CP.
In Constraint Programming (CP) the concept of a constraint
has been formalized as a relation over a set of variables that
defines the tuples of values that the variables can simultaneously be assigned to. The same formalization can be seen
in other approaches to mathematical optimization such as
SAT solving and mixed integer linear programming (MILP).
However, unlike most other approaches, CP has eschewed
structural restrictions on the definition of a constraint: there
is no real restriction on the language in which constraints
can be expressed. This lack of restriction can be most obviously seen in the in relation constraint (also called
the table constraint) (Beldiceanu et al. 2007)1 where the
relation is defined explicitly as a list of allowed tuples.
More commonly, and often more usefully, in CP a
constraint (and especially a “global constraint”) embodies some, often combinatorial, sub-structure of a problem together with algorithms for inference on that substructure. For example, the cumulative constraint (Aggoun and Beldiceanu 1993) which has been developed in
constraint-based scheduling, enforces that the total resource
capacity used by a set of activities executing at any time
point must be less than or equal to some maximum. A variety of inference algorithms have been developed to remove
possible values from the start time domains of the activities
when it can be proved that the values do not participate in
any feasible solution. Of course, for a global constraint to be
a useful object for modeling and solving, the sub-structure it
represents must be frequently encountered across a variety
of problems that people wish to solve.
The primacy of global constraints results in a particular
perspective on both modeling and solving constraint programs. More than many other approaches, modeling consists of the composition of global constraints and therefore
the modeling task is more about the, sometimes non-trivial,
Logic-based Benders Decomposition
Over the past 15 years, the Logic-based Benders Decomposition (LBBD) approach has been developed, primarily in
the area of hybrid algorithms. Formalized by John Hooker
(Hooker 2000), a number of authors have used this pattern to
solve hard combinatorial problems in planning and scheduling (Jain and Grossmann 2001; Hooker 2005; Beck 2010;
Heinz, Ku, and Beck 2013; Ciré, Coban, and Hooker 2013),
call center scheduling (Benoist, Gaudin, and Rottembourg
2002), steel production scheduling (Harjunkoski and Grossmann 2001), dispatching of automated guided vehicles (Correia and Captivo 2006), multicore architecture optimization (Benini et al. 2008), traffic diversion (Xia, Eremin, and
Wallace 2004), two-dimensional bin-packing (Pisinger and
Sigurd 2009), transportation network design (Peterson and
Trick 2009), aircraft maintenance scheduling (Aramon Bajestani and Beck 2013), location-allocation problems (FazelZarandi and Beck 2012), and queue design and control
(Terekhov, Beck, and Brown 2009).
The core of LBBD is to model a problem as a master
problem (a relaxation of the original problem) and one or
more sub-problems. The master problem is solved first, assigning values to the master variables. The sub-problem(s)
are then solved based on the assignments to the master variables. A sub-problem is formally an inference dual (Hooker
2000) which derives the tightest bound on the global objective function, given the values of the master variables and
the constraints of the sub-problem. If this bound is greater
∗
This extended abstract accompanies an invited talk given
jointly at the Tenth Symposium on Abstraction, Reformulation, and
Approximation (SARA2013) and the Sixth Annual Symposium on
Combinatorial Search (SOCS2013) on July 12, 2013.
c 2013, Association for the Advancement of Artificial
Copyright Intelligence (www.aaai.org). All rights reserved.
1
http://www.emn.fr/z-info/sdemasse/gccat/index.html
2
It is somewhat ironic and unfortunate that “global” constraints
are actually the embodiment of “local” structure and inference.
2
than the master objective (assuming a minimization problem), then there is no sub-problem solution consistent with
the master solution. A Benders cut is therefore generated by
the sub-problem and added to the master. This cut is a new
constraint for the master model, expressing aspects of the
global solution that are not satisfied by the current master solution. After gathering cuts from each of the sub-problems,
the master is re-solved. This iteration continues until all subproblems are consistent with the master solution and therefore an optimal global solution has been found.
For example, in a resource allocation and scheduling
problem studied by Hooker (2005), the global problem requires the assignment and scheduling of a set of independent, time-constrained jobs to resources. In the simplest version, each job has a cost to be assigned to each resource and
the goal is to minimize the total assignment cost subject to
the constraint that each resource must have a feasible schedule. The master problem models the resource assignment decisions while a sub-problem is defined for each resource to
determine if the assigned set of jobs can be feasibly scheduled. The Benders cut in the simple case is a no-good over
the set of resource assignment variables corresponding to the
jobs assigned to a resource. Such a cut removes the assignment of the same set of jobs, or a superset, from the feasible
space of the master problem.
There are three main challenges to using LBBD: deciding how to model the problem as a the master/sub-problem
decomposition, developing a relaxation of the sub-problems
in the master problem, and deriving strong but computationally inexpensive Benders cuts. The first challenge is to conceptualize how the global problem can be modeled to identify the sets of variables and constraints that will form the
master problem and sub-problems. Given a decomposition,
it is typically necessary to augment the master problem with
some relaxation of the sub-problem(s): if the master does not
have some representation of the sub-problems to constrain
its search space, it may end up blindly enumerating all master solutions, depending only on the Benders cuts for guidance to part of the search space where the sub-problems will
be satisfied. Finally, a Benders cut must remove the current
master solution while not removing any globally optimal solutions. In practice, a simple no-good cut is seldom sufficient
for computational performance. A cut should be formulated
not only to remove the current master solution but also other
master solutions which also fail to satisfy the sub-problem.
There are various methods for strengthening and lifting a
cut so that a larger area of the globally infeasible (or suboptimal) search space is removed from the master problem
(Hooker 2007b).
The core idea of logic-based Benders decomposition is
delayed constraint posting. Rather than represent all variables and constraints in a monolithic model, structure is
introduced so that sub-problem constraints are only taken
into account when a master solution has already been found.
If the master problem and sub-problems are modeled such
that the interaction among their variables is low, exponential
gains can be observed.
The idea of delayed constraint posting is present in other
approaches to combinatorial problem solving, sometimes
for reasons of expressivity as well as computational gains.
Prime examples are found in Satisfiability Modulo Theory
(SMT) (Nieuwenhuis, Oliveras, and Tinelli 2006) as well
as the common MILP technique of branch-and-cut (Miller,
Tucker, and Zemlin 1960).
LBBD and CP
The expressivity of global constraints and the perspective
of modeling and solving by combining active and encapsulated combinatorial sub-structure may provide a connection between standard CP techniques and LBBD. In particular, global constraints can be seen as LBBD sub-problems
while the master problem is an extreme relaxation: the domain store (Hooker 2007a). The domain store is clearly a
relaxation of the globally feasible problem space. As search
is performed (e.g., by branching) each global constraint responds by pruning domain values of variables that it can
prove do not participate in any local solution. One can view
a global constraint as a sub-problem where the Benders cuts,
communicated to the master problem, are unary constraints,
removing values from domains. This definition is entirely
consistent with the formalization of LBBD (Hooker 2000)
where sub-problem solving can be triggered by partial solutions to the master problem.
This conceptualization, though not the link with Benders
decomposition, is explicit in work in constraint logic programming where a distinction was made between primitive
constraints and propagation constraints (Wallace, Novello,
and Schimpf 1997; Marriott and Stuckey 1998). Loosely,
primitive constraints are simple constraints (e.g., unary constraints that remove domain values, linear constraints) which
are included with the domain store in the core representation
of a solver. Propagation constraints correspond to what we
now think of as global constraints: constraints with accompanying propagation algorithms that generate new primitive
constraints.
Making this connection between CP and LBBD is only
useful if the concepts and techniques in one area can be successfully used to extend the modeling and solving power
in the other. Two broad and, to some extent, speculative
directions for such development include viewing search in
CP, specifically the relation between branching heuristics
and constraint propagation, as the interplay between primal
and dual3 and the investigation of alternatives to the domain
store as a relaxation in CP. The primary example of the latter is the recent work on multi-valued decision diagrams as
a core representation for CP (Hoda, van Hoeve, and Hooker
2010).
Perhaps the most developed direction is the view that algorithms embedded and information available from global
constraints can be more than consistency algorithms and domain pruning. If global constraints encapsulate useful substructure, perhaps they can share more global information as
3
These ideas were expressed in presentations and discussions
by a number of researchers at the Workshop on General Principles
in Seeking Feasible Solutions for Combinatorial Problems, organized by John Chinneck, Gilles Pesant, and Louis-Martin Rousseau
at CPAIOR 2013.
3
References
well as solve sub-problems. For example:
Aggoun, A., and Beldiceanu, N. 1993. Extending CHIP in
order to solve complex scheduling and placement problems.
Mathematical and Computer Modelling 17(7):57–73.
Aramon Bajestani, M., and Beck, J. C. 2013. Scheduling
a dynamic aircraft repair shop with limited repair resources.
Journal of Artificial Intelligence Research 47:35–70.
Beck, J. C.; Davenport, A. J.; and Le Pape, C. 2000. Introduction. Constraints 5(4):327–334.
Beck, J. C. 2010. Checking-up on branch-and-check.
In Proceedings of the Sixteenth International Conference
on the Principles and Practice of Constraint Programming
(CP2010), 84–98.
Beldiceanu, N.; Carlsson, M.; Demassey, S.; and Petit, T.
2007. Global constraint catalogue: Past, present and future.
Constraints 12(1):21–62.
Benini, L.; Lombardi, M.; Mantovani, M.; Milano, M.; and
Ruggiero, M. 2008. Multi-stage Benders decomposition
for optimizing multicore architectures. In Perron, L., and
Trick, M., eds., In the Proceeding of the Fifth International
Conference on Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems (CPAIOR’08), 36–50.
Benoist, T.; Gaudin, E.; and Rottembourg, B. 2002. Constraint programming contribution to Benders decomposition: A case study. In In the Proceeding of the Eighth International Conference on Principles and Practice of Constraint Programming (CP’2002), 603–617.
Ciré, A.; Coban, E.; and Hooker, J. N. 2013. Mixed integer programming vs. logic-based benders decomposition for
planning and scheduling. In Proceedings of the Tenth International Conference on Integration of AI and OR Techniques
in Constraint Programming for Combinatorial Optimization
Problems (CPAIOR2013), 325–331.
Correia, I., and Captivo, M. E. 2006. Bounds for the single
source modular capacitated plant location problem. Computers & Operations Research 33:2991–3003.
Fazel-Zarandi, M. M., and Beck, J. C. 2012. Using logicbased Benders decomposition to solve the capacity and distance constrained plant location problem. INFORMS Journal on Computing 24:399–415.
Fox, M. S. 1983. Constraint-Directed Search: A Case
Study of Job-Shop Scheduling. Ph.D. Dissertation, Carnegie
Mellon University, Intelligent Systems Laboratory, The
Robotics Institute, Pittsburgh, PA. CMU-RI-TR-85-7.
Harjunkoski, I., and Grossmann, I. E. 2001. A decomposition approach for the scheduling. Computers and Chemical
Engineering 25:1647–1660.
Heinz, S.; Ku, W. K.; and Beck, J. C. 2013. Recent improvements using constraint integer programming for resource allocation and scheduling problems. In Proceedings of the
Tenth International Conference on Integration of AI and OR
Techniques in Constraint Programming for Combinatorial
Optimization Problems (CPAIOR2013), 12–27.
Heinz, S.; Schulz, J.; and Beck, J. C. 2013. Using dual
• Global constraints can be a source of heuristic information, based on their structure, though the calculation, approximation, and aggregation of the number of solutions
to each global constraint (Pesant, Quimper, and Zanarini
2012).
• Global constraints and the sub-structure that they represent can be used generate explanations of inferences that
can be aggregated and reasoned about in a global clause
store (Schutt et al. 2011).
• Global constraints can detect when they have become
independent sub-problems and use a generic or specialized algorithm to find a solution or to prove infeasibility
(Heinz, Ku, and Beck 2013).
• Global constraints can be used for automated remodeling
via dual presolving techniques that decompose constraints
and fix and/or remove variables (Heinz, Schulz, and Beck
2013).
Epilogue: From Optimization Back to AI?
One of the historical areas that influenced constraint programming was the knowledge-based problem solving approach from Artificial Intelligence (Beck, Davenport, and
Le Pape 2000). One of the first uses of the word “constraint”
in this literature is due to Simon (1973). Building on the
cognitive science work of Reitman (1965), Simon uses constraints “to refer to any or all of the elements of that enter
into the definition of a problem” (p. 189). For example, in
the goal of designing a house, an architect may decide that
it is to be two storeys high, generating a constraint that there
must be some way to move between floors. This constraint
evokes from memory (or a knowledge base) various design
alternatives such staircases, elevators, and slides. Resolving
this constraint produces other constraints on the design (e.g.,
the space required for a staircase) and other sub-design alternatives (e.g., a spiral staircase). A constraint, therefore, is
a generative object in the knowledge representation. It contains or evokes alternative approaches to its own satisfaction
(solution components) and these components in turn create
sub-goals which impose further constraints, interleaving the
process of modeling and solving.
In short, rather than the restricted definition of a constraint generally accepted today within CP, Simon’s concept
was of a much richer component of knowledge representation. This approach was explicitly followed in early work
on constraint-based scheduling (e.g., (Fox 1983)), where a
constraint contained knowledge, for example, about how it
could be relaxed.
While it is undoubtedly true that much of the power of
modern CP arises from the formalization of a constraint and
accompanying inference algorithms, it may be time to once
again broaden the knowledge and algorithms embodied by
constraints. Without losing the hard-won formal characterizations, generalizing global constraints to sub-problems of
a logic-based Benders pattern is a promising direction forward, perhaps toward “good old-fashioned AI”.
4
presolving reductions to reformulate cumulative constraints.
Constraints 18(2):166–201.
Hoda, S.; van Hoeve, W.-J.; and Hooker, J. N. 2010. A systematic approach to mdd-based constraint programming. In
Proceedings of the International Conference on Principles
and Practices of Constraint Programming (CP2010), 266–
280.
Hooker, J. N. 2000. Logic-based Methods for Optimization.
Wiley.
Hooker, J. N. 2005. A hybrid method for the planning and
scheduling. Constraints 10(4):385–401.
Hooker, J. N. 2007a. Integrated Methods for Optimization.
Springer.
Hooker, J. N.
2007b.
Planning and scheduling by
logic-based benders decomposition. Operations Research
55:588–602.
Jain, V., and Grossmann, I. E. 2001. Algorithms for hybrid MILP/CP models for a class of optimization problems.
INFORMS Journal on Computing 13:258–276.
Marriott, K., and Stuckey, P. J. 1998. Programming with
Constraints: An Introduction. The MIT Press.
Miller, C. E.; Tucker, A. W.; and Zemlin, R. A. 1960. Integer
programming formulation of traveling salesman problems.
Journal of the ACM 7(4):326–329.
Nieuwenhuis, R.; Oliveras, A.; and Tinelli, C. 2006. Solving SAT and SAT modulo theories: From an abstract DavisPutnam-Logemann-Loveland procedure to DPLL(T). Journal of the ACM 53(6):937–977.
Pesant, G.; Quimper, C.-G.; and Zanarini, A.
2012.
Counting-based search: Branching heuristics for constraint
satisfaction problems. Journal of Artificial Intelligence Research 43:173–210.
Peterson, B., and Trick, M. A. 2009. A Benders approach to
a transportation network design problem. In In the Proceeding of the Sixth International Conference on Integration of
AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems (CPAIOR’09), 326–327.
Pisinger, D., and Sigurd, M. 2009. Using decomposition
techniques and constraint programming for solving the twodimensional bin-packing problem. INFORMS Journal on
Computing 19(1):36–51.
Reitman, W. R. 1965. Cognition and Thought. Wiley.
Schutt, A.; Feydy, T.; Stuckey, P. J.; and Wallace, M. G.
2011. Explaining the cumulative propagator. Constraints
16(3):250–282.
Simon, H. A. 1973. The structure of ill-structured problems.
Artificial Intelligence 4:181–200.
Terekhov, D.; Beck, J. C.; and Brown, K. N. 2009. A constraint programming approach for solving a queueing design and control problem. INFORMS Journal on Computing
21(4):549–561.
Wallace, M.; Novello, S.; and Schimpf, J. 1997. Eclipse:
A platform for constraint logic programming. ICL Systems
Journal 12(1):159–200.
Xia, Q.; Eremin, A.; and Wallace, M. 2004. Problem decomposition for traffic diversions. In In the Proceeding of
the First International Conference on Integration of AI and
OR Techniques in Constraint Programming for Combinatorial Optimization Problems (CPAIOR 2004), 349–363.
5