From: AAAI Technical Report FS-01-04. Compilation copyright © 2001, AAAI (www.aaai.org). All rights reserved.
Stochastic
Constraint Programming
Toby Walsh
Departmentof ComputerScience
University of York
York
England
twOcs,york.ac. uk
Abstract
act nowand observe later. For example, we mayhave to
decide nowwhichnurses to haveon duty and only later discover the actual workload.Wecan easily invert the instantiation order if the application demands,
with the stochastic
variables givenvalues before the decisionvariables are set.
Tomodeldecisionproblems
involvinguncertaintyand
probability,weproposestochasticconstraintprogramming. Stochastic constraint programscontain both
decisionvariables(whichwecan set) andstochastic
variables(whichfollowsomeprobabilitydistribution),
and combine
togetherthe best features of traditional
constraintsatisfaction,stochasticintegerprogramming,
andstochasticsatisfiability. Wegive a semantics
for
stochastic constraint programs,and proposea number
of completealgorithmsand approximation
procedures.
Usingthesealgorithms,weobservephasetransitionbehaviorin stochasticconstraintprograms.
Interestingly,
the cost of bothoptimization
andsatisfactionpeaksin
the satisfaction phaseboundary.
Finally, wediscussa
number
of extensionsof stochasticconstraintprogrammingto relax variousassumptionslike the independencebetween
stochasticvariables.
Introduction
Manyreal world decision problems contain uncertainty.
Data about events in the past maynot be knownexactly
dueto errors in measuring
or difficulties in sampling,whilst
data about events in the future maysimply not be known
with certainty. For example, whenscheduling powerstations, we need to cope with uncertainty in future energy
demands.As a second example, nurse rostering in an accident and emergencydepartmentrequires us to anticipate
variability in workload.Asa final example,whenconstructing a balancedbondportfolio, wemustdeal with uncertainty
in the future price of bonds. Todeal with such situations,
we propose an extension of constraint programming
called
stochastic constraint programming
in whichwe distinguish
betweendecision variables, whichweare free to set, and
stochastic (or observed)variables, whichfollowsomeprobability distribution.
Stochastic constraint programs
Wedefine a numberof modelsof stochastic constraint programming
of increasing complexity.In an one stage stochastic constraint satisfaction problem(stochastic CSP),the decision variables are set before the stochastic variables are
given values. This can modelsituations in whichwe must
Constraintsare defined(as in traditional constraint satisfaction) by relations of allowedtuples of values. Constraints
can, however,be implemented
with specialized and efficient
algorithms for consistency checking. The stochastic variables independentlytake values with probabilities given by
a fixed probabilitydistribution. Wediscuss later howto relax these assumptions, and howthis modelcompareswith
related frameworks
like mixedconstraint satisfaction. Aone
stage stochasticCSPis satisfiable iff there exists valuesfor
the decision variables so that, given randomvalues for the
stochastic variables, the probabilitythat all the constraints
are satisfied equals or exceedssomethreshold probability,
8. Theprobabilistic satisfaction of constraints allowsus to
ignore worlds(values for the stochastic variables) whichare
too rare to require consideration.
In a twostage stochastic CSP,there are twosets of decision variables, Vatand Vd2,and twosets of stochastic variables, Vet and V82.Theaim is to find values for the variables in Vat, so that given randomvalues for Vst, we can
find values for Va2,so that givenrandomvalues for V82,the
probabilitythat all the constraintsare satisfied equalsor exceeds 8. Notethat the values chosenfor the secondset of
decision variables Va2are conditionedon both the values
chosenfor the first set of decision variables Vat and on the
randomvalues given to the first set of stochastic variables
Vst. This can modelsituations in whichitems are produced
and can be consumedor put in stock for later consumption.
Future productionthen dependsboth on previous production
(earlier decision variables) and on previousdemand
(earlier
stochastic variables). A mstage stochastic CSPis definedin
an analogouswayto one and two stage stochastic CSPs.
A stochastic constraint optimization problem(stochastic
COP)is a stochastic CSPplus a cost function defined over
the decision and stochastic variables. Theaim is to find a
solution that satisfies the stochastic CSPwhichminimizes
(or, if desired, maximizes)the expectedvalue of the cost
function.
Copyright@2000,American
Associationfor Artificial Intelligence(www.aaai.org).
All rights reserved.
129
Production planning example
The followingmstage stochastic constraint programmodels
a simple mquarter production planning problem. In each
quarter, there is a equal chancethat we will sell anywhere
between100 and 105 copies of a book. To keep customers
happy,wewantto satisfy demandin all mquarters with 80%
probability. At the start of each quarter, wemustdecidehow
manybooksto print for that quarter. This problemcan be
modellingby a mstage stochastic CSP.There are mdecision variables, zi representingproductionin each of the ith
quarter. Thereare also ra stochastic variables, Yl representing demandin the ith quarter. Thesetakes values between
100and 105with equal probability. Thereis a constraint to
ensurefirst quarter productionmeetsfirst quarter demand:
Xl ~> Yl
Thereis a also constraint to ensure secondquarter production meetssecondquarter demandeither plus any unsatisfied
demand
fromthe first quarter or less any stock carried forwardfromthe first quarter:
x2 > y2 + (yl - xl)
Andthere is a constraint to ensure jth quarter production
(j >_ 2) meetsjth quarter demandeither plus any unsatisfied demandfromearlier quarters or less any stock carried
forwardfromearlier quarters:
j-1
i=l
Wemust satisfy these mconstraints with a threshold probability 0 = 0.8. This stochastic CSPhas a numberof solutions includingxl = 105for each i (i.e. alwaysproduceas
manybooks as the maximum
demand). However,this solution will tend to producebookssurplus to demandwhichis
undesirable.
Supposestoring surplus book costs $1 per quarter. We
can define a mstage stochastic COPbasedon this stochastic
CSPin whichweadditionally miminizethe expectedcost of
storing surplus books. Asthe numberof surplus booksin the
jth quarter is min(~"~=
1 xi - Yi, 0), wehavea cost function
overall quartersof:
m
j
all the stochastic variables. In a twostage stochastic CSP,
Vcontainsthe first set of decisionvariables, the first set of
stochastic variables, then secondset of decision variables,
andfinally the secondset of stochasticvariables.
Apolicyis a tree withnodeslabelled withvariables, starting withthe first variablein Vlabelling the root, andending
with the last variable in V labelling the nodesdirectly above
the leaves. Nodeslabelled with decision variables have a
single child, whilst nodeslabelled with stochastic variables
have one child for every possible value. Edgesin the tree
are labelled with values assigned to the variable labelling
the nodeabove.Finally, each leaf nodeis labelled with 1 if
the assignmentof values to variables along the path to the
root satisfies all the constraints, and 0 otherwise.Eachleaf
node correspondsto a possible world and has an associated
probability; if 81 is the ith stochastic variable on a path to
the root, di is the valuegivento si on this path(i.e. the label
of the followingedge), and prob(8i = di) is the probability that sl =di, then the probabilityof this worldis simply
l-IiProb(8i = dl). Wedefine the satisfaction of a policy
as the sumof the leaf values weightedby their probabilities. Apolicysatisfies the constraintsiff its satisfaction is
at least 0. Astochastic CSPis satisfiable iff there exists a
policy which satisfies the constraints. WhenS = {} and
/9 = 1, this reducesto the traditional definitionof constraint
satisfaction. Theoptimalsatisfaction of a stochastic CSP
is the maximum
satisfaction of all policies. For a stochastic
COP,the expectedvalue of a policy as sumof the objective
valuationsof each of the leaf nodesweightedby their probabilities. Apolicyis optimalif it satisfies the constraintsand
maximizes
(or, if desired, minimizes)the expectedvalue.
Consider again the production planning problemand a
two-quarter policy that sets xl = 104 and if yl > 100then
x2 = Yl + 1 else Yl = 100 and z2 = 100. Wecan represent
this policyby the following(partial) tree:
~1104
10~~~~-~1
x~106 xl05
10~0~Y2
i:l
Semantics
A stochastic constraint satisfaction problemis a 6-tuple
(V, S, D, P, C, 0) whereV is a list of variables, S is the
subset of V whichare stochastic varibles, D is a mapping
from V to domains,P is a mappingfrom S to probability
distributions for the domains,C is a set of constraints over
V,and0 is a thresholdprobabilityin the interval [0, 1]. Constraints are defined(as in traditional constraintsatisfaction)
by a set of variables anda relation giving the allowedtuples
of values for these variables. Variablesare set in the order
in whichthey appear in V. Thus, in an one stage stochastic CSP,Vcontains first all the decisionvariables and then
130
O0
x~104x ~ x~102 x~100
y2 .........
/~
01 11 11
000000 1 1 1 1 1 1
Bydefinition, eachof the leaf nodesin this tree is equally
probable. Thereare 62 leaf nodes, of whichonly 7 are labelled 0. Hence,this policy’s satisfaction is (36 - 7)/36,
and the policy satisfies the constraints as this just exceeds
0=0.8.
Complexity
Constraint satisfaction is NP-complete
in general. Notsurprisingly, stochastic constraint satisfaction movesus up the
complexityhierarchy. It maytherefore be useful for modelling problemslike reasoning underuncertainty whichlie
in these higher complexityclasses. Weshowhowa number
of satisfiability problemsin these higher complexityclasses
reduceto stochastic constraintsatisfaction. In eachcase, the
reduction is very immediate.Note that each reduction can
be restricted to stochastic CSPson binary constraints using
a hiddenvariable encodingto mapnon-binaryconstraints to
binaryconstraints. Thehiddenvariablesare addedto the last
stage of the stochastic CSP.
PP, or probabilistic polynomialtime is characterized by
the PP-completeproblem,MAJSAT
whichdecides if at least
half the assignmentsto a set of Booleanvariables satisfy a
given clausal formula. This can be reducedto a one stage
stochastic CSPin which there are no decision variables,
the stochastic variables are Boolean,the constraints are the
clauses, the twotruth valuesfor eachstochastic variable are
equally likely and the threshold probability 0 = 0.5. Anumber of other reasoningproblemslike plan evaluationin probabilistic domainsare PP-complete.
NPPPis the class of problemsthat can be solved by nondeterministic guessing a solution in polynomialtime (NP)
and then verifyingthis in probabilistic polynomial
time(PP).
Givena clausal formula, E-MAJSAT
is the problemof deciding if there exists an assignmentfor a set of Booleanvariables so that, given randomizedchoices of values for the
other variables, the formulais satisfiable withprobabilityat
least equal to somethreshold O. This can be reduced very
immediatelyto an one stage stochastic CSP. A numberof
other reasoningproblemslike finding optimal size-bounded
PP-com
plans pinlete.
uncertain domainsare NP
PSPACE
is the class of problemsthat can be solved in
polynomialspace. Note that NPC_ PP C_ NPPP C_ PSPACE.
SSAT, or stochastic satisfiability is an exampleofa PSPACEcomplete problem. In SSAT, we are given a clausal formulawith ra alternating decision and stochastic variables,
and mustdecide if the formulais satisfiable with probability at least equal to somethreshold 0. This can be immediately reducedto a mstage stochastic CSP.Anumberof other
reasoningproblemslike propositional STRIPSplanningare
PSPACE-complete.
Complete algorithms
Wepresent a backtrackingalgorithm, whichis then extended
to a forwardcheckingprocedure.
Backtracking
Weassumethat variables are instantiated in order. However, if decisionvariables occur together, wecan choosein
whichorder to instantiate them. A branchingheuristic like
fail first maytherefore be useful to order decisionvariables
whichoccur together. Onmeetinga decision variable, the
backtracking(BT) algorithmtries each value in its domain
in turn. Themaximum
value foundis returned to the previous recursive call. Onmeetinga stochastic variable, the BT
algorithmtries eachvalue in turn, and returns the sumof the
all answersto the subproblems
weightedby the probabilities
of their occurrence.At any time, if instantiating a decision
or stochastic variable breaks a constraint, we return 0. If
wemanageto instantiate all the variables withoutbreaking
any constraint, we return 1. The algorithmcan be trivially
adaptedto record the optimalpolicy.
131
procedureBT(i,Ol,Oh
ifi > n thenreturn 1
0:=O
q:=l
for each dj 6 D(zi)
ifxl E S then
p := prob(xi ~ dj)
q:=q-p
ff consistent(zi -} dj ) then
0 := 0 +px BT(i + 1~, o~-o-v
’ o~-o~
p
p J
if 8 > Ohthenreturn0
if 0 + q < 0l then return 0
else
if consistent(xi --~ dj) then
0 := max(O,BT(i+ 1,max(0, Ot),Oh))
if 0 > Ohthenreturn0
return 0
Figure 1: The backtracking (BT) algorithm for stochastic
CSPs.Thealgorithmis called with the search depth, i and
with upperand lowerbounds,8h and 01. If the optimalsatisfaction lies betweenthese bounds,BTreturns the exact satisfaction. If the optimalsatisfaction is Ohor more,BTreturns
a value greaterthan or equalto Oh.If the optimalsatisfaction
is 0~or less, BTreturnsa valueless than or equal to Or. S is
the set of stochasticvariables.
As in the Davis-Putnam
like algorithmfor stochastic satisfiability (Littman, Majercik, &Pitassi 2000), upperand
lower bounds, OhandOt are used to prune search. Bysetting Ot = Oh= 0, we can determine if the optimal satisfaction is at least 0. Alternatively, by setting Ot = 0 and
Oh = 1, we can determine the optimal satisfaction. The
calculation of upperand lowerboundsin recursive calls requires someexplanation. Supposethat the current assignmentto a stochastic variable returns a satisfaction of 0o. We
can safely ignore other values for this stochastic variable if
0 + p x 00 >_ Oh. That is, if 0o --> 0h--0
p " This gives the
upperboundin the recursive call to BTon a stochastic variable. Alternatively,wecannothopeto satisfy the constraints
adequatelyif O +p x Oo+q < Ot as q is the maximum
that
the remainingvalues can contribute to the satisfaction. That
is, if Oo
This gives the lower boundin the re- < 0~-0-q.
p
cursive call to BTon a stochastic variable. Finally, suppose
that the current assignmentto a decision variable returns a
satisfaction of 0. If this is morethat 0l, then any other values mustbetter 0 to be part of a better policy. Hence,we
can replace the lowerboundin the recursive call to BTon
a decision variable by max(0,Or). Note that value ordering heuristics mayreducesearch. For decision variables, we
shouldchoosevaluesthat are likely to return the optimalsatisfaction. For stochastic variables, we shouldchoosevalues
that are morelikely.
Forward checking
The ForwardChecking(FC) procedure is based on the
algorithm.Oninstantiating a decisionor stochastic variable,
the FCalgorithmchecks forwardand prunes values fromthe
domainsof future decision and stochastic variables which
breakconstraints. Checkingforwardsfails if a stochastic or
decision variable has a domainwipeout,or if a stochastic
variable has so manyvalues removedthat we cannot hope
to satisfy the constraints. As in the regular forwardchecking algorithm, we use an 2-dimensionalarray, prune(i,j)
to record the depth at whichthe value dj for the variable xi
is removedby forwardchecking.This is used to restore values on backtracking.In addition, each stochastic variable,
zi has an upperbound,qi on the probability that the values
left in its domaincancontribute to a solution. When
forward
checkingremovessomevalue, dj from xl, we reduce ql by
prob(xi --} dj), the probability that xi takes the value dj.
This reduction of qj is undoneon backtracking. If forward
checkingever reducesqi to less than 0t, wecan immediately
backtrackas it is nowimpossibleto set xi and satisfy the
constraints adequately.
procedureFC(i,Ot,Oh)
ifi > n then return 1
O:=O
for each dj E D(xi)
ifprune(i,j) = 0 then
ifcheck(xi -+ dj,Ot)then
if xi E S then
p := prob(zi ~ dj)
qi := qi - P
0 := O+p×
FC(i + -°)
1,°’-°-~’,°~
restore(i)
if 0 + qi < Ot then return 0
if 0 > Ohthenreturn 0
else
0 := max(O,FC(i+ 1,max(O,Ot),Oh))
restore(i)
if 0 > Ohthenreturn0
else restore(i)
return 0
Phasetransition behavior
As in other search problems(Cheeseman,Kanefsky,&Taylor 1991),stochastic constraint programsdisplay phasetransition behavior.Toinvestigate such behavior,wehavedevelopeda modelof randomstochastic constraint programswith
four parameters:the numberof variables n, the domainsize
m, the constraint density Pt, and the constraint tightness/~.
For simplicity, weassumethat decision and stochastic variables strictly alternate, and that eachof the rn valuesfor a
stochasticvariable are equallylikely. It wouldbe interesting
to relax these assumptions.
Binary constraints are generated betweentwo decision
variables or one decision and one stochastic variable. As
with the usual models of randomCSPs, Pl and pz can be
either probabilities or fractions. For example,we can construet a fraction Pt of the possible binary constraints. Alternatively, wecan add each possible binary constraint with
probability/71.Similarly, whengeneratingconflict matrices,
either we generate a fixed fraction ofpz.mz nogoodsor we
add each of the m2 possible nogoodswith probability p2. To
prevent trivially insoluble problems,we recommend
"flawless" conflict matricesare generatedso that each value has
at least one support(Maclntyreet al. 1998).
In Figure3, weplot the averageoptimalsatisfaction as the
constraint densityand tightness is varied. Asin traditional
constraint satisfaction, weobservea ridge alongwhichthere
is a rapid changein the satisfaction of problems.As might
be expected,there is a complexitypeakin the cost to solve
problemsalong this ridge. In Figure 4, weplot the search
cost to find the optimalsatisfaction. Althoughthis is an optimization and not a decision problemas often studied in
phasetransition experiments,we observea complexitypeak
along the ridge. The search cost for the decision problemof
determiningif a policy exists to meeta given fixed threshold 0 displaysa similar (but slightly lower)complexitypeak.
The runtime(as well as nodes visited) for FCdominates
at least a factor of 2 on all but the easiest problems.
procedurecheck(zl --+ dj,Ot)
for k =i + l to n
dwo:= true
fordt E D(zk)
if prune(k, i) = 0 then
ifinconsistent(xl ~ dj,xk ~ all)then
prune(k,i) := i
ifzk E S then
qk := qk-- prob(xk ~ dr)
ifqk < Ol then return false
else dwo:= false
if dwothenreturn false
return true
procedurerestore(i)
for j = i +l to n
ford~ ~ D(xj)
if prune(j, k) =then
prune(j, k) =
ifxj E S then qj := qj+ prob(zj --+ dR)
Figure 2: The forwardchecking(FC) algorithmfor stochastic CSPs.The algorithmis called with the search depth, i
and with upper and lower bounds, Ohand Ol. S is the set
of stochastic variables. The array qi is an upper boundon
the probability that the stochastic variable xi satisfies the
constraintsandis initially set to 1, whilst prune(i,d) is the
depthat whichthe value d is prunedfromzi and is initially
set to 0 whichindicates that the valueis not yet pruned.
132
Extensions
n=lO,m=3 --
0.6
0.4
o.~
1
8
Figure3: Averageoptimalsatisfaction (z-axis) against constraint density(x-axis) and tightness (y-axis). Problems
10 variables, each with 3 values. 100problemsare ~enerated
at eachvalue of pl andp2 from0 to 1 in steps of i’6.
n=lO, m=3 -nodes
visited
70000
6OOO0
5OOO0
40000
3O0O0
20000
o.
I__~ p~
Figure 4: Meannodes visited by FCcomputingthe optimal
satisfaction (z-axis) against constraint density(x-axis)
tightness(y-axis).
Approximationprocedures
There are a numberof methodsfor approximatingthe answer to a stochastic constraint program.For example, we
can replace the stochastic variables in a stochastic CSPby
their mostprobablevalues (or in ordered domainslike integers by their medianor integer meanvalues), and then
solve (or approximatethe answerto) the resulting traditional constraint satisfaction problem.Similarly, wecanestimate the optimalsolution for a stochastic COPby replacing the stochastic variables by their mostprobablevalues
and then finding (or approximating
the answerto) the resulting traditional constraint optimizationproblem.Wecan also
use MonteCarlo samplingto test a subset of the possible
worlds. For example, we can randomlygenerate values for
the stochastic variablesaccordingto their probabilitydistribution. It wouldalso be interesting to developlocal search
procedures like GSAT
and WalkSAT
which explore the"policy space"of stochastic constraint programs.
133
Wehave assumedthat stochastic variables are independent.
Thereare problemswhichmayrequire us to relax this restriction. For example,a stochastic variable representing
electricity demandmaydependon a stochastic variable representing temperature. It maytherefore be useful to combine stochastic programmingwith techniques like Bayes
networkswhichallow for conditional dependenciesto be efficiently andeffectively represented.Analternative solution
is to replace the dependentstochastic variables by a single
stochastic variable whosedomainis the productspaceof the
dependentvariables. This is only feasible whenthere are a
small numberof dependentvariables with small domains.
Wehavealso assumedthat the probability distribution of
stochastic variables is fixed. In particular, wehaveassumed
that it doesnot dependon earlier decisionvariables. Again,
there are problemswhichmayrequire us to relax this restriction. For example,the decision variable representing price
mayinfluence a stochastic variable representing demand.A
solution mayagain be to combinestochastic programming
with techniqueslike Bayesnetworks.
Wehave also assumedthat all variable domainsare finite. Thereare problemswhichmayrequire us to relax this
restriction. For example,in schedulingpowerstations, we
mayuse 0/1 decision variables to modelwhether a power
station runs or not, but havecontinuous(observed)variables
to modelfuture electricity demands.A continuousprobability densityfunctioncouldbe associatedwiththese variables.
Similarly, a continuousdecision variable couldbe useful to
modelthe poweroutput of the powerstations that we decide
to operate. Interval reasoningtechniquescould be extended
to deal withsuch variables.
Related work
Stochastic constraint programmingis inspired by both
stochastic integer programming
and stochastic satisfiability
(Littman, Majercik, &Pitassi 2000). It shares the advantages that constraint programming
has over integer programming(e.g. global constraints, non-linear constraints, and
constraint propagation). It also shares the advantagesthat
constraint programming
has over satisfiability (e.g. global
constraints, and arithmetic constraints). Optimizationis
likely to play a fundamental
role in stochastic constraintprogramming.
This adds to the distinction with stochastic satisfiability as the latter purelyconcernsdecisionproblems.
Mixedconstraint satisfaction (Fargier, Lang, &Sehiex
1996)is closely related to one stage stochastic constraint
satisfaction. In a mixedCSP,the decision variables are set
after the stochastic variables are givenrandomvalues. In addition, the randomvalues are chosenuniformly.In the ease
of full observability,the aimis to find conditionalvaluesfor
the decisionvariablesso that wesatisfy all possibleworlds.
In the case of no observability,the aimis to find valuesfor
the decision variables so that wesatisfy as manypossible
worlds.Anearlier constraint satisfaction modelfor decision
makingunderuncertainty(Fargier et al. 1995)also included
a probabilitydistribution overthe spaceof possibleworlds.
Constraintsatisfaction has beenextendedto include prob-
abilistie preferences on the values assigned to variables
(Shazeer, Littman, &Keim1999). Associatedwith the values for eachvariable is a probability distribution. A "best"
solution to the constraint satisfaction problemis then found.
This maybe the maximum
probability solution (whichsatisfies the constraints and is mostprobable),or the maximum
expectedoverlapsolution (whichis mostlike the true solution). Thelatter can be viewedas the solution whichhas the
maximum
expected overlap with one generated at random
using the probability distribution. The maximum
expected
overlap solution could be found by solving a suitable one
stage stochastic constraint optimizationproblem.
Branchingconstraint satisfaction (Fowler&Brown2000)
modelsproblemsin whichthere is uncertainty in the number
of variables. For example,we can modela nurse rostering
problemby assigningshifts to nurses. Branchingconstraint
satisfaction then allowsus to deal with the uncertainty in
whichnurses are available for duty. Wecan represent such
problemswith a stochastic CSPwith a stochastic 0/1 variable for eachnurserepresentingtheir availability.
Anumberof extensionsof the traditional constraint satisfaction problemmodelconstraints that are uncertain, probabilistic or not necessarily satisfied. For example,in partial constraint satisfaction we maximize
the numberof constraints satisfied (Freuder&Wallacs1992). Asa secondexample,in probabilisticconstraintsatisfaction eachconstraint
has a certain probability independentof all other probabilities of being part of the problem(Fargier &Lang1993).
a third example,both valued and semi-ringbasedconstraint
satisfaction (Bistarelli et al. 1996)generalizesprobabilistic
constraint satisfaction as well as a numberof other frameworks.In semi-ringbasedconstraint satisfaction, a value is
associatedwith each tuple in a constraint, whilst in valued
constraint satisfaction, a value is associatedwith eachconstraint. However,noneof these extensions deal with variables that mayhaveuncertain or probabilistic values. Indeed, stochastic constraint programming
can easily be combined with most of these techniques. For example,we can
definestochastic partial constraint satisfaction in whichwe
maximize
the numberof satisfied constraints, or stochastic
probabilisticconstraint satisfaction in whicheachconstraint
has an associatedprobability of beingin the problem.
Stochastic constraint programsare closely related to
Markovdecision problems (MDPs).These have been very
influential in AIof late for dealingwithsituations involving
reasoningunder uncertainty (Kaelbling, Littman, &Cassandra 1998). Stochastic constraint programming
could be used
to modelproblemswhichlack the Markovproperty that the
next state and rewarddependonly on the previousstate and
action taken. Stochastic constraint optimizationcould also
be used to modelmore complexreward functions than the
(discounted)sumof individual rewards. Onthe other hand,
MDPs
can, at least in theory, be solvedefficiently using linear programming(Littrnan, Dean, &Kaelbling 1995) and
canhaveinfinite state spaces.
cision variables (whichwe can set) and stochastic variables
(whichfollow someprobability distribution). This frameworkis designedto take advantageof the best features of traditional constraint satisfaction, stochastic integer programming,and stochastic satisfiability. It canbe used to modela
widevariety of decision problemsinvolvinguncertainty and
probability. Wehavegiven a semanticsfor stochastic constraint programsbased uponpolicies. Thesedeterminehow
decision variables are set dependingon earlier decision and
stochastic variables. Wehave proposed a numberof complete algorithmsand approximationproceduresfor stochastic constraint programming.Using these algorithms, we
have observedphasetransition behaviorin stochastic constraint programs.Interestingly, the cost of bothoptimization
and satisfaction peaksalong the satisfaction phaseboundary. Finally, we have discussed a numberof extensions of
stochastic constraint programming
to relax assumptionslike
the independencebetweenstochastic variables.
Acknowledgements
The author is an EPSRCadvanced research fellow. He
thanks the other membersof the APESresearch group
(http://apes.cs.strath.ac.uk/), especially Ian Gentfor his
helpful discussions.
Conclusions
Wehave proposed stochastic constraint programming,an
extension of constraint programming
to deal with both de-
134
References
Bistarelli, S.; Fargier, H.; Montanari,U.; Rossi,F.; Schiex,
T.; and Verfaillie, G. 1996. Semi-ring based CSPsand
valued CSPs:Basic properties and comparison.In Jample,
M.; Freuder, E., and Maher, M., eds., Over-Constrained
Systems, 111-150. Springer-Verlag. LNCS1106.
Cheeseman,P.; Kanefsky,B.; and Taylor, W.1991. Where
the really hard problemsare. In Proc. of the 12th IJCAL
331-337.Int. Joint Conference
on Artificial Intelligence.
Fargier, H., and Lang,J. 1993. Uncertaintyin constraint
satisfaction problems:a probabilistic approach.In Proc.
ofECSQARU.
Springer-Verlag. LNCS747.
Fargier, H., Lang,J.; Martin-Clouaire,R.; and Schiex,T.
1995.A constraint satisfaction framework
for decision under uncertainty. In Proc. of the 11th Int. Conferenceon
Uncertainty
in Artificial Intelligence.
Fargier, H.; Lang, J.; and Schiex, T. 1996. Mixedconstraint satisfaction: a framework
for decision problemsunder in completeinformation.In Proc. of the 13th National
Conference
on Artificial Intelligence.
Fowler, D., and Brown,K. 2000. Branching constraint
satisfaction problemsfor solutions robust under likely
changes.In Proc. of 6th lnt. Conferenceon Principles and
Practices of Constraint Programming.
Springer-Verlag.
Freuder,E., and Wallacs,R. 1992.Partial constraint satisfaction. Artificial Intelligence58:21-70.
Kaelbling, L.; Littman, M.; and Cassandra,A. 1998. Planning and acting in partially observablestochastic domains.
Artificial lntelligence 101(I-2):99-134.
Littman, M.; Dean, T., and Kaelbling, L. 1995. On the
complexityof solving Markovdecision problems.In Proc.
of the 8th AnnualConferenceon Uncertaintyin AL
Littman,M.; Majereik,S.; and Pitassi, T. 2000.Stochastic
Booleansatisfiability. Journalof Automated
Reasoning.
Maclntyre,E.; Prosser, P.; Smith, B.; and Walsh,T. 1998.
Random
constraint satisfaction: Theorymeetspractice. In
4th Int. Conferenceon Principles and Practices of Constraint Programming
(CP-98), 325-339. Springer.
Shazeer, N.; Littman, M.; and Keim,G. 1999. Constraint
satisfaction with probabilistic preferenceson variable values. In Proc. of the 16th NationalConference
on Artificial
Intelligence.
135