Lectures
TDDD10AIProgramming
CooperationAndCoordination2
CyrilleBerger
1AIProgramming:Introduction
2Introductionto
3AgentsandAgents
4Multi-Agentand
5Multi-AgentDecision
6CooperationAndCoordination
7CooperationAndCoordination2
8MachineLearning
9AutomatedPlanning
10PuttingItAll
2/73
Lecturecontent
TaskAllocation
AssignmentProblems
LinearSumAssignmentProblemFormulation
andHungarianMethod
SerialAssignmentProblem
ContraintSatisfactionsProblems
CentralizedContraintSatisfactionsProblems
DistributedConstraintOptimizationProblems
Task-SpecificationTrees
3/73
TaskAllocation
AssignmentProblems
Assigningnagentstomtasks
Forexample,passengerstodrivers(carsharingdomain)
Cleaningrobotstorooms
Weddingmatchingwithindividualsympathymeasure
Ambulancestocivilians,Firebrigadtofire
…
AssignmentProblems
IndividualAssignmentCostscᵢⱼoccurfor
eachassignmentofagentitotaskj
Theobjectiveistofindaone-to-one
matchingfromagentstotasksattheleast
possibletotalcost
6
AssignmentProblems-Example
AssignmentProblems-ExampleDynamicRideSharing
GivenannxmcostmatrixC=(cᵢⱼ)match
eachrowtoadifferentcolumninsucha
waythatthesumofthecorresponding
entriesisminimized
Example:Fourpersonshavetobeassignedto
4cleaningjobs,eachonehasdifferentcosts:
Thesingletripsof
4drivers
BathroomFloorsWindowsKitchen
Gabi 8€
10€ 17€
9€
Malte 3€
8€
5€
6€
Tom 10€
12€ 11€
9€
Patrick 6€
13€ 9€
7€
Theirsharedrides
computedbyanAPsolver
Costscᵢⱼareassumednon7
8
DifferentRepresentationsofAssignmentProblems
PermutationMatrix
BijectiveMappingbetween
twosets!
Representationas
Note:Therearen!validsolutions.
Forexamplen=10,10!=3.63e6
Matrix
Representation
Graph
Representation
9
10
AssignmentProblemFormulation
AssignmentMatrix(xij):
Formulation:
LinearSumAssignmentProblemFormulationandHungarianMethod
12
HeterogenousAgents
HungarianMethod
Whathappenifnotallagentscanaccomplish
thetasks
TheHungarianMethodforsolvingLinear
SumAssignmentProblems
Theorem1:
Whenadding(orsubtracting)aconstanttoeveryelementof
anyrow(orcolumn)ofthecostmatrix(cᵢⱼ)thenanassignment
whichminimizesthetotalcostforthenewmatrixwillalso
minimizetheoriginalcostmatrix.
Theorem2:
Ifallcᵢⱼ≥0andthereexistsasolutionsuchthat∑cᵢⱼxᵢⱼ=0then
thesolutionisoptimal.
Wherexᵢⱼ=1ifagentiisassignedfortaskj,
qᵢⱼ=1ifagentiisqualifiedfortaskj,andcᵢⱼthe
TheHungarianMethod(originalversion)
solvesLSAPsinO(n⁴)(KuhnandMunkers)
13
HungarianMethod(1/7)
Step1:RowReduction
14
HungarianMethod(2/7)
Step2:ColumnReduction
Subtracttheminimumentryofeachrowfrom
allentriesinthisrow
Subtracttheminimumentryofeachcolumn
fromallentriesinthiscolumn
BathroomFloorsWindowsKitchen
Gabi 8
10
17
9
Malte 3
8
5
6
Tom 10
12
11
9
Patrick6
13
9
7
BathroomFloorsWindowsKitchen
Gabi 0
2
9
1
Malte 0
5
2
3
Tom 1
3
2
0
Patrick0
7
3
1
BathroomFloorsWindowsKitchen
Gabi 0
2
9
1
Malte 0
5
2
3
Tom 1
3
2
0
Patrick0
7
3
1
BathroomFloorsWindowsKitchen
Gabi 0
0
7
1
Malte 0
3
0
3
Tom 1
1
0
0
Patrick0
5
1
1
15
16
HungarianMethod(3/7)
Step3:ZeroAssignment
HungarianMethod(4/7)
Finally,allzerosareeithercrossedoutor
assignedandthenumberofassigned
zerosisnandthusthesolutionisoptimal!
Rows:Examineeachrow(startingwiththefirstone)untilfindingarowthat
containsexactlyonezero.Markthiszeroastemporaryassignmentandcross
allentriesinthecolumnwheretheassignmenthasbeenmade.
BathroomFloorsWindowsKitchen
Gabi 0
0
7
1
Malte 0
3
0
3
Tom 1
1
0
0
Patrick 0
5
1
1
BathroomFloorsWindowsKitchen
Gabi 0
0
7
1
Malte 0
3
0
3
Tom 1
1
0
0
Patrick 0
5
1
1
Bathroom FloorsWindowsKitchen
Gabi 0
0
7
1
Malte 0
3
0
3
Tom 1
1
0
0
Patrick0
5
1
1
Columns:Examineeachcolumn(startingwiththefirstone)untilfindinga
columnthatcontainsexactlyonezero.Markthiszeroastemporaryassignment
andcrossallentriesintherowwheretheassignmenthasbeenmade.
Result:Gabidoesthefloors,Maltedoes
thewindows,Tomdoesthekitchen,and
Patrickdoesthebathroom.
17
HungarianMethod-Anotherexample(1/2)
18
HungarianMethod-Anotherexample(2/2)
5driverscanpick-uppassengersfrom5areasatdifferentcosts
Step3a:ZeroRowAssignment
Wiehre HerdernMerzhausenZähringenStühlinger
Gabi 6
12
3
11
15
Malte 4
2
7
1
10
Tom 8
11
10
7
11
Patrick 16
19
12
23
21
Robert 9
5
7
6
10
WiehreHerdernMerzhausenZähringenStühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick3
7
0
11
5
Robert 3
0
2
1
1
Step1:Rowreduction
Wiehre HerdernMerzhausenZähringenStühlinger
Gabi 3
9
0
8
12
Malte 3
1
6
0
9
Tom 1
4
3
0
4
Patrick 4
7
0
11
9
Robert 4
0
2
1
5
Step3b:ZeroColumnAssignment
WiehreHerdernMerzhausenZähringenStühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick3
7
0
11
5
Robert 3
0
2
1
1
Step2:Columnreduction
Wiehre HerdernMerzhausenZähringenStühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick 3
7
0
11
5
Robert 3
0
2
1
1
19
20
HungarianMethod(5/7)
Step4:Drawtheminimumnumberoflinestocoverallzero’s
(a)Markallrowsinwhichtheassignmenthasnotbeendone
HungarianMethod(6/7)
Step5:Selectthesmallestelementfromthe
uncoveredelements
(a)Subtractthiselementfromalluncoveredones
WiehreHerdernMerzhausenZähringenStühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick3
7
0
11
5
Robert3
0
2
1
1
WiehreHerdernMerzhausen Zähringen Stühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick 3
7
0
11
5
Robert 3
0
2
1
1
(b)Seethepositionofzerosinthemarkedrowsandmarkthecorrespondingcolumns
WiehreHerdernMerzhausenZähringenStühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick3
7
0
11
5
Robert3
0
2
1
1
(b)Addthiselementtoallelementswhichareattheintersection
oftwolines
WiehreHerdernMerzhausen Zähringen Stühlinger
Gabi 0
7
0
6
6
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick 1
5
0
9
3
Robert 3
0
2
1
1
(c)Markallotherrowswithazerointhosecolumns
WiehreHerdernMerzhausenZähringenStühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick3
7
0
11
5
Robert3
0
2
1
1
21
HungarianMethod(7/7)
22
PracticalconsiderationsabouttheHungarianMethod
Negativecostscanbeconvertedby
addingtoeachelementofCthevalue
v=-min{cij}
Theobjectivefunctionistobe
Step6:Nowwehave
increasedthenumberofzeros
RepeatStep3.
WiehreHerdernMerzhausenZähringen Stühlinger
Gabi 0
7
0
6
6
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick1
5
0
9
3
Robert 3
0
2
1
1
Multiplyallcijby-1or
replaceeachcijbycmax-cijwherecmax=max{cij}
Therearemoreagents(n>m)or
moretasks(m>n)
addasufficientnumberof‘‘dummy’’tasksoragents
(whicheverisinshortersupply)withcostsof0(or
inf)
23
24
Multi-RobotExploration
Arewereallyinterestedin
minimizingthesumofindividual
costs?
SerialAssignmentProblem
25
TheLinearBottleneckAssignmentProblem(1/2)
TheLinearBottleneckAssignmentProblem(2/2)
TheLinearSumAssignmentProblem
minimizesthesummedtotalcostasif
tasksareexecutedinseries
Whenwehaveparallelmachines(MAS)
wewanttominimizethetotalcompletion
timeandthusthemaximaldurationofall
parallelexecutedjobs!
Wewanttominimizetheso-called
bottleneckobjectivefunction:
Canbesolvedin
polynomialtime,using
thethresholding
algorithm.
Whatabout
dependency
betweentasks?
max{1≤i≤n}(cᵢφ(i))
27
28
MultilevelGeneralizedAssignmentProblems(1/2)
MultilevelGeneralizedAssignmentProblems(2/2)
Appearsinthecontextoflarge-scaletask
allocation,e.g.,RoboCupRescue
Simulation!
Theproblemofassigningnagentsto
mtasksatllevels(i.e.timeslots)
Theresourcerequirement(consumption)
aᵢⱼₖdependentsontheagent,task,and
level
Asingleagentimayhavemorethan
onetaskassignedbutthesumof
neededresourcesaᵢⱼₖmaynotexceedbᵢ
Therearealsocostscᵢⱼₖofthe
ThisisNP-hard,anddeterminingwhetherafeasiblesolutionexistsisNPcomplete.
AllMultilevelversionsofLinearSumandLinearBottleneckAssignment
Problemsareintractable!
29
30
CaseStudy:Task-SequenceAllocationwithHumanintheLoop(1/3)
CaseStudy:Task-SequenceAllocationwithHumanintheLoop(2/3)
Coordinationofresponseteamsby
resourceallocation
Humanassistedtasksequenceassignment
MixedIntegerLinearProgramming
(MILP)solverbiasedbyhuman
expertise
TwoAdvantages:Fasterassignment
computationwithoptimalsequencing
31
32
CaseStudy:Task-SequenceAllocationwithHumanintheLoop(3/3)
ContraintSatisfactionsProblems
33
ConstraintOptimizationProblems
Applications
Insearchproblems,thestatedoesnot
haveastructure(everythingisinthedata
structure).InCSPs,statesareexplicitly
representedasvariableassignments.
ACSPconsistsof
Timetabling(classes,
rooms,times)
Configuration(hardware,cars,…)
Spreadsheets
Scheduling
Floorplanning
Frequencyassignments
…
asetofvariables{x₁,x₂,…,xₙ}to
values{d₁,d₂,…,dₖ}canbeassigned
suchthatasetofconstraintsoverthevariablesis
respected
ACSPissolvedbyavariableassignment
thatsatisfiesallgivenconstraints.
35
36
ExampleMapColoring:AustralianStatesandTerritories
CentralizedContraintSatisfactionsProblems
Coloreachoftheterritories/
statesred,greenorbluewithno
neighbouringregionhavingthe
samecolor
ConstraintGraph:nodesare
variablesarcsareconstraints
38
ExampleMapColoring
OnePossibleSolution
Variables:WA,NT,SA,Q,NSW,V,T
Values:{red,green,blue}
Constraints:adjacentregionsmusthavedifferentcolors,e.g.,NSW≠V
Solutionassignment:
{WA=red,NT=green,Q=red,NSW=green,V=red,SA
=blue,T=green}
39
40
BacktrackingSearchoverAssignments
Algorithm
Assignvaluestovariablesstep
bystep(orderdoesnotmatter)
Consideronlyonevariableper
searchnode!
DepthFirstSearch(DFS)with
single-variableassignments
iscalledbacktrackingsearch
Cansolven-queensforn≈25
41
Backtracking-ExampleMapColoring(1/4)
43
42
Backtracking-ExampleMapColoring(2/4)
44
Backtracking-ExampleMapColoring(3/4)
Backtracking-ExampleMapColoring(4/4)
45
ImprovingEfficiency:CSPHeuristics&PruningTechniques
46
ConstraintOptimizationProblem
ConstraintSatisfactionProblem(CSP)
Variableordering:Whichoneto
assignfirst?
Valueordering:Whichvaluetotry
first?
Trytodetectfailuresearly
Trytoexploitproblemstructure
Note:allthisisnotproblemspecific!
Objective:findanassignmentforallthe
variablesinthenetworkthatsatisfiesall
constraints
ConstraintOptimizationProblem
(COP)
Objective:findanassignmentforallthe
variablesinthenetworkthatsatisfiesall
constraintsandoptimizesaglobalfunction
Globalfunction=aggregation(typicallysum)
oflocalfunctions.F(x)=∑iFi(xi)
47
48
LinearAssignmentisaConstraintOptimizationProblem
Variations
Binary,ternary,oreven
higherarity
example:x<y,x+y<z,x+y+z<w...
Finitedomains(dvalues)=>
dnpossiblevariable
assignments
Infinitedomains(reals,
linearconstraints
nonlinearconstraints
example:log(x)<sin(y)
49
50
DistributedConstraintOptimizationProblem(DCOP)
DistributedConstraintOptimizationProblems
Ageneralmodelfordistributedproblemsolving
DCOPsarecomposedofagents,eachholding
oneormorevariables
Constraintsamongvariables(possiblyheldby
differentagents)assigncoststocombinationsof
valueassignments
Agentsassignvaluestotheirvariablesandtry
togenerateasolutionthatisgloballyoptimal
withrespecttothesumofthecostsofthe
constraints
52
DistributedConstraintOptimizationProblem(DCOP)
WhyDistributedsolvers?
Costofformalization:
Examples
Incentralizedsolving,allimaginableoptionshaveto
beformulatedbeforehand
Indistributedconstraintsatisfaction,onlyaminimalnumber
ofconstraintsareconsideredbyeachagent
TargetAllocation(AgentstoTargets)
Meetingscheduling
Difficulties
Privacyissues:
Noglobalcontrol/knowledge
Localizedcommunication
Limitedtime
Incentralizedscenariosthesolverseesallmeetings
andconstraints
Distributedsolutionscanbeconstructedinsuchaway
thatagentsonlyrevealinformationpiecemealwhen
evaluatingconstraints
Robustness:
Acentralizedsolvercreatesacentralpointof
Distributedsolvingallowsloadbalancingand
53
54
Example:MeetingScheduling
ConstraintNetworks
Aconstraintnetworkisformallydefinedbya
tuple<X,D,C>where:
X={X₁,X₂,...,Xₙ}isasetofdiscretevariables
D={D₁,D₂,...,Dₙ}isasetofvariabledomains,
whichenumerateallpossiblevaluesofthe
correspondingvariables
C={C₁,C₂,...,Cₙ}isasetofconstraints,where
eachconstraintCiisdefinedonsubsetSi⊂
Xandassociatedwithaarityr=|Sᵢ|(hardor
soft)
55
56
DCOPformalizationfortheMeetingSchedulingProblem
Example:MeetingScheduling
Asetofagentsrepresenting
Asetofvariablesrepresentingmeeting
startingtimesaccordingtoa
participant.
Hard
Startingmeetingtimesacrossdifferentagentsareequal
Meetingsforthesameagentarenon-overlapping.
Soft
Representagentpreferencesonmeetingstartingtimes.
Objective:findavalidschedulefor
themeetingwhilemaximizingthe
sumofindividuals’preferences
57
DCOPExample:MeetingScheduling
58
Example:TargetTracking(1/3)
Asetofsensorstrackingasetoftargetsinordertoprovideanaccurate
estimateoftheirpositions.
59
60
Example:TargetTracking(2/3)
Example:TargetTracking(3/3)
Sensorscanhavedifferentsensingmodalitiesthatimpactontheaccuracyof
theestimationofthetargets’positions.
Collaborationamongsensorsiscrucialtoimprovesystemperformance.
61
DCOPFormalizationfortheTargetTrackingProblem
62
DCOPAlgorithms
Complete
Agentsrepresentsensors
Variablesencodethedifferentsensing
modalitiesofeachsensor(positions...)
Constraints
Alwaysfindtheoptimal
Exhibitanexponentiallyincreasingcoordination
Verylimitedscalabilityongeneralproblems
Search-based&Synchronous:SyncBB,AND/ORsearch
Search-based&Asynchronous:ADOPT(Modietal.,2005),NCBB,AFB
DynamicprogrammingbasedDPOP(PetcuandFaltings,
relatetoaspecifictarget
representhowsensormodalitiesimpactsonthe
trackingperformance
LocalGreedyApproximative
Sacrifyoptimalityinfavorofcomputationaland
communicationefficiency
Activationprobabilitydomaindependentandneedstobe
Well-suitedforlargescaledistributedapplications,e.g.sensor
networksandrobotics
DSA-A(Zhangetal.,2005),MGM-1(Maheswaranetal.,2004)
Objective:
Maximizecoverageoftheenvironment
Provideaccurateestimationsofpotentiallydangerous
targets
63
64
Centralizedvsdecentralized
Usedecentralized
Privacyisaconcern
Thecostofsharinginformationishigherthan
exchangingconstraintmessages
Task-SpecificationTrees
Otherwiseusecentralized,but
makesureyourcentralnode
canbereplaced
65
TaskSpecificationTrees
Example
ATaskSpecificationTree(TST)
isadistributeddatastructure
withadeclarative
representationthatdescribes
acomplexmulti-agenttask.
AnodeinaTSTcorresponds
toatask.Ithasanode
interfacewithparametersand
asetofnodeconstraintsthat
restricttheparameters.
67
68
TaskSpecificationTreesStructure
Example
Therearecurrentlysix
typesofnodes:Sequence,
concurrent,loop,select,
goal,andelementary
action.
ATSTisassociatedwitha
setoftreeconstraints
expressingconstraints
betweentasksinthetree.
69
DelegationwithTaskSpecificationTrees
70
TSTSemanticsExample
ConstraintSatisfactionProblem
TSTSemantics
Scan1
ToallowtasksspecifiedasTSTstobedelegated,the
TSTsemanticsisdefinedintermsoftheCan
predicate.
Can(B,τ,[ts,te,…],cons)assertsthatanagentBhas
thecapabilitiesandresourcesforachievingataskτ
intheinterval[ts,te]withtheconstraintscons.
Ataskτ=(α,φ)consistsofanactionα,whichcanbe
compositeorelementary,andagoalφ,whichmaybe
empty.
Scan2
71
72
Summary
LinearAssignmentProblem:usedfor
problemwheretasksaresimpleandcan
beexecutedinanyorderandagentsare
homogenous
Otherwise,useaConstraint
SatisfactionandOptimizationSolver
Godecentralized,ifsharinginformationis
aproblem
TaskSpecificationTrees:arepresentation
ofacomplexsetoftasksanddelegation
73/73