Lectures
AIProgramming:Introduction
IntroductiontoRoboRescue
3
AgentsandAgentsArchitecture
4
Multi-AgentandCommunication
5
Multi-AgentDecisionMaking
6
CooperationAndCoordination1
7
CooperationAndCoordination2
8
MachineLearning
9
KnowledgeRepresentation
10
PuttingItAllTogether
1
TDDD10AIProgramming
CooperationAndCoordination1
CyrilleBerger
2
2/81
Lecturecontent
TaskAllocation
AssignmentProblems
LinearSumAssignmentProblemFormulationandHungarian
Method
SerialAssignmentProblem
ContraintSatisfactionsProblems
CentralizedContraintSatisfactionsProblems
DistributedConstraintOptimizationProblems
Task-SpecificationTrees
TaskAllocation,asummary
IndividualandGroupAssignment
3/81
TaskAllocation
AssignmentProblems
Assigningnagentstom
AssignmentProblems
Forexample,passengerstodrivers(carsharingdomain)
Cleaningrobotstorooms
Weddingmatchingwithindividualsympathy
Ambulancestocivilians,Firebrigadtofire
…
IndividualAssignmentCostscᵢⱼoccurfor
eachassignmentofagentitotaskj
Theobjectiveistofindaone-to-onematching
fromagentstotasksattheleastpossibletotal
cost
6/81
AssignmentProblems-Example
GivenannxmcostmatrixC=(cᵢⱼ)match
eachrowtoadifferentcolumninsucha
waythatthesumofthecorresponding
entriesisminimized
Example:Fourpersonshavetobeassignedto
4cleaningjobs,eachonehasdifferentcosts:
BathroomFloors WindowsKitchen
Gabi 8€
10€ 17€
9€
Malte 3€
8€
5€
6€
Tom 10€
12€ 11€
9€
Patrick6€
13€ 9€
7€
AssignmentProblems-ExampleDynamicRideSharing
Thesingletripsof
4drivers
Theirsharedrides
computedbyanAPsolver
Costscᵢⱼareassumednon7/81
8/81
DynamicRideSharing-HorizonProblem
DifferentRepresentationsofAssignmentProblems
BijectiveMappingbetweentwosets!
Matrix
Representation
9/81
Graph
Representation
10/81
PermutationMatrix
RepresentationasPermutation:
LinearSumAssignmentProblemFormulationandHungarianMethod
Note:Therearen!validsolutions.
Forexamplen=10,10!=3.63e6
11/81
AssignmentProblemFormulation
AssignmentMatrix
HeterogenousAgents
Whathappenifnotallagentscanaccomplish
thetasks
Formulation:
Wherexᵢⱼ=1ifagentiisassignedfortaskj,
qᵢⱼ=1ifagentiisqualifiedfortaskj,andcᵢⱼthe
13/81
HungarianMethod
14/81
HungarianMethod(1/5)
Step1:RowReduction
TheHungarianMethodforsolvingLinear
SumAssignmentProblems
Theorem
Whenadding(orsubtracting)aconstanttoeveryelementofany
row(orcolumn)ofthecostmatrix(cᵢⱼ)thenanassignmentwhich
minimizesthetotalcostforthenewmatrixwillalsominimizethe
originalcostmatrix.
Theorem
Ifallcᵢⱼandthereexistsasolutionsuchthat∑cᵢⱼxᵢⱼ=0thenthe
solutionisoptimal.
TheHungarianMethod(originalversion)
solvesLSAPsinO(n4)(KuhnandMunkers)
15/81
Subtracttheminimumentryofeachrowfromallentriesin
thisrow
BathroomFloorsWindowsKitchen
Gabi 8
10
17
9
Malte 3
8
5
6
Tom 10
12
11
9
Patrick6
13
9
7
BathroomFloorsWindowsKitchen
Gabi 0
2
9
1
Malte 0
5
2
3
Tom 1
3
2
0
Patrick0
7
3
1
16/81
HungarianMethod(2/5)
Step2:ColumnReduction
Subtracttheminimumentryofeachcolumnfromallentriesin
thiscolumn
BathroomFloorsWindowsKitchen
Gabi 0
2
9
1
Malte 0
5
2
3
Tom 1
3
2
0
Patrick0
7
3
1
BathroomFloorsWindowsKitchen
Gabi 0
0
7
1
Malte 0
3
0
3
Tom 1
1
0
0
Patrick0
5
1
1
HungarianMethod(3/5)
Step3:ZeroAssignment
Rows:Examineeachrow(startingwiththefirstone)untilfindingarowthat
containsexactlyonezero.Markthiszeroastemporaryassignmentandcrossall
entriesinthecolumnwheretheassignmenthasbeenmade.
BathroomFloorsWindowsKitchen
Gabi 0
0
7
1
Malte 0
3
0
3
Tom 1
1
0
0
Patrick 0
5
1
1
BathroomFloorsWindowsKitchen
Gabi 0
0
7
1
Malte 0
3
0
3
Tom 1
1
0
0
Patrick 0
5
1
1
Columns:Examineeachcolumn(startingwiththefirstone)untilfindinga
columnthatcontainsexactlyonezero.Markthiszeroastemporaryassignment
andcrossallentriesintherowwheretheassignmenthasbeenmade.
17/81
HungarianMethod(4/5)
Finally,allzerosareeithercrossedoutor
assignedandthenumberofassignedzerosis
nandthusthesolutionisoptimal!
BathroomFloorsWindowsKitchen
Gabi 0
0
7
1
Malte 0
3
0
3
Tom 1
1
0
0
Patrick0
5
1
1
18/81
HungarianMethod-Anotherexample(1/2)
5driverscanpick-uppassengersfrom5areasatdifferent
WiehreHerdernMerzhausenZähringenStühlinger
Gabi 6
12
3
11
15
Malte 4
2
7
1
10
Tom 8
11
10
7
11
Patrick 16
19
12
23
21
Robert9
5
7
6
10
Step1:Row
Result:Gabidoesthefloors,Maltedoesthe
windows,Tomdoesthekitchen,and
Patrickdoesthebathroom.
19/81
WiehreHerdernMerzhausenZähringenStühlinger
Gabi 3
9
0
8
12
Malte 3
1
6
0
9
Tom 1
4
3
0
4
Patrick 4
7
0
11
9
Robert4
0
2
1
5
Step2:Column
WiehreHerdernMerzhausenZähringenStühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick 3
7
0
11
5
Robert3
0
2
1
1
20/81
HungarianMethod-Anotherexample(1/2)
Step3a:ZeroRowAssignment
HungarianMethod(5/)
Step4:Drawtheminimumnumberoflinestocoverallzero’s
(a)Markallrowsinwhichtheassignmenthasnotbeendone
WiehreHerdernMerzhausen Zähringen Stühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick3
7
0
11
5
Robert 3
0
2
1
1
Wiehre HerdernMerzhausenZähringenStühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick 3
7
0
11
5
Robert 3
0
2
1
1
(b)Seethepositionofzerosinthemarkedrowsandmarkthecorrespondingcolumns
Step3b:ZeroColumnAssignment
Wiehre HerdernMerzhausenZähringenStühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick 3
7
0
11
5
Robert 3
0
2
1
1
WiehreHerdernMerzhausen Zähringen Stühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick3
7
0
11
5
Robert 3
0
2
1
1
(c)Markallotherrowswithazerointhosecolumns
Wiehre HerdernMerzhausenZähringenStühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick 3
7
0
11
5
Robert 3
0
2
1
1
21/81
HungarianMethod(6/7)
22/81
HungarianMethod(7/7)
Step6:Nowwehaveincreased
thenumberofzeros
Step5:Selectthesmallestelementfromthe
uncoveredelements
(a)Subtractthiselementfromalluncoveredones
RepeatStep3.
WiehreHerdernMerzhausen Zähringen Stühlinger
Gabi 2
9
0
8
8
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick 3
7
0
11
5
Robert 3
0
2
1
1
(b)Addthiselementtoallelementswhichareattheintersectionof
twolines
WiehreHerdernMerzhausen Zähringen Stühlinger
Gabi 0
7
0
6
6
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick 1
5
0
9
3
Robert 3
0
2
1
1
23/81
WiehreHerdernMerzhausenZähringen Stühlinger
Gabi 0
7
0
6
6
Malte 2
1
6
0
5
Tom 0
4
3
0
0
Patrick1
5
0
9
3
Robert 3
0
2
1
1
24/81
PracticalconsiderationsabouttheHungarianMethod
Negativecostscanbeconvertedby
addingtoeachelementofCthevaluev=
-min{cij}
Theobjectivefunctionistobe
Multi-RobotExploration
Arewereallyinterestedinminimizing
thesumofindividualcosts?
Multiplyallcijby-1or...
replaceeachcijbycmax-cijwherecmax=max{cij}
Therearemoreagents(n>m)ormore
tasks(m>n)
addasufficientnumberof‘‘dummy’’tasksoragents
(whicheverisinshortersupply)withcostsof0(orinf)
25/81
26/81
TheLinearBottleneckAssignmentProblem(1/2)
SerialAssignmentProblem
TheLinearSumAssignmentProblem
minimizesthesummedtotalcostasif
tasksareexecutedinseries
Whenwehaveparallelmachines(MAS)we
wanttominimizethetotalcompletiontime
andthusthemaximaldurationofallparallel
executedjobs!
Wewanttominimizetheso-called
bottleneckobjectivefunction:
max{1≤i≤n}(cᵢφ(i))
28/81
TheLinearBottleneckAssignmentProblem(2/2)
Canbesolvedin
polynomialtime,
usingthe
thresholding
algorithm.
Whatabout
dependency
between
tasks?
MultilevelGeneralizedAssignmentProblems(1/2)
Appearsinthecontextoflarge-scaletask
allocation,e.g.,RoboCupRescueSimulation!
Theproblemofassigningnagentstomtasks
atllevels(i.e.timeslots)
Theresourcerequirement(consumption)aijk
dependentsontheagent,task,andlevel
Asingleagentimayhavemorethanonetask
assignedbutthesumofneededresourcesaijk
maynotexceedbi
Therearealsocostscijkoftheassignment
29/81
MultilevelGeneralizedAssignmentProblems(2/2)
ThisisNP-hard,anddeterminingwhethera
feasiblesolutionexistsisNP-complete.
AllMultilevelversionsofLinearSumandLinear
BottleneckAssignmentProblemsare
31/81
intractable!
30/81
CaseStudy:Task-SequenceAllocationwithHumanintheLoop(1/3)
Coordinationofresponseteamsby
resourceallocation
Humanassistedtask-sequence
assignment
MixedIntegerLinearProgramming
(MILP)solverbiasedbyhuman
expertise
TwoAdvantages:Fasterassignment
computationwithoptimalsequencing
32/81
CaseStudy:Task-SequenceAllocationwithHumanintheLoop(2/3)
CaseStudy:Task-SequenceAllocationwithHumanintheLoop(3/3)
33/81
34/81
ConstraintOptimizationProblems
ContraintSatisfactionsProblems
Insearchproblems,thestatedoesnothavea
structure(everythingisinthedata
structure).InCSPs,statesareexplicitly
representedasvariableassignments.
ACSPconsists
asetofvariables{x1,x2,…,xn}to
values{d1,d2,…,dk}canbeassigned
suchthatasetofconstraintsoverthevariablesis
respected
ACSPissolvedbyavariableassignment
thatsatisfiesallgivenconstraints.
36/81
Applications
Timetabling(classes,rooms,times)
Configuration(hardware,cars,…)
Spreadsheets
Scheduling
Floorplanning
Frequencyassignments
…
CentralizedContraintSatisfactionsProblems
37/81
ExampleMapColoring:AustralianStatesandTerritories
Coloreachoftheterritories/
statesred,greenorbluewithno
neighboringregionhavingthe
samecolor
ConstraintGraph:nodesare
variablesarcsareconstraints
39/81
ExampleMapColoring
Variables:WA,NT,SA,Q,NSW,V,T
Values:{red,green,blue}
Constraints:adjacentregionsmusthavedifferentcolors,e.g.,NSW≠
V
40/81
OnePossibleSolution
Solution
{WA=red,NT=green,Q=red,NSW=green,V=red,SA=
blue,T=green}
41/81
Algorithm
BacktrackingSearchoverAssignments
Assignvaluestovariablesstepbystep
(orderdoesnotmatter)
Consideronlyonevariablepersearch
node!
DepthFirstSearch(DFS)withsinglevariableassignmentsiscalled
backtrackingsearch
Cansolven-queensforn≈25
42/81
Backtracking-ExampleMapColoring(1/4)
43/81
44/81
Backtracking-ExampleMapColoring(2/4)
45/81
Backtracking-ExampleMapColoring(4/4)
Backtracking-ExampleMapColoring(3/4)
46/81
ImprovingEfficiency:CSPHeuristics&PruningTechniques
Variableordering:Whichoneto
assignfirst?
Valueordering:Whichvaluetotry
first?
Trytodetectfailuresearly
Trytoexploitproblem
Note:allthisisnotproblem-specific!
47/81
48/81
ConstraintOptimizationProblem
LinearAssignmentisaConstraintOptimizationProblem
ConstraintSatisfactionProblem(CSP)
Objective:findanassignmentforallthevariables
inthenetworkthatsatisfiesallconstraints
ConstraintOptimizationProblem(COP)
Objective:findanassignmentforallthevariables
inthenetworkthatsatisfiesallconstraintsand
optimizesaglobalfunction
Globalfunction=aggregation(typicallysum)of
localfunctions.F(x)=∑iFi(xi)
49/81
50/81
Variations
Binary,ternary,orevenhigherarity
Finitedomains(dvalues)=>dn
possiblevariableassignments
Infinitedomains(reals,integers)
linearconstraints
nonlinearconstraints
51/81
DistributedConstraintOptimizationProblems
DistributedConstraintOptimizationProblem(DCOP)
Ageneralmodelfordistributedproblemsolving
DCOPsarecomposedofagents,eachholding
oneormorevariables
Constraintsamongvariables(possiblyheldby
differentagents)assigncoststocombinationsof
valueassignments
Agentsassignvaluestotheirvariablesandtry
togenerateasolutionthatisgloballyoptimal
withrespecttothesumofthecostsofthe
constraints
53/81
WhyDistributedsolvers?
Costofformalization:
Incentralizedsolving,allimaginableoptionshavetobe
formulatedbeforehand
Indistributedconstraintsatisfaction,onlyaminimalnumber
ofconstraintsareconsideredbyeachagent
Privacy
Incentralizedscenariosthesolverseesallmeetingsand
Distributedsolutionscanbeconstructedinsuchawaythat
agentsonlyrevealinformationpiecemealwhenevaluating
constraints
Robustness:
Acentralizedsolvercreatesacentralpointof
Distributedsolvingallowsloadbalancingand
55/81
DistributedConstraintOptimizationProblem(DCOP)
Examples
TargetAllocation(AgentstoTargets)
Meetingscheduling
Difficulties
Noglobalcontrol/knowledge
Localizedcommunication
Limitedtime
54/81
ConstraintNetworks
Aconstraintnetworkisformallydefinedbya
tuple<X,D,C>where:
X={X1,X2,...,Xn}isasetofdiscretevariables
D={D1,D2,...,Dn}isasetofvariabledomains,
whichenumerateallpossiblevaluesofthe
correspondingvariables
D={C1,C2,...,Cn}isasetofconstraints,where
eachconstraintCiisdefinedonsubsetSi⊂X
andassociatedwithaarityr=|Si|(hardorsoft)
56/81
Example:MeetingScheduling
57/81
DCOPformalizationfortheMeetingSchedulingProblem
Example:MeetingScheduling
58/81
DCOPExample:MeetingScheduling
Asetofagentsrepresenting
Asetofvariablesrepresentingmeeting
startingtimesaccordingtoaparticipant.
Hard
Startingmeetingtimesacrossdifferentagentsareequal
Meetingsforthesameagentarenon-overlapping.
Soft
Representagentpreferencesonmeetingstartingtimes.
Objective:findavalidscheduleforthe
meetingwhilemaximizingthesumof
individuals’preferences
59/81
60/81
Example:TargetTracking(1/3)
Asetofsensorstrackingasetoftargetsinordertoprovideanaccurate
estimateoftheirpositions.
Example:TargetTracking(2/3)
Sensorscanhavedifferentsensingmodalitiesthatimpactontheaccuracyof
theestimationofthetargets’positions.
61/81
Example:TargetTracking(3/3)
62/81
DCOPFormalizationfortheTargetTrackingProblem
Agentsrepresentsensors
Variablesencodethedifferentsensing
modalitiesofeachsensor
(positions...)
Constraints
relatetoaspecifictarget
representhowsensormodalitiesimpactsonthetracking
performance
Objective:
Collaborationamongsensorsiscrucialtoimprovesystemperformance.
63/81
Maximizecoverageoftheenvironment
Provideaccurateestimationsofpotentiallydangerous
targets
64/81
DCOPAlgorithms
Centralizedvsdecentralized
CompleteAlgorithms
Alwaysfindtheoptimal
Exhibitanexponentiallyincreasingcoordination
Verylimitedscalabilityongeneralproblems
Search-based&Synchronous:SyncBB,AND/ORsearch
Search-based&Asynchronous:ADOPT(Modietal.,2005),NCBB,AFB
DynamicprogrammingbasedDPOP(PetcuandFaltings,
LocalGreedyApproximativeAlgorithms
Sacrifyoptimalityinfavorofcomputationaland
communicationefficiency
Activationprobabilitydomaindependentandneedstobe
Well-suitedforlargescaledistributedapplications,e.g.sensornetworks
androbotics
DSA-A(Zhangetal.,2005),MGM-1(Maheswaranetal.,2004)
Usedecentralizedwhen:
Privacyisa
Thecostofsharinginformationishigher
thanexchangingconstraintmessages
Otherwiseusecentralized,butmake
sureyourcentralnodecanbe
replaced
65/81
66/81
TaskSpecificationTrees
Task-SpecificationTrees
ATaskSpecificationTree(TST)
isadistributeddatastructure
withadeclarative
representationthatdescribesa
complexmulti-agenttask.
AnodeinaTSTcorresponds
toatask.Ithasanode
interfacewithparametersand
asetofnodeconstraintsthat
restricttheparameters.
68/81
Example
TaskSpecificationTreesStructure
Therearecurrentlysixtypes
ofnodes:Sequence,
concurrent,loop,select,
goal,andelementaryaction.
ATSTisassociatedwitha
setoftreeconstraints
expressingconstraints
betweentasksinthetree.
69/81
Example
70/81
DelegationwithTaskSpecificationTrees
ConstraintSatisfaction
TST
ToallowtasksspecifiedasTSTstobedelegated,
theTSTsemanticsisdefinedintermsoftheCan
predicate.
Can(B,τ,[ts,te,…],cons)assertsthatanagent
Bhasthecapabilitiesandresourcesfor
achievingataskτintheinterval[ts,te]with
theconstraintscons.
Ataskτ=(α,φ)consistsofanactionα,which
canbecompositeorelementary,andagoalφ,
whichmaybeempty.
71/81
72/81
TSTSemanticsExample
Scan1
TaskAllocation,asummary
Scan2
73/81
TaskAllocation
LinearAssignmentProblem:usedfor
problemwheretasksaresimpleandcan
beexecutedinanyorderandagentsare
homogenous
Otherwise,useaConstraint
SatisfactionandOptimizationSolver
Godecentralized,ifsharinginformationis
aproblem
TaskSpecificationTrees:arepresentation
ofacomplexsetoftasksanddelegation
IndividualandGroupAssignment
75/81
Project
Tasks
Agroupof8to9
ImplementaRoboRescue
Workindividuallyonasubpartofthe
problem
77/81
Reports
Foundationtasks
Navigation
Communication
Agents:police,ambulance,fire
Exploration
brigad
Prediction
Tasks
78/81
WhatisagoodReport
Indiviualplan
Gradebasedonthereportquality,suchasreadability,language,pictures,
structureandlength,and
theleveloftechnicaldetailweightedwiththedifficultyofthechosen
approaches
Thereportsshouldbe5-6pages,butitismoreimportanttomakeitpossible
forthereadertounderstandyourteamthantogettheexactrightnumber
ofpages.
Youcangetfeedback,submityourdraftbeforeDecember14th,andwegeta
seminaronDecember17thfordiscussingthedrafts
Findaround4related
Writeaonepage
Deadline:October,
Individualreport
Implementandevaluatethe
Writeareportdescribingthetechnique,resultsanda
Deadline,draft:December,14th,final:January
Groupreport
Oneper
Adescriptionofthealgorithmsandstrategies
79/81
80/81
Presentations
Groupplan:October15th
Individualassignmentpresentation
onNovember,19thor26th
GrouppresentationonDecember,
15th
81/81