MULTI-AGENT BASED
SCHEDULING
D. Ouelhadj
ASAP (Automated Scheduling Optimisation and Planning) Research Group
School of Computer Science and IT
University of Nottingham, UK
Open Issues in Grid Scheduling, 2003
Contents
1. Introduction
2. Multi-agent systems
3. Multi-agent based scheduling
4. Multi-agent systems for integrated and
dynamic scheduling of steel production
5. Conclusion
Open Issues in Grid Scheduling, 2003
Introduction
Characteristics of most scheduling systems developed in
manufacturing environments:
 Centralised or hierarchical.
 Tractable.
Supervisor
level
Supervisor
level
 Stochastic.
Intermediate
levels
Resource
level
Resource
level
Centralised and hierarchical scheduling
Open Issues in Grid Scheduling, 2003
Introduction
Classical scheduling techniques:
 Operational research-based techniques: branch and bound, etc.
 Artificial intelligence-based techniques: heuristics, metaheuristics, hyper-heuristics, knowledge-based systems, case-based
reasoning, fuzzy logic, etc.
Distributed Scheduling systems
using MULTI-AGENTS
Open Issues in Grid Scheduling, 2003
Motivations
 Real-life scheduling problems are usually physically or functionally
distributed (air traffic control, manufacturing systems, health care, etc.).
 Complex systems are beyond direct control. They operate through the
cooperation of many interacting subsystems, which may have their
independent interest, and modes of operation.
 Complexity of real-life scheduling problems dictates a local point of view.
When the problems are too extensive to be analysed as a whole, solutions
based on local approaches are more efficient.
 Centralised structures are difficult to maintain and reconfigure, inflexible,
inefficient to satisfy real-world needs, costly in the presence of failures, and
the amount of knowledge to manage is very large.
Open Issues in Grid Scheduling, 2003
Motivations
 Need for integration of multiple legacy systems and expertise.
 Heterogeneity. Heterogeneous environments may use different data and
models, and operate in different modes.
 Robustness and reliability against failures.
 Scalability and flexibility.
 Computational efficiency. Agents can operate asynchronously and in parallel,
which can result in increased overall speed.
 Clarity of design and reusability.
 Costs. It may be much more cost-effective than a centralised system, since it
could be composed of simple subsystems of low unit cost.
Open Issues in Grid Scheduling, 2003
What is a multi-agent system
An agent is an intelligent entity that
is situated in some environment,
and that is capable of flexible and
autonomous action in this
environment in order to meet its
design objectives. By flexible we
mean that the system must be
responsive, proactive, and social
Wooldrige and Jennings (1995).
 autonomy
 goal-driven
 reactivity and
proactivity
 social ability
 persistent
A Multi-Agent System is a system composed
of a population of autonomous agents,
which interact with each other to reach
common objectives, while simultaneously
each agent pursues individual objectives
Ferber (1997).
 mobility
Agent
 adaptability
communication
perception
action
Open Issues in Grid Scheduling, 2003
environment
Cooperation in multi-agent systems
Contract Net Protocol
Task
agent
Task
announcement
announcement
agent
Bid
The contract net protocol is a
high level protocol for achieving
efficient cooperation introduced
by Smith (1980) based on a
market-like protocol.
Contract
Open Issues in Grid Scheduling, 2003
Multi-agent-based scheduling
 Local autonomy. An agent has the responsibility
for carrying out local scheduling for one or more
(functional or physical) components, such as
machines and jobs.
 Agents have the ability to observe their
Announcement
Resource
agent: Local
Scheduling
environment and to communicate and cooperate with
other agents in order to ensure that local scheduling
leads to a globally desirable schedule.
of production
requirements
Local
scheduling
Local
scheduling
 Autonomy allows the agents to respond to local
Resource
agents
variations, increasing the flexibility of the system.
 Concurrency. Negotiation-based decision making
instead of totally pre-planned scheduling.
 Robustness: fast detection of and recovery from
the failures.
I am free in
that period
 Open and dynamic scheduling structures.
Open Issues in Grid Scheduling, 2003
broken
down
Multi-agent-based scheduling architectures
Autonomous.
Mediator.
Open Issues in Grid Scheduling, 2003
Multi-agent-based scheduling architectures
Autonomous architectures
Manufacturing
entities
Agents representing manufacturing
entities (resources, tasks, etc.) have
the ability to define their local
schedules, react locally to local
changes, and cooperate directly with
each other to generate the global
optimal and robust schedules.
Physical or
functional agents
(resources, parts,
tasks, etc)
Open Issues in Grid Scheduling, 2003
Multi-agent-based scheduling architectures
Mediator architectures
A mediator architecture has a basic
structure of autonomous
cooperating local agents that are
capable of negotiation with each
other in order to achieve production
targets.That basic structure is
extended with mediator agents to
coordinate the behaviour of the
local agents to generate the global
optimal and robust schedules.
Mediator agent
Coordinator for global
scheduling
Mediator agent
Physical or
functional agents
(resources, parts,
tasks, etc)
Manufacturing
entities
Open Issues in Grid Scheduling, 2003
A multi-agent system for integrated and
dynamic scheduling of steel production
Open Issues in Grid Scheduling, 2003
Steel production scheduling
Integration: how to integrate the scheduling systems of
the continuous caster and the hot strip mill ?
Dynamic scheduling: Robustness against failures ?
Use Of MULTI-AGENT SYSTEMS
Open Issues in Grid Scheduling, 2003
Multi-agent architecture proposed
`
user
User agent
HSM Agent
SY Agent
coils
Hot Strip Mill
Slabyard
CC-1 Agent CC-2 Agent CC-3 Agent
Continuous
Casters
Ladle
Open Issues in Grid Scheduling, 2003
Slabs
Dynamic scheduling of the HSM and CC agents
Presence of real-time events
On the CC agent: steel with wrong chemical compositions.
On the HSM agent: non availability of slabs.
Robust predictive-reactive scheduling
first constructs a predictive schedule and then modifies the schedule
in response to real-time events so as to minimise deviation between
the performance measure values of the realised and predictive schedule.
Open Issues in Grid Scheduling, 2003
Dynamic scheduling of the HSM and CC agents
Predictive schedules are generated using tabu search.
Robust predictive-reactive schedules are generated
using:
 Utility, stability, and robustness measures.
 Rescheduling strategies: complete rescheduling and
schedule repair.
Open Issues in Grid Scheduling, 2003
Dynamic scheduling of the HSM and CC agents
Utility, stability and robustness measure the effect of real-time
events, and are used to select the best rescheduling strategy
(schedule repair or complete rescheduling) to react to realtime events.
Utility measures the change in the value of the schedule
objective function following the schedule revision.
Stability measures the deviation from the original predictive
schedule caused by schedule revision.
Robustness combines the maximisation of utility and the
minimisation of stability.
Open Issues in Grid Scheduling, 2003
Rescheduling strategies
Schedule repair and complete rescheduling strategies
On the HSM agent
On the CC agent
•Do-nothing (DON)
• Insert- at- end Schedule repair (IESR)
• Simple Replacement (SR)
• Insert-Heat Schedule Repair (IHSR)
• Closed Schedule Repair (CSR)
• Shift Schedule Repair (SHSR)
• Open Schedule Repair (OSR)
• Swap Schedule Repair (SWSR)
• Hybrid Closed Schedule Repair (HCSR)
• Hybrid Schedule Repair (HBSR)
• Hybrid Open Schedule Repair (HOSR)
• Complete Rescheduling (CR)
• Partial Reschedule (PR)
• Complete Rescheduling (CR)
Open Issues in Grid Scheduling, 2003
Negotiation protocol for inter-agent
cooperation
The negotiation protocol is a two-level bidding mechanism based
on the contract net protocol involving negotiation at HSMA-SYA
and SYA-CCA(s) levels.
At the HSMA-SYA negotiation level, the HSMA requests the
supply of slabs from the SYA.
At the SYA-CCA (s) negotiation level, the SYA requests the
production of slabs not available in the slabyard from the CCA(s).
A commitment duration is attached to the the negotiation messages
to specify the time windows by which the agents must respond to a
given negotiation message.
Open Issues in Grid Scheduling, 2003
Negotiation protocol for inter-agent
cooperation
The
negotiation
protocol
incorporates
a
decommitment mechanism to allow the agents to
decommit by specifying appropriate contract’s
alternatives in response to future real-time events.
Open Issues in Grid Scheduling, 2003
Negotiation protocol for inter-agent
cooperation
Steps of the negotiation protocol
Announcing
Bidding
2. SYA-Announcement for
the production of slabs not CC-1
agent
available in the SY.
HSM
agent
3. CCA-bid(s)
1.HSMA-announcement for
the supply of the slabs for
the current turn.
4. SYA-bid
CC-n
agent
6. Forward of the contract, or
renegotiation of the nonsatisfied slabs.
Contracting or
renegotiating
HSM
agent
SY
agent
HSM
agent
SY
agent
CC-1
agent
SY
agent
5. Establishment of a contract, or
renegotiation of the non-satisfied
slabs.
CC-1
agent
CC-n
agent
Open Issues in Grid Scheduling, 2003
CC-n
agent
Prototype developed in simulation
• A prototype has been developed in Microsoft Visual C++/MFC.
• Cooperation between the agents is done with the exchange of
asynchronous messages formatted in XML using MSMQ.
Open Issues in Grid Scheduling, 2003
Prototype developed in simulation
Open Issues in Grid Scheduling, 2003
Evaluation of the performance of the local
and global predictive schedules
Objective function value
800
700
600
500
400
300
200
100
0
)
)
)
)
)
)
)
)
bs
b s labs labs labs labs labs labs
la
la
s
s
s
s
s
s
s
s
3
4
07
17
20
1 7 11 4 17 4
01
(9
(8
(1
(1
(1
(1
(
(
(1
3
0
0
0
0
0
0
0
0
0
14
30
30
00
00 1 00 1 00
1 0 1 00
1/
1/
2/
/1
/1
/
/
/
5/
n
n
n
1
2
3
4
6
r
r
r
rn
n
rn
rn
rn
rn
Tu
Tu
Tu
Tu Tur
Tu
Tu
Tu
Tu
sla
)
bs
Scheduled slabs
Objective function value of the initial local schedule
Objective function value of the global schedule after
negotiation/renegotiation
Open Issues in Grid Scheduling, 2003
Average frequency of schedule repair
and complete rescheduling strategies
0.80
0.60
0.70
0.50
0.60
0.30
0.40
0.30
0.20
On the CC agent
l ue
Va
of
R
On the HSM agent
Open Issues in Grid Scheduling, 2003
1.00
0.95
0.75
0.50
0.25
0.01
gs
t ra
teg
ies
0.00
CSR
lin
SR
Re
sc
he
du
OSR
HOSR
0.00
CR
0.00
PR
0.10
HCSR
R
1.00
0.95
0.75
0.50
0.25
0.01
0.00
of
lue
Va
0.10
NOT
tra
teg
i es
IESR
IHSR
uli
ng
s
SHSR
ed
SWSR
Re
sch
HBSR
CR
0.20
Frequency
0.40
Frequency
0.50
Performance of the utility and stability
measures
170.0
50
45
150.0
40
130.0
35
Stability
25
90.0
Stability
30
110.0
20
15
70.0
10
50.0
5
-130.0 -120.0 -110.0 -100.0 -90.0
-80.0
-70.0
-60.0
-50.0
-40.0
30.0
-30.0
0
-180
-165
-150
-135
IHSR
SHSR
-105
-90
-75
-60
-45
-30
-15
Utility
Utility
PESR
-120
SWSR
HBSR
On the CC agent
CR
NOT
SR
CSR
OSR
HCSR
HOSR
On the HSM agent
Open Issues in Grid Scheduling, 2003
PR
CR
Conclusion
• Dynamic and autonomous distributed scheduling. The dynamic
scheduling problem is distributed across a set of agents.
• Local autonomy allows the agents to respond to local variations
and self-adaptation to real-time events , increasing the robustness
and flexibility of the system.
• The cooperation protocol allows the agents to cooperate and
coordinate their local tasks in order to generate desirable globally
predictive and robust schedules.
• Dynamic task allocation.
Open Issues in Grid Scheduling, 2003
Conclusion
• Natural load-balancing as busy agents do not need to bid.
• Increased Flexibility.
• Robustness against failures.
• Heterogeneity.
• Open and extensible scheduling architectures: Agents can be
introduced and removed dynamically.
• Reduced complexity.
• Reduced costs.
Open Issues in Grid Scheduling, 2003