Expanding the Boundaries of Health Informatics Using Artificial Intelligence: Papers from the AAAI 2013 Workshop
Addressing Preemption Costs in Multi-Agent
Resource Allocation for Medical Applications
John A. Doucette
Robin Cohen
David R. Cheriton School of Computer Science
University of Waterloo
200 University Avenue West
Waterloo, Ontario, N2L 3G1
David R. Cheriton School of Computer Science
University of Waterloo
200 University Avenue West
Waterloo, Ontario, N2L 3G1
lize values computed when generating ‘transfer of control’
(TOC) strategies (Scerri, Pynadath, and Tambe 2002) to provide a basis for accurately estimating costs.
Abstract
In this paper we offer an approach for reasoning about
resource allocation and scheduling in multiagent systems that takes into consideration the costs of preempting an agent from its current task. We apply our methodology to the motivating medical application of allocating doctors to patients in hospitals during mass casualty
incidents and demonstrate noticeable improvements in
performance (generating far fewer problem patients)
over competing approaches that do not model the costs
of preemption in sufficient detail. In particular, our approach offers a method for addressing the challenges
of cyclical dependencies in the estimation of preemption costs by localized agents through a combination of
planning techniques.
Background
Medical Scheduling and Resource Allocation
Recently, partially and fully distributed agent based approaches have begun to appear in hospital scheduling. One
of the principal reasons for adopting these approaches is
the inherently distributed nature of the hospital scheduling
problem: each department within a hospital may set its own
schedule independent of the schedules used by other units,
which can create inharmonious global schedules. Decker
and Li applied their Generalized Partial Global Planning algorithm to hospital scheduling to improve throughput and
decrease the duration of patient stays (Decker and Li 1998),
but acknowledged that their approach did not generate optimal schedules (though optimal plans for dynamic environments offer limited improvements). Vermeulen et al. proposed a different approach through the utilization of automated Pareto efficient appointment time swaps between different patient agents (Vermeulen et al. 2007). This system
had the advantages of being incentive compatible for patients, and of being fully distributed, but limited its scheduling changes to voluntary exchanges. This can result in suboptimal scheduling when the incentives of individual patients do not align with the goals of the hospital itself.
Paulussen et al. (Paulussen et al. 2004) address this with a
market based coordination mechanism called MedPAge for
the problem of patient scheduling, in which the agents representing patients attempt to obtain appointments that maximize improvement in their patients’ ‘years of well being’.
This was accomplished through negotiated timeslot swaps,
managed by agents on behalf of their respective patients. An
agent holding a slot would relinquish it if they expected the
adverse effects on their patient to be less than the positive effects of allowing the swap. The mechanism is market based
in the sense that the preempting agent specifies an amount it
would be willing to pay in order to facilitate the swap, and
the present holder of the timeslot will relinquish it only if
the amount exceeds its expected losses in terms of years of
well being. However, the MedPAge system computes only a
Introduction
Recent work applying multiagent systems in the medical domain to assign doctors’ limited time to patients with different degrees of need (Paulussen et al. 2003) has produced
promising results using a market-based coordination mechanism. The MedPAge approach adopted by Paulussen et al.
however falls short in scenarios where preemption of resources is an issue, because of a worst case estimate of resource preemption costs. We envision the scenario of a Mass
Casualty Incident (MCI), wherein a few doctors must be dynamically assigned to many patients, as an application area
where this shortcomming would become especially important.
In this work, we provide a more detailed description of the
costs associated with preemption in a MedPAge-like system.
We show through simulations that the new model can better
balance the requirements of patients during situations where
very large numbers of patients are present at once, as in a
MCI, and so can reduce the frequency of ‘problem patients’
(i.e. those with critical status). This is accomplished utilizing techniques from the multi-agent subfield of adjustable
autonomy (Hexmoor and Castelfranchi 2003), which deals
with coordination and planning for agents that act as codecision makers with human entities. In particular we utic 2013, Association for the Advancement of Artificial
Copyright Intelligence (www.aaai.org). All rights reserved.
14
first-order approximation of the true costs of these preemptive swaps. In this approximation, the present holder of a
resource considers the costs of preempting other timeslots
itself when computing its expected losses. Its losses are simply the value of its current appointment time, less the value
of the next best time it could reasonably expect to obtain.
To determine which timeslots this first agent may reasonably expect to acquire, this agent must discover the costs of
preempting the appointment times of other agents. However,
each of these agents has the same problem: to determine the
costs of losing their current timeslots, they need to know the
costs associated with taking the slots of other agents. The
presence of cycles in these dependencies can make the discovery of true costs computationally expensive.
To avoid this issue, when one agent (the preemptor) asks
another (the preempted) to compute the costs of a preemption, the preempted MedPAge agent assumes all other agents
in the system lack capacity for planning, and that any third
agent (the victim) that it attempts to take the timeslot of will
be left with the worst timeslot of all: the back of the queue.
This worst case estimation of the costs of these victims can
cause overestimation of the preempted agent’s costs. As a
consequence, the preempted agent will require a larger payment from the preemptor than is strictly necessary, which
can prevent needed preemptions from taking place.
The model we present in this paper attempts to address
this issue using a combination planning and user modeling
techniques. Using these techniques, our agents can more accurately estimate the costs of preempted agents, and thus
allow more preemptions to take place.
tems like the Electronic Elves project (Scerri, Pynadath, and
Tambe 2002). A TOC strategy is a plan specifying who to
engage when an agent needs to obtain a decision. For example, an agent might be asked to schedule a meeting time
between many different humans. The agent may have some
idea about the optimal time, but knows that there is some uncertainty in its guess. To minimize the probability of a mistake, the agent can transfer control of decision making to an
alternative entity (i.e. a human or an agent) which it believes
could make a more informed decision. If the entity declines
to make the decision, or if circumstances change before the
entity decides, the agent can transfer control to other entities or make the decision itself (Scerri, Pynadath, and Tambe
2002). A sample TOC strategy might look like {e1 t1 e2 t2 A},
representing a transfer of control to entity 1 initially, then to
entity 2 at time 1 if e1 has declined to respond, and then
having the agent (A) make the decision itself if necessary at
time 2. The agent can plan out the optimal sequence of TOC
operations by estimating the quality of decisions each entity
could make, and by incorporating information about their
availability for decision making (e.g. a human’s calendar and
sleep habits; an agent’s available computational resources).
(Cheng and Cohen 2005) extended TOC strategies to incorporate partial transfers of control (PTOCs) which first ask
entities questions, before deciding to make full transfer of
control. Vitally, the process of generating transfer of control
strategies is one of optimization, typically conducted using
branch-and-bound search heuristics, and the value used in
the optimization procedure is equal to the expected utility
of executing the strategy. That is, optimization proceeds by
maximizing the integral of the expected quality of the decision made by the entity currently in control of decision making multiplied by the probability that the decision is actually
made at the current time:
∞
PT (t) ∗ EUedcurrent (t) dt
EU =
Mass Casualty Incidents
A mass casualty incident (MCI) occurs when a large number of patients simultaneously and unexpectedly suffer (potentially) critical injuries in the same time and place (Mistovich 2000). They are distinct from large scale emergencies such as floods insofar as such events are typically anticipated by communities in which they occur, and plans for
mitigating their effects already exist. In regions where MCI
are common, first responders rapidly reach the site of an attack and transport patients to a nearby hospital (Hirshberg
2004). Crucially, patients are overwhelmingly likely to be
transported to the nearest hospital, rather than the hospital
best equipped to care for a large number of serious patients
(the associated odds ratio has been measured at nearly 250)
(Hirshberg 2004). Consequently, MCI represent scenarios
where a large number of critical patients may unexpectedly
arrive at an ill-equipped hospital simultaneously, and so may
pose problems for existing scheduling systems.
0
For example, if the expected utility of a decision made by
entity 1 in the TOC strategy above is 5, and the probability
of entity 1 making the decision before time 1 is 0.4, then
the expected utility of starting a TOC strategy with e1 t1 is
0.4 ∗ 5 = 2. PTOCs are handled by applying a weight to the
expected value of the TOC operations that would be taken
given each possible response, equal to the probability estimate for that response being produced. The value of EU for
a TOC strategy represents the expected value of executing
the strategy. Later, we use this fact to provide an estimate of
the costs of preemption.
Transfer of Control Strategies
Cyclic Dependencies in Preemption Cost
Estimation
Although many planning techniques could be used with our
proposed system, we elect to use Transfer of Control (TOC)
strategies. In the health domain, resources may often take the
form of doctors or other humans. TOC strategies are a natural representation of this domain, as they were developed to
model systems involving human interaction.
TOC strategies were created as a planning technique for
use in mixed-initiative adjustable autonomy multi-agent sys-
The principal contribution of this work is a novel method
for addressing the appearance of cyclic dependencies in preemption cost estimation. Cyclic dependences arise when the
cost of an attempt by one agent (A) to preempt the resource
or timeslot of another agent (B) is coupled to the cost of B
preempting the timeslot of a third agent (C). If the cost of B
preempting C is also dependent on the cost of C preempting
A, then computation of the costs cannot be accomplished
15
directly. In previous work on multi-agent medical scheduling (e.g. (Paulussen et al. 2003)), these issues were avoided
through the implicit use of worst-case bounds as estimations
of the costs of the second preemption (i.e. B preempting C),
but this approach results in an overly cautious use of preemption. In contrast, the system we propose in this paper
has agents relinquish their timeslots only if, given their local information, they believe a reasonable appointment time
could be acquired. The following example illustrates issues
with the existing approach which can prevent valid, important preemption from taking place.
Suppose for example, three agents (Alice, Bob, and Charlie) all need a medical procedure, and the present schedule
for the procedure is {Bob, Charlie, Alice}. Yet suppose:
Alice must be treated first, or she will suffer life long debilitating illness; Bob needs to be treated either first or second
to avoid the same fate, but the order does not make a difference; Charlie can be treated at any of the first three time
steps, but suffers a high chance of unpleasant, but transient,
illness if he is treated in the third step. The optimal schedule
is thus {Alice, Bob, Charlie}. Given the present schedule,
Bob and Charlie are both content with their arrangement,
and do not have any incentive to change it. 1: Alice needs to
be treated first, so her agent opens negotiations to preempt
Bob’s appointment. 2: A swap of Alice and Bob’s slots will
only result in a positive net utility if Bob can preempt Charlie’s slot, so Bob’s agent opens negotiations with Charlie’s
agent. 3: A swap of Bob and Charlie’s slots will only produce a positive net utility if Charlie can preempt Alice’s slot.
4: If we do not allow cyclic dependencies, and use a worst
case heuristic (following (Paulussen et al. 2003)), Charlie’s
agent reports the cost for placing Charlie in position 4 in the
schedule. This would result in no net gain, so the preemption is rejected. Using a worst case heuristic for estimating
the costs of preemption prevents convergence to the optimal
schedule. Consequently, a more precise estimate is required
to obtain optimal performance.
disengage from the system. Similarly, if the professional
is involved in a sensitive task (e.g. surgery), they may not
acknowledge interruptions by the system. Following (Cohen, Cheng, and Fleming 2005), for each user, we adopt a
user-specific exponentially decaying model of bother, where
requests for direct preemption contribute to a human resource’s bother, and the contribution is proportionate to
the importance and relevance of the request. Relevance is
estimated using a U ser U nwillingness F actor (U U F )
and an Attentional State F actor (ASF ), problem specific values which are based respectively on the user’s ability or willingness to do the task in question and the inverse
of the user’s attention level. For example, a doctor who is
presently doing routine work in a clinic would probably have
very low U U F if asked to save an emergency patient’s life
instead. Once in surgery, the doctor would have a very high
ASF from the system’s prospective because she is unlikely
to respond to requests while engaged in that task.
The base cost of initiating a new TOC operation in
this model is given by Init = U U F × ASF ×
T OC Base Bother Cost, where the last term refers to the
inate difficulty of the task. Typically this is low when querying a user for their preferences and high when asking a user
to make a complex decision, but the precise values are problem dependent. The ultimate bother cost of a TOC operation
is given by BC = Init + BC Inc F n(BSF, U U F ). The
Bother Increase Function BC Inc F n describes the rate at
which bother increases as a function of the user’s present
perception of how often the system is bothering them, and
how frivilious the task appears to the user. Typically this is a
function of the form BSF U U F , where BSF is an exponentially weighted moving average of the bother from previous
interactions.
To support a fully distributed system, we utilize the type
IV multiagent coordination algorithm from (Cohen, Cheng,
and Fleming 2005). Agents build an estimate of the bother
costs associated with a request by querying a resource’s
proxy agent to obtain the values of U U F and ASF at the
current time. When the time comes to make the TOC request, the querying agent verifies with the resource’s proxy
agent that the estimate is within some small range of the true
value. If it is not, the agent adjusts it TOC strategy as necessary, and may try again.
Finally, patients within the system are modeled by an initial health state H(0), and an initial condition C = dH
dt . At
time t from the patient’s appearance at the hospital, the patient will have health state
t
Cs ds
H(t) = H(0) +
Model
In our scenarios,the resources we have in mind are primarily
medical professionals (e.g. doctors). Medical resources are
assigned proxy agents, who can estimate the time taken by a
resource to perform tasks (e.g. a doctor treating a patient).
User Modeling
In the case of human medical resources, we also incorporate techniques from user modeling to estimate the probability of response to a direct preemption. In a direct preemption, an agent requests a resource from the person
who is currently using it, rather than acquiring an appointment for some future work. Direct preemptions facilitate
the acquisition of medical resources presently in routine
use for treatment of an emergency patient. For example,
a doctor making the rounds might be called from his current patient into surgery. If a system makes many direct
preemption requests, especially requests which do not account for the severity of the situation currently being addressed by the medical professional, the professional may
0
where Cs is the condition of the patient at time s (since conditions can change over time). Note that C = dH
dt may contain non-deterministic components if desired, though we do
not make use of this possibility here. A patient whose health
state reaches 0 is deemed to be cured, while a patient whose
state reaches some problem-specific maximum (Hmax ) requires immediate treatment to avoid long term health repercussions, and is referred to as a ‘problem patient’. Problem
patients consume a disproportionately large amount of hos-
16
pital resources, and should consequently be avoided.
TOC strategy with respect to the associated resource. When
another agent attempts to preempt an agent’s presently held
appointment time, or inquires about the present value of the
appointment to the agent for use in construction of a plan
of its own, the agent reports the expected utility of keeping
its current timeslot, less the expected gain from executing its
plan. The former term is the negation of the expected change
in the patient’s health state H(t) if they keep their current resource, perhaps in years of life lost, over a fixed time scale.
The latter is simply the expected utility of the agent’s optimal TOC strategy: the maximization of the integral in the
Transfer of Control Strategies section above, with respect to
the associated patient’s utility, again in terms of H(t). The
plan may also be periodically regenerated to avoid having it
become out of date.
Our TOC strategies support three operations. A full transfer of control (FTOC) is used to initiate negotiation with another agent over a timeslot. The expected value of this FTOC
is based on the probability that the preemption will succeed,
computed using G(told , tnew ), the preemption cost estimate
reported by the other agent at the time of strategy generation, and a gradual decay in value based on the length of
time into the future that the FTOC would take place. A partial transfer of control, or PTOC, can be used to ask other
agents or hospital staff for information regarding the status
of other patients, and take actions dependent on their reply.
Finally, a strategy generation node, or SG Node, is used to
initiate generation of a new TOC strategy to replace the existing one. This might take place if new information comes
to light which could dramatically improve the agent’s planning, or if the agent makes a TOC request which the bother
model determines to be using out of date information.
TOC strategy generation can be accomplished using
branch-and-bound search, or using a dynamic programming
approach. The latter produces solutions which are certain to
be optimal, while the former has superior run times for many
problems(Scerri, Pynadath, and Tambe 2002). In this work,
we use the dynamic programming approach to remove this
possible source of error from our preemption cost estimates.
Using TOC strategies constitutes pre-computing contingency plans and allows agents to respond to requests for preemption without querying other agents at the time of the request. This resolves the cyclic dependency problem by providing a reasoned estimate of the true preemption cost without allowing cycles to form.
Negotiation
The core of our new framework lies in its negotiation protocol. We use a protocol similar to MedPAge (Paulussen et al.
2003), but with several notable differences. First, our protocol places no restrictions on the number of agents who can
act at once. That is, there is no concurrency requirement in
our protocol. Second, our protocol allows agents to interact
with human actors directly, as well as with other agents. This
is useful insofar as the system can be proactive in gathering
information, or attempting direct preemptions (which might
be required in MCI). The underlying bother model mentioned above helps ensure that the system makes only reasonable requests of human users. Third, our system allows
for probabilistic response times as a natural consequence of
supporting interaction with humans. Instead of receiving an
immediate response, agents in our system expect some delay (following a known distribution) which might be caused
by network latency, damage to the hospital, or prolonged
reasoning (by humans or machines). Finally, our system assumes fully cooperative agents, rather than the partially cooperative ones used in earlier work (Paulussen et al. 2003).
The basic negotiation process is presented below as part
of the behavior of a patient agent throughout:
1. Let A be an agent representing a patient with health state
HA , and condition C.
2. Let P be a plan of action for A which specifies an optimal
(or near optimal) order for A to request medical appointments (see Preemption Cost Estimation below).
3. Let p be the appointment time suggested for acquisition at
the current time by P , and told be the present appointment
time for A with the specified resource.
4. If p is unowned, acquire it.
5. Otherwise, if p is owned by agent B, compare the gain
made by A preempting (G(told , p)) (as calculated below)
with the TOC-based preemption cost estimate produced
by B.
6. If the gain exceeds the cost, A takes p, and B accepts the
closest open appointment time as a temporary location,
and now searches for a new appointment time based on
its own plan of action.
7. If p is the last step in P , generate a new plan.
8. Otherwise, unless A’s patient has recovered, remove p
from P , and GOTO step 3.
Experiment
To validate the new model, we implemented a version of it
for a MCI scenario, and compared it with two ablated models using worst-case estimates of preemption costs, following (Paulussen et al. 2004). The first ablated model additionally prohibits direct preemption (as in MedPAge) while
the second relaxes this constraint and constructs a MedPAge
variation that is more fairly compared to our proposed algorithm. The comparison systems were otherwise identical
to our proposed framework, including the same bother cost
model, and the same modified negotiation procedure.
Our simplified scenario featured four abstracted conditions, Routine, Minor, Major and Emergency. These con-
Preemption Cost Estimation
Our framework ties the costs of preemption to the generation of the plans of action mentioned above. In essence, a
plan of action is a set of TOC strategies, each of which specifies the sequence of TOC operations which is expected to
lead to the optimal decision for the appointment time of the
agent’s patient. At each time step, an agent processes any
responses received since the last time step, and then executes any TOC operations required by its plan. Whenever
an agent acquires an appointment time, it computes a new
17
80
●
●
20
40
%PP / %PS
60
action. We assume that the hospital had sufficient capacity
to hold all patients who arrived there.
Three parameters were varied in our simulations. First, the
number of medical resources was varied between 5, 10 and
15. Again, small numbers were used to facilitate efficient
computation on the limited computational resources. Second, the number of patients produced by the MCI was varied
between 25, 50, 100, 200 and 400. Finally, the distribution
of initial patient health states was varied between U (0, 50),
U (0, 100), and U (50, 100), where U (x, y) is a uniform distribution over (x, y). We set Hmax to 200 for consistency
with the bother model. In the first experiment, we utilized
a single parameterization with 400 patients, 15 doctors and
low initial patient health states. In the second experiment,
we conducted runs on every combination of values for the
three parameters. In each case, we ran both the new model
and the ablated comparison model on the same 100 random
seeds, resulting 100 unique initial conditions. A patient who
reached a health state of Hmax was ruled a problem patient,
and removed from the simulation in much the same way as
a cured patient to insure the eventual end of the simulation.
A single parameterization took approximately 30 minutes to
simulate from start to finish on a single 1.8GHz CPU core.
●
Full
Ablated
Paired
Figure 1: Violin plots (Hintze and Nelson 1998) depicting
the distribution of problem patients (%PP) for the new system (left) and an ablated version incapable of preemption
(center). Also shown is the distribution of the pairwise improvement produced by using the new system on each scenario, in terms of patient’s saved (%PS) by using the new
system (right). Violin plots depict a box plot with a rotated
kernel density plot (smoothed histogram) on either side.
Results
Our results are summarized in figures 1 and 2. The former depicts the mean difference between the performance
of our method and the very simple ablated model in terms
of the percentage of patients who became ‘problem patients’
(%PP) during the course of the simulation, or the percentage
of patients saved (from becoming problem patients) by using
the new system (%PS). The latter shows the same measurements for the more complex ablated model. Tests of significance used parametric or non-parametric measures depending on whether the data were consistent with a normal distribution, and Bonferroni corrections were used where applicable. R produced the results (Team 2009).
The results of comparing our model with an ablated version incapable of direct preemption are shown in figure 1.
The first two plots show the overall distributions of %PP for
each method over 100 runs, while the third shows the paired
difference for each scenario in terms of %PS. The paired difference is approximately normally distributed with a mean
%PS of 20.79% in favour of the new method and 95% confidence interval (19.55, 22.05).
The results of comparing our model with an ablated version capable of direct preemption appear in figure 2. Confidence intervals were omitted for clarity, but were never
larger than ±1%. We a used four factor ANOVA to compare the %PP of the two models, with the three simulation
parameters and the preemption model as predictors. All predictors were highly significant (p < 1.0e − 16), with a mean
difference in favour of the new model of roughly 4%PS averaged across all parameter settings. Nearly all interaction
terms were significant as well. In particular, we found that
larger numbers of medical resources and patients tended to
produce larger gains for the new system, up to some saturation point, while low initial patient health states produced
gains as well.
ditions produced a value for dH
dt of 0.04H, 0.08H, 0.12H
and 0.16H respectively. Thus, for example, a patient with a
Routine condition and a criticality of 50 would have a criticality of 50 ∗ 0.04 + 50 = 52 if left untreated for a single
time step. Medical resources were generated with uniformly
random efficiencies for treating each condition. Efficiencies
were drawn uniformly from the range [0 − 20], and functioned as a negative modifier to dH
dt during treatment. For
example, a doctor efficiency for 20 for the Emergency condition, treating a corresponding patient produces a change in
health state of dH
dt = 0.16 ∗ H − 20
We limited TOC strategies to a fixed depth of 5 time
steps ahead. Since patients had only one condition, a plan
of action consisted of just a single TOC strategy. This was
done to limit the computational cost and number of experiment parameters. We utilized the default values for the
bother cost model suggested by (Cohen, Cheng, and Fleming 2005) for simplicity. Attentional State F actor and
U ser U nwillingness F actor were generated as a function of health state of a medical resource’s current patient
(if any) and the expected utility gain of the preemption (G).
Our MCI scenario starts with a small number of patients at
the hospital (drawn with uniform probability from the range
(0 − 15), all with routine conditions. The MCI is simulated
by generating a large number of patients (the exact number
is a simulation parameter, see below). Each patient is assigned one of the four conditions with uniform probability,
and an initial health state drawn from a simulation specific
distribution. Patients are then assigned an arrival time at the
hospital, drawn with uniform probability from the range of
(0 − 20) time steps. At the time of arrival, each patient is
assigned a new patient agent, which generated a new plan of
18
40
●
●
●
5 Resources
10 Resources
15 Resources
20
40
●
5 Resources
10 Resources
15 Resources
20
●
20
40
●
Very High Initial Health State
High Initial Health State
Medium Initial Health State
5 Resources
10 Resources
15 Resources
●
●
●
200
300
400
●
●
●
−20
−40
−40
−40
100
●
0
●
%PS
0
%PS
●
−20
0
●
−20
%PS
●
●
100
# Patients
200
# Patients
300
400
100
200
300
400
# Patients
Figure 2: Mean improvements in %PS over the three parameters. Points above 0 show the advantage of using the new algorithm.
Discussion
namically. We also hope to apply the system directly to realworld MCI data.
We have proposed a method for multi-agent resource allocation using TOC strategies to provide more detailed estimates
of preemption costs, illustrated for the scenario of medical
scheduling. Our new system relies on pre-computed contingency plans in the form of TOC strategies to provide estimates of the utility costs of a preemption. Using TOC strategies also provides us with the ability to interact directly with
human entities (inside a hospital, nurses and doctors), to
gather additional information or to make ‘direct’ preemption
requests (moving a doctor from a routine case to an emergency mid-treatment). A detailed bother cost model prevents
the system from harassing human users with frivolous requests. We verified our proposed framework in simulated
mass casualty incidents, and showed that it performed better
than using worst-case bounds on preemption cost, when: (i)
There were many resources; (ii) There were many users; (iii)
Most users could be serviced given the time allowed and the
resources available. Conversely, although the ablated system
most similar to existing work did not perform well, a variant
which supported direct preemption performed well when:
(i) There were very few resources; (ii) There were very few
users; (iii) Most users could not be serviced, even with an
optimal schedule. We suspect that these highly congested
domains cause the agents in our system to overplan, which
leads to incorrect estimates of preemption costs. Though this
issue will need to be addressed in future versions of the system, we conclude that the new system could be well suited
for use in hospitals during MCI, and in other similar problem domains which require direct preemption and have both
many users and many resources, possibly since these scenarios are more likely to contain cyclic dependencies.
In future, we plan to repeat our experiments with the incorporation of learning algorithms. We expect to address the
fact that agent plans may be out of date by integrating the
learning of both congestion (ratio of supply and demand for
resources) and churn (rate of change of assignments in the
system), with past experiences providing the starting point.
This should provide more robust performance across all scenarios by adapting the plan length and utility estimates dy-
References
Cheng, M. Y. K., and Cohen, R. 2005. A hybrid transfer
of control model for adjustable autonomy multiagent systems. In Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, AAMAS ’05, 1149–1150. New York, NY, USA: ACM.
Cohen, R.; Cheng, M. Y. K.; and Fleming, M. W. 2005. Why
bother about bother: Is it worth it to ask the user? In AAAI05
Symposium on Mixed-Initiative Problem-Solving Assistants.
Decker, K., and Li, J. 1998. Coordinated hospital patient
scheduling. In Proceedings of ICMAS’98, 104 –111.
Hexmoor, H., and Castelfranchi, C. 2003. Agent Autonomy.
Kluwer Academic Publishers Group.
Hintze, J. L., and Nelson, R. D. 1998. Violin plots: a box
plot-density trace synergism. American Statistician 52.
Hirshberg, A. 2004. Multiple casualty incidents lessons
from the front line. Annals of Surgery 239(3):322–324.
Mistovich, J. J. e. a. 2000. Brady Prehospital Emergency
Care Sixth Edition. Prentice Hall Health.
Paulussen, T. O.; Jennings, N. R.; Decker, K. S.; and Heinzl,
A. 2003. Distributed patient scheduling in hospitals. In
Proceedings of IJCAI’03, 1224–1232.
Paulussen, T. O.; Zller, A.; Heinzl, A.; Pokahr, A.; Braubach,
L.; and Lamersdorf, W. 2004. Dynamic patient scheduling
in hospitals. Agent Technology in Business Applications.
Scerri, P.; Pynadath, D. V.; and Tambe, M. 2002. Why the
elf acted autonomously: Towards a theory of adjustable autonomy. In Proceedings of AAMAS’02.
Team, R. D. C. 2009. R: A Language and Environment for
Statistical Computing. R Foundation for Statistical Comp.
Vermeulen, I.; Bohte, S.; Somefun, K.; and Poutr, H. 2007.
Multi-agent pareto appointment exchanging in hospital patient scheduling. Service Oriented Computing and Applications 1:185–196.
19