A research and education initiative at the MIT
Sloan School of Management
A Knowledge Based Methodology for
Designing Robust Electronic Markets
Paper 126
Mark Klein
Chrysanthos Dellarocas
Juan Antonio Rodriguez-Aguilar
August 2001
For more information,
please visit our website at http://ebusiness.mit.edu
or contact the Center directly at ebusiness@mit.edu
or 617-253-7054
A Knowledge-Based Methodology for Designing Robust
Electronic Markets
Mark Klein
Center for Coordination Science
Massachusetts Institute of Technology
m_klein@mit.edu
Chrysanthos Dellarocas
Sloan School of Management
Massachusetts Institute of Technology
dell@mit.edu
Juan Antonio Rodriguez-Aguilar
Center for Coordination Science
Massachusetts Institute of Technology
jarjar@mit.edu
1. The Challenge
The ‘new economy’ of ubiquitous electronic markets entails new risks and demands
novel responses. Such markets, for one, give unprecedented scope to open participation
by software agents, whose unique strengths and weaknesses enable a whole range of
novel failure modes. Software agents can, for example, be replicated almost indefinitely,
operate extremely rapidly, and can be very difficult to trace back to their humans
‘owners’. These attributes make electronic markets susceptible to such phenomena as
denial of service attacks, which have halted operations at many major businesses [1] [2],
in addition to many others such as ‘bid snatching’ (wherein a set of malicious colluding
agents effectively halt another agent by using attractive bids to fraudulently ‘snatch’ and
non-perform on all the targets’ subcontracts), ‘bid collision loops” (where a pair of agents
halt a Dutch auction by establishing an infinite loop of colliding bids), and so on. The
range of attack types and corresponding responses is growing (see [3] for example) and
seems limited only by human creativity. Even the mechanisms used to help avoid
malicious agent behavior, such as reputation servers, are themselves prone to such attacks
as collusive reputation manipulation [4]. Electronic markets also give unprecedented
scope to the deployment of relatively untried market mechanisms whose vulnerabilities
have not been fully understood. Large-scale remote bidding has been enabled by
ubiquitous telecommunications, for example, and combinatoric auctions have only
recently made practical by the identification of tractable winning bid identification
algorithms [5]. The combination of software agents and novel market mechanisms,
finally, leads to the potential for emergent dysfunctions, such as resource allocation
thrashing (when agents spend a disproportionate amount of time re-allocating resources
among them) [6], wild inventory swings [7], and price oscillations [8]. It has been argued, for
example, that the 1987 stock crash was due in part to the action of computer-based “program
traders” that were able to execute trade decisions at unprecedented speed and volume, leading to
unprecedented stock market volatility [9]. The consequences for unwary electronic market
developers thus include the potential for serious disruptions in the operations of targeted
market participants and/or the markets themselves.
This paper describes a general methodology for identifying and finding suitable
responses for potential failure modes (henceforth called ‘exceptions’) in multi-agent
coordination mechanisms, and shows how it can be applied to the particular context of
electronic markets. We present the methodology itself in section 2, and then describe (in
section 3) how an augmentation of the MIT Process Handbook captures exception
handling expertise in a way that can greatly increase the speed and comprehensiveness of
exception analysis. Both sections will make use of many examples concerning the market
mechanisms known as auctions.
2. Our Exception Analysis Methodology
Our exception analysis methodology is based on the insight that coordination
fundamentally involves the making of commitments [10] [11] [12], and that exceptions
(i.e. coordination failures) can as a result be viewed as violations of the commitments
agents require of one another. Exception analysis thus consists of the following three
steps:
1. Identify the commitments agents require of one another in the coordination
mechanism
2. Identify the ways these commitments can be violated (i.e. the exceptions)
3. Identify the ways these exception can be handled (i.e. the exception handlers)
We consider each of these steps in the paragraphs below. To make the discussion more
concrete, we will describe them in context of the well-known auction-based task-sharing
mechanism known as the “Contract Net” (CNET) [13] [14] [15] [16] [17]:
Contractor Role
Subcontractor Role
Create RFB
(Request For
Bids)
Sen
dR
FB
d
Sen
Select Bid
Aw
ard
d
Sen
Create Bid
Bid
Con
trac
t
ults
Res
Perform Work
Receive Results
Figure 1. The Contract Net Coordination Mechanism.
In this protocol, an agent (the “contractor”) identifies a task that it cannot or chooses not
to do locally and attempts to find another agent (the “subcontractor”) to perform the task.
It begins by creating a Request For Bids (RFB) which describes the desired work, and
then sending it to potential subcontractors (typically identified using an agent known as a
‘matchmaker’). Interested subcontractors respond with bids (specifying such issues as the
price and time needed to perform the task) from which the contractor selects a winner.
This is thus a first-price sealed-bid auction. The winning agent, once notified of the
award, performs the work (potentially subcontracting out its own subtasks as needed) and
submits the results to the contractor. We can see that there are thus at least three key
agent types in CNET: the contractor, subcontractor, and matchmaker.
Identifying Commitments: Commitments can be identified by finding all the places in the
protocol where one agent depends on some other agent in order to discharge its own
responsibilities successfully [10]. There are in general two kinds of commitments:
design-time commitments that are part of the mechanism definition (e.g. that a bidding
agent will send only one bid at a given price level), as well as run-time commitments
created as a result of the operation of the mechanism (e.g. that an agent will in fact
perform the task it was allocated with the contracted cost quality and delivery time).
The first commitment in CNET arises when the contractor queries the matchmaker to get
the list of agents potentially appropriate for performing a subtask:
C1:
matchmaker provides contractor with correct candidates in a timely way
In order for the matchmaker to discharge this commitment, it in turn requires
commitments on its own, including:
C2:
C3:
C4:
C5:
C6:
contractor provides matchmaker with correct skills requirements
subcontractors provide matchmaker with up-to-date skills descriptions
messaging-mechanism provides matchmaker with timely and correct delivery of
results to contractor
host-computer provides matchmaker with sufficient computing cycles
coder provides matchmaker with correct code
Each of these commitments, in turn, may imply further commitments that must be
honored in order for them to be discharged correctly. For example, in order to discharge
commitment C5, a host computer requires that:
C7:
system-operator ensures that host has sufficient available capacity
The process of tracing what commitments are required to discharge others allows us, in
principle, to identify commitments exhaustively, even those that are often left implicit
(e.g. those involving the messaging-mechanism and system-operator).
Note that it is critical that commitments be enumerated from the idealized perspective of
each agent. For example, in an auction the seller ideally wants the following
commitment:
C8:
bidders provide seller with bids up to their true valuation for good
even though in many situations the commitment target (the bidder) may have no intention
of fulfilling it (e.g. in auctions where true-value revelation is not the dominant strategy1).
Many important exceptions represent violations of ideal-case commitments.
Identifying Commitment Violations (Exceptions): The next step is to identify, for each
commitment, how that commitment can be violated, i.e. what its characteristic exceptions
are. The basic set of possible exceptions can be identified simply as the possible
negations of the commitment itself. For example, the message-sender is responsible for
discharging three commitments: delivering the right message to the right place at the
1
A dominant strategy is a strategy that it is rational for an agent to pursue no matter what strategies are
pursued by the other agents.
right time. There are thus three main exceptions: wrong message (the message is
garbled), wrong place (the message is delivered to other than the intended target), and
wrong time (the message is either too early or, more likely, excessively delayed). As
another example, the subcontractor commitment to perform a task with a given quality
cost and duration can be violated by late results, sub-contractual quality, or higher-thanproduced cost (it is logically possible, though unlikely, that early results, unexpectedly
high quality or low costs could also be exceptions).
It is not enough to identify these basic exceptions, however, since by the time the
commitment is violated it is often too late to deal effectively with it. Ideally, one would
prefer to identify, as early as possible, the events that might lead to commitment
violations, i.e. the causes of the root exceptions. We consider these causes to be
exceptions as well, since they also represent system states that can lead to commitment
violations. Unfortunately, it is not possible to exhaustively identify all possible exception
causes deductively. For example, commitment C5 above (host-computer provides
matchmaker with sufficient cycles) could be violated because the host computer has
experienced a software virus or distributed denial of service attack. Before such attacks
were invented, we could not expect to have simply inferred them from the commitment
description. Another example is emergent dysfunctions like those mentioned above.
There is as yet no complete analytic theory concerning which kinds of coordination
mechanisms are apt to face which kinds of emergent dysfunctions. These limitations are a
fundamental property of open systems (i.e. ones that are built from ‘black box’
components). This introduces a experiential component into exception analysis; one must
rely on one’s previous experience in order to be able to fully identify possible exception
types, and an exhaustive identification may not be possible.
Identifying Exception Handlers: Once we have identified the exceptions that potentially
characterize a given coordination mechanism, we are ready to identify possible
techniques for handling these exceptions. If the coordination mechanism does not
currently provide any handlers for an important exception type, then the mechanism
designer needs to select one or more handlers and incorporate them into the coordination
mechanism being analyzed. There is a vast range of possible exception handling
techniques. One can argue in fact that a very significant proportion of human
organizational innovations, including the police, law courts, disaster relief agencies,
insurance, contracts and so on, can all be viewed as being exception handling techniques.
As with exception causes, however, the range of possible handlers appears to be limited
only by human creativity and there appears to be no way to identify all potentially useful
exception handlers from first principles.
3. Exploiting A Repository of Exception Handling Knowledge
The fact that exception and handler identification can be time-consuming as well as at
least partly experiential in nature has led us to explore whether it is possible to
systematically accumulate this knowledge so that we can analyze new mechanisms more
quickly and completely. We have been able to do so using a substantively extended
version of the MIT Process Handbook , a process knowledge repository which has been
under development at the Center for Coordination Science (CCS) for the past six years
[18] [19]. In the following sections we will present the core concepts underlying the
Handbook, describe how these concepts and associated tools were extended to enable
more effective and efficient exception analysis, and conclude with an extended example
of how this methodology was applied successfully to a real-life case.
3.1. Underlying Process Handbook Concepts
The Handbook takes advantage of three simple but powerful concepts to capture and
organize process knowledge: attributes, decomposition, dependencies, and specialization.
Decomposition: Also like most process modeling techniques, the Handbook uses the
notion of decomposition: a process is modeled as a collection of activities that can in turn
be broken down (“decomposed”) into subactivities that themselves, potentially, are
processes.
Dependencies: Another key concept we use is that coordination can be viewed as the
management of dependencies representing flows (of control, data, material …) between
activities [18]. Dependencies can be arranged into a taxonomy, where the three primitive
types include: “flow” which represent one-to-one resource flows (e.g. transferring money
between accounts), “sharing” which represent one-to-many resource flows (e.g.
allocating investment money to stocks) , and “fit” which represent many-to-one resource
flows (e.g. fitting together the designs for multiple subsystems of a given system). Every
dependency is “managed” by an associated coordination mechanism, which is simply the
process that controls the resource flow and thereby coordinates the activities connected
by the dependency. Different kinds of dependencies have different potentially appropriate
coordination mechanisms, e.g.:
Dependency Type
Flow
Fit
Sharing
Possible Coordination Mechanisms
Make to order (just-in-time)
Make to forecast and stockpile
Pre-defined subsystem design interfaces
Post-hoc design conflict resolution
First-come, first-served
Market-like bidding
The key advantage of this conceptualization is that it allows us to distinguish the domainspecific activities of a process from the generic coordination functions, thereby enabling
cross-domain sharing of coordination mechanism ideas.
Process Specialization: The final key concept is that processes can be arranged into a
taxonomy, with very generic processes at one extreme and increasingly specialized
processes at the other. Processes are organized based on their function, so that processes
with similar purposes appear close to each other. This facilitates finding and comparing
alternative ways for performing functions of interest, thereby fostering easy transfer of
ideas. Sibling processes can be grouped into “bundles” with tradeoff tables that capture
the relative pros and cons of these alternatives.
3.2. Extending the Handbook to Capture Exception Handling Knowledge
While the Handbook as described above is well-suited for describing processes, it does
not capture information concerning possible exceptions in these processes. The key
innovation required to do so is the addition of the exception taxonomy, a specialization
hierarchy of exception types [20] [21]. Using this additional element we can extend the
Handbook schema as follows (Figure 2):
processes
requires
coordination
mechanisms
exception
handlers
includes
dependency
is
handled
by
dependencies
(commitments)
exception
types
has exception
is caused by
Figure 2. The Schema for Capturing Exception Handling Knowledge.
A coordination mechanism is simply a kind of process that involves multiple agents, and
resides therefore in the Handbook process taxonomy. Commitments are inter-agent
dependencies and reside in the Handbook’s dependency taxonomy. Every commitment can be
mapped (using the has-exception link) to the characteristic ways it can be violated (its
exceptions); these exceptions reside in the exception taxonomy. Finally, all exceptions can be
mapped (using an is handled by link) to the processes (exception handlers) that can be used to
deal with them. Handlers are themselves members of the process taxonomy, and potentially have
their own characteristic commitments, exceptions, and exception handlers. This simple
augmentation of the Handbook schema has proven adequate, in our experience, to capture all
important aspects of exception handling expertise.
3.3. Using the Knowledge Base for Exception Analysis
Let us now consider how an exception handling knowledge base organized using the
schema described above can improve the speed and completeness of exception analysis.
The key idea is that one can learn much about an entity (e.g. a process or exception type)
by looking at generalizations of that entity. In other words, we can expect that
information (e.g. exceptions or handlers) associated with the generalization is often
inherited by its specializations.
The Coordination Mechanism Taxonomy: In order to exploit this effect, we have been
building a taxonomy of coordination mechanisms (Figure 2):
Figure 3. A fragment of the coordination mechanism taxonomy.
Every coordination mechanism is linked to the exception types that characterize it; a
mechanisms’ exceptions are inherited by its specializations unless explicitly over-ridden.
Every exception is annotated with its typical impact on the associated coordination
mechanism. Auctioneer fraud, for example, typically results in increased buyer costs in
sealed-bid second-price auctions (when the auctioneer inserts a bogus second bid that is
just below to the winning bid).
Our recent efforts have focused on developing the branch of the coordination mechanism
taxonomy that concerns auctions. We have defined an abstract auction model (building
on [22] [23] [24]) consisting of the following steps:
•
•
•
•
•
Bid call. How buyers are invited to submit bids.
Ask call. How sellers are invited to submit “asks” (desired bids).
Bid collection. How bids are collected by auctioneer.
Ask collection. How asks are collected by auctioneer.
Winner determination. How the buyer-seller match is found.
•
•
•
Clearing. How the end of the bidding round is determined.
Information revelation. How/when bid information is revealed to bidders.
Closing. When to close the auctioning permanently.
All known auction types can be represented as specializations of this abstract model. A
typical Dutch auction [25] for example, uses downward bidding for 'bid call', includes no
'ask call' step, determines the winner as the first bid received as the auctioneer calls prices
downwards, clears the bidding round when a bid above the reservation price is received
or the reservation price is reached, reveals the winning bid (and possibly the winner's
identity) after clearing, and resolves ties by selecting randomly one of the bidders
involved. A typical multi-unit Japanese auction [26] involves a calls for bids every time
the auctioneer raises the price, includes no 'ask call' step, collects bids as the number of
units requested by each bidder, all bidders which do not explicitly request the auctioneer
to drop out as part of the winning set, does not reveal any information about bidders' bids
during bidding, and clears the bidding round when the total demand from bidders
matches the supply (number of units of the lot at auction).
Our analysis to date shows that many auction exceptions are inherited by all
specializations of the abstract auction model. All auctions, for example, are potentially
liable to the ‘auctioneer agent crashes’ exception, as well as impersonated bids (where a
bidder presents a bid as if it came from someone else), multiple bids from a single buyer
at a given price (a protocol violation that may represent a denial of service attack),
unsupported bids (where the winner is unable to pay the winning price), or too few
bidders (which may reduce the competitiveness of the auction below a level the
auctioneer or seller is willing to accept).
In other cases we are able to identify exceptions that apply to an entire subclass of
auction protocols. One example concerns information revelation. In any auction where
bid information is not revealed, for example (e.g. as with closed bid auctions such as the
Vickrey [25]), we have the possibility of buyer-auctioneer collusion, since bidders will
not be in a position to determine the correct winners on their own [27]. For another
example, all ascending price auctions (e.g. English or Japanese protocols) are prone to
non-termination of the winner determination step if enough agents (due either to bugs or
malice) simply do not ever drop out.
Some exceptions, however, are idiosyncratic to a quite specific class of auction
mechanism because that mechanism involves unusual commitment types. Japanese
auctions, for example, rely on each bidder explicitly indicating that it is reducing its
demand or dropping out entirely, as the price rises. If an active bidding agent crashes then
the auctioneer will still consider its bids active, prices will be driven above the
appropriate level, possibly infinitely (i.e. non-terminating) if the dead agent’s last demand
exceeded the total supply, and the dead agent may be inappropriately selected as a
winner. The exception handling knowledge base stored in the Handbook thus does not
guarantee that all exceptions can be identified simply by inheritance from the abstract
auction model.
The fact that so many exceptions apply to all or most auction types is a good result
because it means that we immediately know a lot about the potential failure modes in
auctions we have never analyzed before. Even for unusual auction variants, much of the
analysis effort is obviated and the market designer can focus on identifying exceptions
involving the commitment types idiosyncratic to that variant.
Note that even though an unusual action variant may not already reside in the process
taxonomy, some important portion(s) of it may. For example, the Handbook taxonomy
includes a simple generic model for a query/response interaction between two agents,
whose constituent commitment include the following:
C10:
C11:
C12:
C13:
C14:
C15:
agent1 provides agent2 with correct query
agent2 provides agent1 with correct response
messaging-mechanism provides agent2 with timely and correct delivery of results
host-computer provides agent2 with sufficient computing cycles
coder provides agent2 with correct code
system-operator ensures that host has sufficient available capacity
Any query in a coordination mechanism can be viewed as an instance of this abstract
process, and inherits as a result the exceptions associated with it.
The Exception Taxonomy: Exceptions, like coordination mechanisms, are arranged into a
taxonomy (Figure 4):
Figure 4. A fragment of the exception type taxonomy.
Exception types are linked, in turn, to the exception handling processes suitable for
dealing with them. Handlers are inherited down the exception taxonomy unless
specifically over-ridden. Since different handlers have different strengths and
weaknesses, exception types can include tradeoff tables that present the relative pros and
cons of the different handlers for dealing with that exception.
There are three main classes of exceptions:
•
Infrastructure commitment violations, which represent violations of commitments
made by the infrastructure to provide dependable communication and computation.
Examples include crashed host computers, and unreliable or slow communication
links.
•
Agent commitment violations, which represent violations of commitments agents
make to each other. Examples include agents performing tasks late, doing sub-quality
work, or not adhering to the agreed-upon coordination protocols.
•
Systemic commitment violations, which represent violations of commitments made by
the system operator to create an environment well-suited to the tasks at hand.
Examples include not populating the market with agents containing all the skills
needed to perform the tasks at hand, or allowing emergent dysfunctions.
A key benefit of the exception taxonomy is that one can identify potentially useful
handlers for an exception by looking at those linked to its generalizations in the
taxonomy. For example, an agent not paying for a purchased good can be classified as an
‘agent commitment violation’ exception, which is linked to the generic exception
avoidance mechanism called ‘reputation mechanisms’. To give another example,
excessive bandwidth use in a network can be classified as a ‘tragedy of the commons’
exception (a type of emergent dysfunction exception wherein agents over-use a public
resource). The tragedy of the commons exception can be avoided by ‘privatizing’ the
resource e.g. by charging use fees. It is our experience that a relatively compact corpus of
generic exception handlers covers a wide range of exception types, which is good
because that means a relatively small investment in creating the exception and exception
handler taxonomies can be amortized over a broad range of uses.
The Exception Handler Taxonomy: The exception handing process taxonomy (Figure 5)
is where the bulk of the exception handling knowledge base content resides:
Figure 5. A subset of the exception handling process taxonomy.
There are four main classes of exception handlers, divided into two pairs. If a exception
has not yet occurred, we can use:
• Exception anticipation processes, which uncover situations where a given class of
exception is likely to occur.
• Exception avoidance processes, which reduce or eliminate the likelihood of a given
class of exception.
If the exception has already occurred, we can use:
•
•
Exception detection processes, which detect when an exception has actually occurred.
Exception resolution processes, which resolve an exception once it has happened.
One of the key advantages of arranging exception handlers into a process taxonomy is
that it allows us to generate new and potentially useful handlers by re-combining the
elements of the taxonomy into novel configurations. The methodology for generating
novel process alternatives is described in detail in [28].
Closing the Loop: Exception handlers can themselves have their own characteristic
exceptions, which can be of great practical importance. We have already encountered, for
example, the idea that reputation mechanisms can be sabotaged. This is because they
typically assume the following idealized commitments:
C16:
C17:
agents provide all other agents with honest ratings
agents perform for other agents without discrimination i.e. they are no more likely
to non-perform for one agent than another
We can easily imagine agents that violate these commitments, however. Imagine for
example a clique of agents that (perhaps fraudulently) provide each other with a large
number of highly positive ratings, while selectively non-performing for a few targeted
customers. The average ratings of such misbehaving agents will remain high, especially
if, as is sometimes done, outlier reputation scores are discarded by the reputation service.
Handlers exist for dealing with reputation manipulation exceptions, however, including
for example anonymizing the rated agents to foil ‘bad-mouthing’ [29].
A second, multi-tiered, example of this concerns tie bids in Dutch auctions. A typical
response to this exception is to reset the price to a higher level and restart the descending
price clock. A potential exception with this handler is infinite bid collisions, wherein two
or more agents collide indefinitely at the same price and as a result the auction does not
terminate. A typical response to the infinite bid collision exception is to terminate the
bidding after a pre-determined number of collisions and then rely on the auctioneer to
break the tie, for example by selecting the first bid, last bid, or a random bid. This tie
breaking mechanism is, however, potentially subject to the ‘buyer/auctioneer collusion’
exception, wherein the auctioneer favors one bidder over others, e.g. by always rewarding
tie bids to the favored bidder. This exception can be avoided by making bids public so
bidders can verify for themselves who the correct winner is.
Our exception handling knowledge base captures knowledge about handling such
exceptions transparently, since exception handlers are themselves just processes that can
be linked to their own exceptions and exception handlers, allowing recursion as needed.
3.4. The Barings Bank Case
We will now illustrate our exception analysis methodology in the context of a wellknown real-life case: the Barings Bank failure.
In February 1994, 233-year old Barings Bank, one of the oldest and most respected
investment houses in the United Kingdom, went bankrupt [30] [31]. The entire bank
collapsed because of losses of $1.4 billion incurred in a matter of days by a single young
trader, Nicholas Leeson. Nicholas Leeson was a futures trader in the Singapore branch of
the bank. For a number of reasons, which are still not entirely clear, Leeson began to
engage in unauthorized futures trading in the Singapore exchange. Due to inadequate
internal controls and other process failures, Leeson was able to maintain his unauthorized
and highly risky activity undetected by the bank headquarters in London until the very
end. As we shall see, our exception analysis methodology can be used to systematically
point out the gaps in the Barings trading process controls that allowed disaster to occur,
as well as to suggest ways for closing those gaps.
As noted above, the first step is to capture the ‘ideal’ business process. Figure 6 depicts a
simplified but accurate model of the Barings futures trading process; boxes correspond to
process activities, and lines correspond to dependencies (commitments) between
activities:
Customer
Place
Request
Trader
Buy
Futures
Contract
Receive
Certificate
Send
Payment
Bank Treasury
Transfer
Funds
Prerequisite
Flow
Figure 6. The Barings Futures Trading Process
When a customer requests a futures trade, the trader asks the bank headquarters for an
advance of funds in order to cover the customer’s margin account2. Once the funds have
arrived, the trader performs the trade, waits to receive the corresponding security
certificate and finally pays the exchange. In an “ideal” world, a trader only performs
trades when authorized to do so by customers, correct certificates are always received,
and payment for trades exactly match the funds forwarded to the trader by the bank
headquarters. These conditions are implied by the “prerequisite” and “exact flow”
commitments.
The next step is to identify the possible exceptions that are associated with each key
commitment in the “ideal” process model. By consulting the exception repository one can
see that one possible exception for any prerequisite commitment is a prerequisite
violation (“B without A”), that is, the possibility of activity B happening without a prior
occurrence of activity A. In the context of the Barings trade process such violations
would translate into unauthorized trading, unwanted security receipts and unnecessary
payment (Figure 7).
Customer
Place
Request
Trader
Unauthorized
trade
Buy
Futures
Contract
Wrong
certificate
Receive
Certificate
Send
Payment
Bank Treasury
Transfer
Funds
Prerequisite
Funds
misused
Flow
Figure 7. The Barings Futures Trading Process with Associated Exceptions
Likewise, one possible exception for the mechanism managing an “exact flow”
commitment is mismatch between the amount produced and the amount consumed. In the
context of the Barings process this would translate into a misuse of headquarter funds.
After possible exceptions have been identified, the next step is to find handlers suitable
for mamaging the possible exceptions identified in step 1. It turns out that, because the
trading process at Barings involves several independent entities (customer, bank,
exchange) and requires some initiative from the part of the trader, there are were no
practical mechanisms for avoiding the exceptions. There were, however, several
mechanisms for detecting them.
Main
Process
A
Update
LogA
B
Exception
Avoidance
Process
Update
LogB
Prerequisite
Periodic Consistency Check
Exclusive access
Figure 8. Logging is a Generic Process for Detecting Prerequisite Violations
For example, logging is one (out of several) generic mechanism for detecting prerequisite
relationship violations (Figure 8). Logging involves recording all occurrences of
activities A and B in some reliable storage medium and periodically conducting checks
for prerequisite violations. In order for logging to be successful it is, in turn, required that
(a) all occurrences of A and B are reliably logged and (b) the log can only be modified by
the processes that do the logging.
If we insert a logging process for all dependencies listed in Figure 8 we get a model of a
properly instrumented trading process (Figure 9).
Flow
Prerequisite
Exclusive access
Customer
Place
Request
Update
Commits
Trader
Update
Reqs
Bank Treasury
Update
Funds
Buy
Futures
Contract
Update
Recv
Receive
Certificate
Update
Paid
Send
Margin
Payment
Transfer
Margin
Funds
Daily Consistency Check
Figure 9. Barings Process Properly Instrumented with Logging Processes
At this point, we can compare the process derived using our approach with the actual
Barings process. It can immediately be seen that, although Barings did log some
information about trades, it had two crucial gaps relative to the properly instrumented
process of Figure 10 (see Figure 10):
Customer
Barings failed to
compare funds
transferred against
funds used for
client transactions
Place
Request
Trader
Update
Reqs
Bank Treasury
Update
Funds
Buy
Futures
Contract
Update
Commits
Barings failed to safeguard
against exclusive access
Updatetrader
violations because
Recv
was given log modification
privileges
Receive
Certificate
Update
Paid
Send
Margin
Payment
Transfer
Margin
Funds
Daily Consistency Check
Figure 10. Comparison between Ideal and Actual Barings Process
First, it failed to log and compare the amount of funds forwarded by headquarters to the
trader to the amounts actually paid by the trader for customer trades (in other words, the
log labeled “Funds” in Figures 9-10 was missing from the Barings process). Second,
Nick Leeson, in addition to being a trader, was also in charge of the back room operations
in the Singapore branch. This gave him the authorization to modify the trades logs (and
thus violated requirement (b) above of the logging process).
Nick Leeson was able to use these two gaps to his advantage as follows: Whenever he
received a trade request from a customer, he requested an amount of funds far greater
than what was required for the customer trade. He then performed the customer trade, as
well as some additional unauthorized trades on his behalf. All of these trades were
automatically logged into logs “Commits”, “Received” and “Paid” (see Figures 9-10).
Leeson then erased the records of his unauthorized trades from logs “Commits”,
“Received” and “Paid”. Therefore, at the end of each day, the log of “Requests” matched
perfectly the other three logs. By not checking for discrepancies between the funds
forwarded to Leeson and the total funds recorded at the “Paid” log, headquarters
remained unaware of Leeson’s activities until it was too late.
It is probably too simplistic to claim that the Barings disaster would have been avoided if
the management of Barings had at their disposal our exception analysis methodology.
Nevertheless, this exercise demonstrates that these methodologies and tools can be used
in real-life cases to identify potential weaknesses and suggest ways alleviating them.
4. Conclusions
This paper has described a generic knowledge-based methodology for the important
problem of identifying possible failure modes and responses in multi-agent coordination
mechanisms such as auctions. Our experience, based on the application of this
methodology to problems in such domains as futures trading, multi-agent system task
allocation, auction mechanisms, and back-office telephone operations, is that the
methodology can be highly effective in helping designers define more robust
coordination mechanisms.
4. References
1.
Gomes, L., Online Trading Halted at Broker After Site Attack, in Wall Street
Journal. 2000. p. 6.
2.
Richtel, M., Yahoo Blames a Hacker Attack for a Lengthy Service Failure, in New
York Times. 2000.
3.
Sandberg, J., Internet Vandals Pose Threat by Using New Mode of Attack Called
"Smurfing", in The Wall Street Journal. 1998.
4.
Dellarocas, C. Immunizing online reputation reporting systems against unfair
ratings and discriminatory behavior. in Proceedings of the 2nd ACM Conference
on Electronic Commerce. 2000. Minneapolis, MN.
5.
Andersson, A., M. Tenhunen, and F. Ygge. Integer Programming for
Combinatorial Auction Winner Determination. in International Conference on
Multi-Agent Systems (ICMAS-2000). 2000. Boston, MA USA: IEEE Computer
Society.
6.
Youssefmir, M. and B. Huberman. Resource contention in multi-agent systems. in
First International Conference on Multi-Agent Systems (ICMAS-95). 1995. San
Francisco, CA, USA: AAAI Press.
7.
Sterman, J.D., Learning in and about complex systems. 1994, Cambridge, Mass.:
Alfred P. Sloan School of Management, Massachusetts Institute of Technology.
51.
8.
Kephart, J.O., J.E. Hanson, and J. Sairamesh, Price and niche wars in a freemarket economy of software agents. Artificial Life, 1998. 4(1): p. 1-23.
9.
Waldrop, M., Computers amplify Black Monday. Science, 1987. 238: p. 602-604.
10.
Singh, M.P., An Ontology for Commitments in Multiagent Systems: Toward a
Unification of Normative Concepts. Artificial Intelligence and Law, 1999.
11.
Jennings, N.R., Coordination Techniques for Distributed Artificial Intelligence, in
Foundations of Distributed Artificial Intelligence, G.M.P. O'Hare and N.R.
Jennings, Editors. 1996, John Wiley & Sons. p. 187-210.
12.
Gasser, L., DAI Approaches to Coordination, in Distributed Artificial
Intelligence: Theory and Praxis, N.M. Avouris and L. Gasser, Editors. 1992,
Kluwer Academic Publishers. p. 31-51.
13.
Smith, R.G. and R. Davis, Distributed Problem Solving: The Contract Net
Approach. Proceedings of the 2nd National Conference of the Canadian Society
for Computational Studies of Intelligence, 1978.
14.
Baker, A. Complete manufacturing control using a contract net: a simulation
study. in Proceedings of the International Conference on Computer Integrated
Manufacturing. 1988. Troy New York USA: IEEE Computer Society Press.
15.
Boettcher, K., D. Perschbacher, and C. Wessel, Coordination of distributed
agents in tactical situations. Ieee, 1987(87CH2450): p. 1421-6.
16.
Bouzid, M. and A.-I. Mouaddib, Cooperative uncertain temporal reasoning for
distributed transportation scheduling. Proceedings International Conference on
Multi Agent Systems, 1998.
17.
Smith, R.G. and R. Davis, Applications Of The Contract Net Framework:
Distributed Sensing. Distributed Sensor Nets: Proceedings of a Workshop, 1978.
18.
Malone, T.W. and K.G. Crowston, The interdisciplinary study of Coordination.
ACM Computing Surveys, 1994. 26(1): p. 87-119.
19.
Malone, T.W., et al., Tools for inventing organizations: Toward a handbook of
organizational processes. Management Science, 1999. 45(3): p. 425-443.
20.
Klein, M. and C. Dellarocas, A Knowledge-Based Approach to Handling
Exceptions in Workflow Systems. Journal of Computer-Supported Collaborative
Work. Special Issue on Adaptive Workflow Systems., 2000. 9(3/4).
21.
Klein, M. Towards a Systematic Repository of Knowledge About Managing
Collaborative Design Conflicts. in Proceedings of the International Conference
on AI in Design (AID-2000). 2000. Boston MA: Kluwer Academic Publishers.
22.
Sandholm, T., eMediator: A Next Generation Electronic Commerce Server. 1999,
Washington University at St. Louis: St Louis, MO, USA.
23.
Wurman, P.R., M.P. Wellman, and W.E. Walsh, The Michigan Internet
AuctionBot: a configurable auction server for human and software agents.
Proceedings of the Second International Conference on Autonomous Agents.
ACM., 1998.
24.
Rodriguez-Aguilar, J.A., et al. Competitive Scenarios for Heterogeneous Trading
Agents. in Second International Conference on Autonomous Agents
(AGENTS'98). 1998.
25.
McAfee, R.P. and J. McMillan, Auctions and Bidding. J. Economics Lit., 1987.
XXV: p. 699--738.
26.
Cassady, R., Auctions and Auctioneering. 1967: University of California Press.
27.
Priest, C. Commodity Trading Using An Agent-based Iterated Double-Auction. in
Third International Conference on Autonomous Agents (AGENTS'99). 1999.
28.
Bernstein, A., M. Klein, and T.W. Malone. The Process Recombinator: A Tool for
Generating New Business Process Ideas. in Proceedings of the International
Conference on Information Systems (ICIS-99). 1999. Charlotte, North Carolina
USA.
29.
Dellarocas, C. Mechanisms for coping with unfair ratings and discriminatory
behavior in online reputation reporting systems. in International Conference on
Information Systems (ICIS-00) (under review). 2000.
30.
Fay, S., The collapse of Barings. 1997, New York: W.W. Norton.
31.
Zhang, P.G., Barings bankruptcy and financial derivatives. 1995, Singapore:
World Scientific Publishing Co.