HD2a
,M414
(
-81
V
ALFRED
P.
WORKING PAPER
SLOAN SCHOOL OF MANAGEMENT
A CONCEPTUAL MODEL
FOR INTEGRATED AUTONOMOUS PROCESSING:
AN INTERNATIONAL BANK'S EXPERIENCE'
VJITH LARGE DATABASES
William
F.
Frank
Stuart E. Madnick
Y.
March 19 8 7
Richard Wana
nrp 1866-87
MASSACHUSETTS
INSTITUTE OF TECHNOLOGY
50 MEMORIAL DRIVE
CAMBRIDGE, MASSACHUSETTS 02139
'mar 181987
A CONCEPTUAL MODEL
FOR INTEGRATED AUTONOMOUS PROCESSING:
AN INTERNATIONAL BANK'S EXPERIENCE
VJITH LARGE DATABASES
"
William
F.
Frank
Stuart E. Madnick
Y.
March 1987
Richard Wang
#V7P
1866-87
A Conceptual Model
for Integrated Autonomous Processing:
An International Bank's Experience
with Large Databases
William F. Frank
Integrated Information Systems Associates
Warren, Vermont
Stuart E. Madnick
Sloan School of Management
Massachusetts Institute of Technology
Y. Richard Wang
Department of Management Information Systems
College of Business and Public Administration
University of Arizona
Working Paper // 1866-87
Sloan School of Management, M.I.T.
Report #CIS-87-04
Composite Information Systems Project
lAAR 1 9 19S7
RECHfVEO
j
.
TABLE OF CONTENTS
1
Banking Environment
1.1 Autonomy, Integration, and Evolution
1.2 Transaction Investigations
1.3 MIS Applications
Conceptual Model
2.1 Model Architecture
2.2 Autonomy Aspects of Model
2.3 Integration and Evolution Aspects of Model
2.4 Model Components
3. Implementation Experience
3.1 Summary of Results
3.2 System Design
3.3 Model Interpretation
3.4 Environmental and Systems Software Requirements
3.5 Development Histories
4. Concluding Remarks
5. References
2.
1
2
5
5
6
6
6
8
9
13
13
15
20
22
27
32
33
.
1.
BANKING ENVIRONMENT
The banking environment has experienced dramatic changes over the
past
and it continues to change at an accelerated pace.
decades
two
Competitive pressure is increasingly
industry
on
a
multitude
being
fronts.
of
imposed
There
on
the
banking
little to constrain
is
wholesale oriented institutions from crossing interstate barriers
encroaching
on
wholesale
international
countries
many
international frontiers; in
Competition is also crossing
territory.
local banks'
and
banking is becoming much more aggressive as banks vie for the business
of multinational corporations.
technology
Information
1985]. For example,
[Lipis,
^
being
is
extensively
used
in banking
in 1986 the value of computerized payments
processed by the New York Federal Reserve and New York Clearing
exceeded
often
critical
to
trillion
$1
keep
with
pace
in
a
day. Technology has been
single
volume
increased
the
House
of
financial
activities; payments processed through New York financial institutions
have
increased 50-fold in the past 20 years, to the point where every
four days an amount equal to the total annual US GNP is turned over.
During the 199Q's, technological innovation
more
dominant
effect
on
financial
the
will
have
an
even
environment than
services
interest rate volatility. The development of sophisticated information
processing facilities
broader
range
of
are
enabling
products and services on
efficiency. Technologies such
machines
[Hsiao
institutions
and Madnick,
distributed
as
1977; Madnick,
advantage
by the industry to gain strategic
19841
-
a
1
offer
to
a
much
global basis with great
systems
1977]
[Keen,
and
database
are being employed
1986;
McFarlan,
INTEGRATION, AND EVOLUTION
1.1 AUTONOMY,
This paper reviews and analyzes the development and deployment of
a
conceptual
model
for
The cultural
international bank.
non-integrated
autonomous
responsibility for getting
possible
lowest
thus
integrated autonomous processing in
tradition
approach:
a
job done is
itself
To
cope
with
satisfied
the
the
changing
it was recognized
designing
in
information
management)
is
developed
Each product manager,
letter of credit,
autonomously
in
general,
buy/modify
suitable application packages).
bank, this autonomy is critical since
responsible
for
his products:
each
loans,
separate component
by
complete
has
his hardware acquisition and software development choices
hire his own development staff, retain outside
or
to
an
autonomy, integration, and evolution.
personnel.
over
which
in
constantly
the
Each bank product (e.g., funds transfer,
cash
favored
distributed
and the distributed autonomous culture,
that three key goals need to be
systems:
bank
the
in
level, and in which the independence of projects was
maximized.
always
environment
a
forces
major
a
contract
freedom
(e.g.,
programmers,
In the culture of this
manager
is
held
solely
excuses such as "the data processing
people didn't do it right" is not acceptable. When information must be
exchanged, it was usually accomplished by "tape hand-offs", usually at
night, as depicted in Figure
1.
On the other hand, the needs for integration have been increasing
rapidly both at the user level and database level
had
its
.
Since
eacli
system
own directly connected terminals, users that required access
to miiltiple systems had to have multiple terminals in their office, or
walk to an area of the building that had
a
terminal tied to the system
^
Terminals
and other
network
interfaces
External
Interface
External
Interface
Routines
Routines
Processing
Processing
Routines
Routines
Database
Database
Routines
Tape
Routines
'Hand-off"
'Shadow"
"Original"
database
database
Database
LETTER OF CREDIT
LOAN
SYSTEM
SYSTEM
Figure
1.
Independent Autonomous Systems
needed. The tape hand-offs were used to create "shadow"
of
Since the shadow database diverges from
other's real databases.
each
databases
the real database during the day,
inconsistencies could result.
The problem of integration has been intensified by the
need
for
evolution in at least three areas: current products, new products, and
new
technology.
the
As
current products become more sophisticated,
information
there is need to acquire more
tape
Increasing
from
the
other
hand-offs lead to processing complexities and do not
address the need for up-to-date information. Many of the new
(e.g., cash management) are,
existing
products
--
to
in fact,
a
completely
produce
requirements
for
systems would be
new
a
dramatic
tape hand-off information sharing.
maintain an efficient and cost-effective environment, it is
to
be
able
to
take
advantage
of
products
repackaging and combination of
expensive, time consuming, and impractical due to
in
systems.
increase
Finally, to
important
new hardware components without
disrupting or discarding existing systems.
Traditional centralized
provide
integration,
and
a
management
system
The purpose of this paper is twofold:
1)
conceptual model, based upon [Lam and Madnick, 1981; Madnick
Wang,
1987], that describes the architecture for an evolutionary,
integrated, and autonomous environment; and 2) report
to
strategies
but have limited capabilities for evolution and
reduce managerial autonomy.
present
database
the
experience
date in implementing this model by the institutional banking group
of a major international bank.
During the past three years,
implemented
five
products/services
have
been
using this conceptual model. This paper focuses on two of
-
4
these systems: transaction investigations and
management
information
system (MIS) reporting.
1.2 TRANSACTION INVESTIGATIONS
Complex
international
volume can be error
discrepancies
prone,
transactions performed in high
resulting
in
significant
a
number
between customers' records and bank records.
inquire about the source
responsible
financial
reviewing
for
these
of
discrepencies
and
Customers
bank
the
is
records to establish what it did and
its
why, supplying these records to the customer, and making good
the
if
The bank's activity with respect to such an inquiry
bank is in error.
is
of
called an investigation.
Investigations
are
performed
They are usually
processing
transaction
the
job
with
systems,
of sizable staffs in a large bank.
the
aid
tracked
satisfaction,
to
the
of
a
history, which
productivity,
customer
and the efficacy of the transaction processing systems.
The Historical Data Base (HDB)
investigations
indicators
allow
on
microfiche, and printed reports. In
addition, the investigation activity itself generates
should be
stored
data
of
and
,
needed by the bank
to
support
these
other applications covering the previous 90 days,
must be able to hold at least 40 million records.
1.3 MIS APPLICATIONS
Each transaction
reports.
processing
system
generates
summary
various
Periodically, integrated MIS reporting for upper management
was accomplished by manually entering data from
a
variety
reports and other sources into an Apple computer spreadsheet.
of
these
.
Management
independently
previously
integrated
presented
new and more comprehensive MIS systems which
desired
In total,
data.
a
generated
inconsistently
and
database of at least 12 million records,
with over 250,000 additions per day, is needed to
hold
relevant
the
Furthermore, these new systems should facilitate the modelling
data.
of the effects of alternative decisions (i.e., what-if analyses fed by
current real performance data)
2.
CONCEPTUAL MODEL
2.1 MODEL ARCHITECTURE
The model developed consists of seven major functional components
as depicted in Figure 2.
into
five
with the integrating application-independent layers (external
layers,
interface, message control, data control, and
surrounding
separated
shared
data
resource)
application processing components [Madnick and Wang,
the
For this bank, these
1987].
separated
These components are
processing
application
components
are
into three classes of applications: transaction processing,
information processing, and administrative processing.
2.2 AUTONOMY ASPECTS OF MODEL
This architecture attempts to mediate the conflicts
goal
the
of
autonomy
except
layers,
themselves
to
between
the
and the goals of integration and evolution. All of
for
message
control
and
data
control,
lend
unlimited autonomy. Each product manager could acquire
and manage his own resources
including
1)
terminal/network
gateway
INPUT/
External
Interface
OUTPUT
Message Control
£
PROCESSING
Transaction
Processing
L
Information
Administrative
Processing
Support
Data Control
I
DATABASE
Shared
Data
Resource
Figure
2.
Conceptual Model
.
hardware,
computers and software, and 3)
processing
application
2)
database computers and software.
In the past,
were
bundled
as shown in Figure
together.
earlier, these three
1
In practice,
decisions
the primary concern for autonomy
involved the application processing, with
lesser
a
concern
for
the
database, and minimal concern for the external interface. It is in the
processing
application
manifest.
as
It
is
that
functionality
the
product is
the
of
important that enhancements and corrections, as
well
the initial development, be able to proceed with minimal needs for
coordination or delays due to the managers of other areas of the
bank
and other computer systems
Given
the
architecture
Figure
in
2,
each manager has complete
control over his application processing system.
Furthermore, as
many
separate database systems as needed, or desired, can be created. It is
expected
that
initially
theoretically needed, due to
integrated
will
there
be
influence
the
of
databases
more
practice.
past
than
The
autonomous architecture provides access to these databases
by other applications as well as an evolutionary path
eventually
for
integrating these databases, as the needs for integration intensify.
2.3 INTEGRATION AND EVOLUTION ASPECTS OF MODEL
There are several underlying concepts and components of the model
which address the issues of integration and evolution.
Message
Control
the model. They are
coordinated.
application.
and Data Control perform unifying functions for
the
points
For example,
Furthermore,
which
at
in principle,
application
-
8
-
all
processing
will
be
any terminal can access any
subsystems
can
utilize
the
.
to manage and maintain data which is common to
Resource
Data
Shared
more than one component
Message Control and Data Control
The
entities.
components
other
both
are
conceptually
single
types of processing functions.
are
transaction
multiple
There may be many instances of each type (e.g.,
processing systems and miiltiple shared data resources).
2.4 MODEL COMPONENTS
It
is
important to realize that the model components of Figure
They could be mapped to
are logically separate.
various
ways.
actual
hardware
2
in
For instance, in the transaction investigation system,
each component resides on a separate processor; whereas,
in
the
MIS
system, many components reside on a single processor.
2.4.1 External Interface
The external entities interfacing with these banking systems fall
into
3)
five categories:
1)
payment networks,
customer terminals, 4)
professional
2)
communication networks,
workstations,
and
5)
other
intra-bank and/or inter-bank systems.
2.4.2 Message Control
Message
processing
coordinates the passage of messages between the
Control
components.
involves
This
routing,
translation,
sequencing, and monitoring:
o
Routing
accepts
a
request
the
for
delivery
of
messages to a
particular logical function and determines the appropriate physical
address to which the message should be delivered. Routing can
accommodate
thus
changes in the availability and location of functions.
9
-
.
Routing can also help coordinate requests for services that involve
several functions.
o Translation maps a
limited number of protocols from one standard
to
another
o
Sequencing
determines
order
the
in
which
messages
are
to be
delivered to recipients on the basis of established priorities.
o Monitoring determines the state of messages within the system at any
given time. Monitoring thus includes
integrity
of
message
a
the
time
the
from
responsibility
it
for
the
presented by one
is
component until it is accepted by another.
2.4.3 Transaction Processing
Transaction Processing refers to the applications
the
financial
customer's
retrieve and
update
balances).
The
significant
a
These
instructions.
sub-functions
amount
of
data
execute
typically
systems
Transaction
of
which
(e.g.,
Processing
client
include
validation, risk management, accounting, and recording.
o Validation functions are those which perform review of
instructions
to ensure that all information needed for processing is present and
that
for
the
data
"repaired"
is
elements.
Incomplete
by
either with
consistent with previously defined rules
information
augmenting
information
or
or
invalid
requests
may
be
clarifying the request information,
available
internally
or
from
external
sources such as the requestor.
o
Risk
Management
functions
those
are
which
verify
that
transactions being processed do not violate limits, conditions,
policies
regulatory
established
agencies.
by
the
customer,
these
Ideally,
10
-
the
or
the bank, or the various
functions
should
be
with
synchronized
where
this
is
"after-the-fact"
feasible,
not
of transactions.
processing
the
In some cases
Management
Risk
functions can be used to initiate corrective action.
o
Accounting
records
completed.
Accounting
impact
cumulative
the
are
functions
transactions
the
of
characterized
being
as
continuous over time, in contrast to the discrete events which take
place in most of the transaction processing functions. For example,
the banking environment, accounting takes place on two distinct
in
levels: customer accounting and organizational accounting.
2.4.4 Information Processing
Information Processing refers to all the subsystems that
analysis,
calculations,
or
restructuring
of
data
the
The sub-functions
consolidated financial statement).
perform
of
(e.g.,
Information
Processing include user interface, static reporting, ad hoc reporting,
and access to outside data resources.
2.4.5 Administrative Support
Administrative Support provides facilities for the performance of
office
functions
by
administrative
or managerial personnel.
activity is required to maintain organization, procedural or
Example
information.
processing,
facilities
correspondence
files,
include
and
electronic
inventory
This
personal
mail,
word
controls.
The
sub-functions of Administrative Support provide
o Template Presentation for the preparation of predefined documents,
o
Editing
and
Formatting
for
the
alteration
and
preparation of
documents and for automated checking of spelling and style.
o Mail Delivery and Storag e for the transmission of documents from one
system to another.
11
.
o Document Storage and Retrieval
maintaining
for
documents,
lists,
and passages from documents in an organization that allows
tables,
on
retrieval
the
basis
of
their
of
any
attributes
and
relationships
Functions
Control
o
to
administrative
monitor
activities
by
maintaining task related information including status, performance,
and activity data,
preparation
o Graphic Functions for the
diagrams,
of
flow
charts,
organization charts, and other illustrative communication tools.
2.4.6 Data Control
Data Control coordinates access, presentation, and the passage of
data
between
processing
functions and the Shared Data Resources.
routes queries and updates to the appropriate component of the
Data
Resources,
performs
It
Shared
security and priority functions, maintains
concurrency control over the shared data, and returns responses to the
appropriate processing
Data
function.
perform
must
Control
the
following functions:
o
Security ensures that there is no unauthorized access to the Shared
and controls the view
Data Resource
of
permitted
data
the
to
different users,
o Presentation provides standard,
query
updates
and
data
flexible and simple means for making
definition
Control contains data manipulation
requests.
and
data
Therefore, Data
definition
language
processing functions,
o
Routing determines which segments of the shared data
access, and passes the
returns
the
request
to
those
segments,
a
request must
as
well
as
results of the requests to the appropriate processing
12
-
one
function. Alternate routing may be used if more than
copy
of
the data exists,
o
Sequencing
determines
the
requests
of
priority
to ensure that
response times for data requests are within acceptable limits,
o Concurrency Control ensures that multiple,
alter
the
active
requests
do
not
data so as to create an inconsistent state within
same
the Shared Data Resource.
2.4.7 Shared Data Resources
Shared Data Resource is the component responsible for holding the
information common to one or
Although
this
activity
is
components
other
more
of
Model.
the
logically centralized in the Shared Data
Resource, it may contain multiple elements (storing different segments
of the shared data, or different organizations of
the
shared
data).
The Shared Data Resource performs two functions:
o
Information Management determines what information must actually be
stored
and
retrieved
transformations
to
satisfy
the
request,
performs
the
necessary to produce the required information, and
determines how the information is to be stored or retrieved,
o Storage Management determines physical
locations of data and
access
storage storage devices.
3.
IMPLEMENTATION EXPERIENCE
3.1 SUMMARY OF RESULTS
The
conceptual
model
Figure
of
2
has
provided
organizational guidelines for system development over the
years.
last
general
three
Two particular projects completed in that time, including two
13
large databases (20 gigabytes and 1.5 gigabytes), will
technology
VAXCLUSTER
the ORACLE relational database management
and
system was used extensively to implement
portion
STRATEGIM
development
the
of
Most
language.
described.
be
one
of
of
other
the
of
conceptual
the
model.
A
applications used the
the
applications
were
either
programmed in the C language or used existing packages.
The goals which lead to the development of these two systems were
threefold
order
(in
importance):
of
applications for particular user
repository
further
and
groups;
dispatch
point
provide
to
2)
according to
which
in
a
MIS
data
of
central
a
for all banking transaction
application
processing historical data; and 3) to provide an
architecture
effective
create
to
1)
various
kinds
partial order of increasing levels
was
systems
organized
abstraction
of
or
aggregation, so that higher levels of data would be created by batched
flows of data from lower levels.
The
were
design and implementation of the systems to meet these goals
greatly
considerations
affected
mentioned
functionality and
by
cultural
the
earlier.
performance
factors
business
and
Furthermore, experiences with the
characteristics
of
ORACLE
also
are
reported along the way.
The first and last of the goals (good applications and structured
aggregation)
have
been
largely achieved by the implementation.
second goal (shared historical
cautiously.
Technology
resource)
data
was
approached
The
very
makes this goal today more feasible than it
was in 1984.
Despite much skepticism about the
database
system
on
databases
as
14
-
performance
largo
and
of
a
active
relational
as
these,
particularly on minicomputer technology, the applications
making
use
of these databases ultimately performed very well.
3.2 SYSTEM DESIGN
3.2.1 Application of the Conceptual Model
plans for new systems called for the gradual segregation
General
of
development
systems
efforts,
and
hardware
of
and
software
components, into specific classes of systems which would correspond to
components of the conceptual model.
important that this be done on an evolutionary basis since
It was
the inventory of existing systems constitutes some 20 million lines of
There was (and is) insufficient business motivation to replace
code.
all these working systems even to achieve such general goals
Instead,
integration.
as
new
as
data
applications are built and old ones
replaced, it is intended that they be brought into greater
conformity
on an application-by-application basis.
This means
with
model,
the
that only those pieces of Integrative components required to support a
developing application may possibly be built.
In addition,
and
hardware
it was not practical that
Instead,
it was expected
that
and accepted as the basis for new development in
Message
Control
available.
and
Over time,
Data
it has
components
these
At the time the model was proposed
purchased commercially.
be
software
required for Message Control and Data Control be custom
built for the bank.
would
integrative
the
Control
software
the
was
bank,
not
complete
commercially
increasingly become available. Thus, one
of the major values of the model has been positioning the bank for the
arrival of such new technology.
-
15
.
The following two sections describe the ways in which
of
the
conceptual
historical database
related
model
were
system
applied
to
ideas
development of the
the
transaction
for
the
investigation
and
the
system, first from the point of view of their functional
MIS
organization, and then
from
point
the
of
view
of
implementation
issues.
3.2.2 Role of a Historical Database
resource shared by
systems
database
historical
The
multiple
envisioned
was
as
providing
Transaction
applications:
a
data
processing
would no longer have the responsibility of storing historical
data nor need to produce
the same
data
Instead,
all
for
a
different
the
systems
information
processing
systems.
transactions and proofed account balances of
completed
each type would simply be
processing
variety of different summaries and views of
would
v^/ritten
to
a
Information
common database.
each extract the data they needed from the
historical database, and use that data independently from the uses put
to it by other systems
The goals of such a historical database was to simplify the
of
transaction
work
processing systems, and more importantly, to simplify
inter-system data flows.
The current flows had
reached
a
level
of
complexity which were, in total, no longer known to any one person and
had
no discoverable principle of organization.
Of most urgency, dependencies
people several months to catalogue.
tape
They required several
on
hands-off between systems meant that each year more systems were
unable to complete their off-line
available,
especially
because
work
in
the
daily
time
periods
customers all over the world demanded
that some systems be available on-line virtually all the
16
-
time.
The
TXN
Processing
Systems
Financial
Accounting
Figure 3.
Database
of
Historical
Role
to be provide by this new organization are shown in Figure
data-flows
3.
3.2.3 Structured Aggregation in MIS Systems
The
integrated
MIS
discussed
benefits
functional
in
addition
above,
was,
system,
providing
to
transaction
the
like
the
investigation system, intended to improve the overall organization
inter-system communications, by deciding in
of
systematic way which MIS
a
applications should communicate with which other applications.
This
to be accomplished by describing the information input
was
requirements
and
applications
in
output
information
the
desired
various
of
MIS
uniform way, and then identifying the lowest cost
a
(least transformation required) connections between such applications,
so that the output of some applications becomes the input to others.
Treating the output of each MIS application as
own
right,
regarding
and
the
primary
role
application (standard cost accounting) as
a
the result is a partial order of aggregated
databases
depend
ultimately
depicts the notion of
databases
a
on
database in
a
of
its
sort of MIS
this
data aggregation function,
databases
which
in
all
Figure 4
the raw historical data.
partial order. The current collection
of
MIS
created consists of seven levels and the dependency diagram
barely fits on
large wall.
a
The general design of the MIS system is in fact to regard various
sets
of
tables
as
levels
of
aggregation,
created
by
specific
applications and supplying data for further applications. Although the
processing
is
quite
maintained
as
a
complex,
single
shared
the
fact
data
that
resource
all
has
simplified the operations of the MIS system components.
18
of
the data is
dramatically
Level 4
Aggregation
3.3 MODEL INTERPRETATION
"interpretation"
By
mean
we
functional
a
components to hardware and software components.
information
For
instance,
environments,
processing
processors
front-end
The interpretation of
processing
and has varied over time.
distributed
being
now
are
model
of
transaction
the model has varied considerably between the
and
mapping
geographically, which has caused some changes and extensions.
interpretation
The
provide
of
environment
the
investigation
system,
the
for
and
its
presented
model
the
development
of
since
the
was used to
the
transaction
later interface to the funds transfer
system. The initial interpretation involved a "null"
layer,
here
message
control
information processing applications did not
early
communicate with each other, and all screen management was done in the
same processor as the applications, although with
Only
when
later,
computers
and
capable
of
processors
applications
management
screen
software
communicating
via
software.
separate
running
on personal
datagrams
with
the
became commercially available, did a message
control layer become significant in information processing.
The five layers of the conceptual model are interpreted as
in
Figure
5:
(1)
an external
interface system consisting of terminal
controllers, wide area gateway boxes,
network
interfaces
terminal
a
to
for the application host machines,
host
of
the
application
software;
(3)
host-to-host Ethernet with data control;
of
software
(4)
ethernet,
and the screen
management software on these hosts; (2) an application layer
only
shown
consists
message control, sharing
data
control
a
consisting
on the application hosts enabling communication with the
database processors over the host-to-host Ethernet, the communications
20
-
t
<
X
UJ
^- -
I
software on the front ends of the database processors, and
of
a
database management system, which includes query analysis and
the
concurrency control; and (5) shared data resources consisting
back
portion
of
the
of the database management systems running on the database
ends
processors, and the storage
systems
which
to
they
connected.
are
These components are described in more detail below.
3.4 ENVIRONMENTAL AND SYSTEMS SOFTWARE REQUIREMENTS
bank's
The
development
emphasize individual projects
policies
that are as small as possible, both in number of
Ideally, projects should take less than
frame.
more than
5
developers [Appleton,
applications
processing
transaction
time
and
year and require no
a
1986].
The environmental software chosen to
processing
personnel
differed
from
applications
support
these
information
that chosen later to support
(such
as
the
transfer
funds
The major reasons for this difference are the technical and
system).
differences
performance
between
transaction
the
processing
and
information processing systems.
The
information
processing systems communicate with the outside
world largely interactively; their databases are the
which
must
addition,
this
components
fully recoverable, and the tolerable time to recovery
be
may be as long as several hours.
systems
only
is
typically
information
a
(While
for
transaction
processing
matter of minutes or even seconds).
processing
systems
are
highly
In
fluid:
requirements change from month to month.
The
primary
environmental
software required on the information
processing side were flexible screen managers and database
22
management
well
systems,
as
addition,
these
communications
host-to-host
as
systems
themselves
lend
to
software.
In
prototyping
and
non-procedural languages.
3.4.1 Operating Environment
hardware and software chosen to realize these components was
The
overwhelmingly based on the DEC VAX. The group's processors
include
thirty
about
VAXes
currently
two IBM mainframes. While the IBMs
and
exchange messages with the VAX systems, the two environments
are
not
integrated into the same conceptual model: the IBM systems,
currently
developed earlier, perform high volume batch
work
running
purchased
packages which provide end-to-end support for these applications.
for this preponderance of VAXes were the experience
reasons
The
base of available developers and support
cultural
VAX
favoring
earlier,
as
the
computers
for
well
analysis
indicated
capable of handling projected workloads, and
were
systems
as
separate
preliminary
Thus in cases where
separate jobs.
that
noted
forces,
personnel
where specialized software was not immediately available off the shelf
for MVS systems, VAXes have invariably been chosen.
In
case
the
of
availability on the VAX of
tracking
was
since here the size of
As
a
a
system,
the
software package to support investigation
a
the
might
database
indicated
have
that
a
better prima facia choice.
result of this choice, VAXCLUSTER technology has been very
widely used in
a
variety of ways.
complexities have been omitted.
processors
investigation
most important factor in the selection of the VAX,
the
mainframe would be
transaction
the
are
To keep Figure
5
simple,
certain
For example, some of the applications
also connected to the Computer Inter-connect (CI) Bus
23
-
.
to allow them to take the place of
Furthermore,
increased
for
defective
a
processor.
database
each Ethernet shown has an
reliability,
additional backup Ethernet.
3.4.2 External Interface
All communications with the ultimate
data
Certain
based.
downloaded
are
physically
terminal
is
for
spreadsheet
Terminal-to-host
and
(part of the External Interface system) is
communications
accomplished
users
PCs
to
printing.
manipulation, graphics, and report
PC-to-host
system
by
Ethernet
an
fact,
(In
one
two:
for
production systems and another for development systems which serves as
backup for the production network)
Bridge
controllers (local terminal controllers, dial up-systems,
connection
and X.25 gateway controllers, which provides a
terminals
via
an
in-house worldwide network) are used to serve both
They communicate with each other via
the terminals and the computers.
XNS
remote
to
Sun Workstations are used on this network as network control and
.
configuration management devises.
Screens are managed from
Viking
VAX
the
using
machines,
application
Forms Management software, which in turn communicates with the
application software.
3.4.3 Applications
The
transaction
investigation
software
application
largely
consists of calls to the screen management and the database management
systems.
In addition,
it performs any processing required to make the
necessary transformations between these
purchased
to
support
two
systems.
investigations,
these
considerably more than this.
-
24
in
The
package
actuality,
does
3.4.4 Data Control
The Data Control
software
and
general
level consists largely
purpose
hardware.
Control could not be implemented, owing
to
management
database
of
All the features of Data
the
lack
commercial
of
distributed database management systems at the time of implementation.
addition
In
to
development
productivity,
concerned performance, because of the large
criteria were:
important
Other
as having the longest
gains,
relatively
2)
1)
databases.
the
of
preferences for relational systems
future and offering
widely
size
issues
major
the
used
the
systems,
productivity
greatest
and
adherence
3)
to
standards and ad-hoc standards when possible, such as CODASYL or SQL.
the
At the start of
project,
there
was
mostly
considerable,
unsolicited, expression of opinion and concern that relational systems
poorly" and were therefore unsuitable for large databases.
"performed
Of course it is meaningless to talk
except
with
performance.
database
respect
performance
bad
or
both the historical database and
be written almost solely in batch mode.
volume of such writes was quite
night),
good
to a specific use and to specific parameters of
In this case,
would
about
large
(300,000
512
the
MIS
Although the
byte
rows
per
the writes were all appends, no update; reads and writes were
never expected simultaneously against the same tables; all reads
were
predictable since very little access to the databases would be ad-hoc;
and
the
database
could
be
designed
in
svich
a
way that most read
requests would require no joins between tables.
Performance projections were established by study of the
of
results
benchmarks performed by other institutions, by review of published
performance analyses, including
a
study
-
25
-
by
the
National
Bureau
of
.
Standards, and by analysis of the likely effects of design features of
the systems.
It
established
quickly
was
that
commercially
the
available
systems based on pointer chains (which happened to be coextensive with
CODASYL
the available
nightly
number
of
batch
while the systems based
would
consideration)
were
systems)
even within
appends
on
permit
incapable
indexes
(the
performing
of
the
period of 24 hours,
a
relational
under
systems
batch appends within a number of
these
hours
The two systems considered most seriously were INGRES and ORACLE.
Benchmarks
well
as
performance
in
opinion
general
as
performing
unlocked
suggested
ORACLE
that
against single tables in
reads
large databases was significantly superior to that of INGRES.
ORACLE
linear (and almost flat) increase in response times to such
exhibited
database
simple queries as the size of the
superior
's
to
ORACLE
INGRES
increased.
in the performance of complex joins,
was
and offered
superior fourth generation development tools.
For
appeared
the
to
type
of
production
applications
of
ORACLE
be a better choice, although neither system would be,
its then current version for VAX hardware,
number
envisioned,
simultaneous
investigation system.
capable
ultimately
users
of
handling
projected
for
in
the
the
Special design approaches were used to overcome
these problems. Furthermore, one could be confident that both hardware
and software would get even faster soon.
3.4.5 Shared Data Resource
The data resource used in all the VAXCLUSTER systems is supported
by DEC cluster storage, which consists of the very high speed
-
26
-
CI
Bus
.
(70
megabits
per
in
such
a
number of processors and a
(currently
controllers
number of intelligent
devices)
connecting
second),
a
total
of
such
16
way that any processor can communicate with any
a
Controllers are in turn connected in pairs to dual ported
controller.
disks, allowing full redundancy of hardware components.
within the cluster, when
For instance,
processor fails, its work can be taken over
a
by another processor which has immediate access to
same
the
of
all
disks
3.5 DEVELOPMENT HISTORIES
3.5.1 Changes in the Plans
it became clear that the strength of
In developing these systems,
certain
features
of
culture
bank
the
was
greater
even
than
anticipated:
independence and competitiveness between managers, and a
predilection
for
achieving
tangible
fast
needed
Steps
results.
smaller than the original plans
integration
actually
called for.
In building and supporting stand-alone applications,
development group has
the
be
to
responsibility
total
to
for
each
delivering
an
to users, and each group has a strong aversion to relying
application
on cooperation from other groups, not
equally
responsible
for
some
deliverable.
Only
a
portion of the transaction processing data (that involved
with funds transfers)
investigation
application.
data from all the
already
had
aggregation
was
required
required
capability
by
support
the
of
needs
the
The MIS system, however, required summary
transaction
the
to
the
processing
of
MIS
27
systems.
providing
the
applications,
systems
These
lowest
since
level
they
of
were
supplying
data
this
to current applications.
A coordinated effort
would therefore provide theoretical future benefits, while
increasing
current development time, cost, and risk.
As
result,
a
transaction
the
investigation
HDB
/
MIS
and
projects, which were started at approximately the same time, wound
going
each
separate
its
That is, instead of extracting and
way.
aggregating data from
the
partially
data
up
HDB,
the
system
MIS
receives
its
own
processing
and
Projections called for at least 100 simultaneous users using
the
aggregated
transaction
from
feeds
financial accounting systems, as shown in Figure
6.
3.5.2 Transaction Investigation System (HDB)
investigation
and
system
having
database, with maximum rates of
read only access to the historical
database request each five seconds.
1
The applications and the database are supported on
as
earlier
shown
Figure
in
A VAX 8600 is used to support the
5.
database, while the investigation application
11/785
"front
end"
machines,
which
runs
communicate
machine via DECNET and the VMS mailbox facility.
application constructs a query in the form of
of parameters.
VAXCLUSTER,
a
a
on
multiple
VAX
with the database
The
investigation
"query type" and
a set
The historical database system translates this into an
SQL query, and returns the response table to the application.
The
configuration has grown to the two front-end ll/785s and the
8600 "data base processor" shown in Figure 5, while
through
evolutionary
have
also
capability
been
has
gone
Several changes in design to improve
two major new releases.
performance
ORACLE
made
over
the
years.
of the conceptual model
facilitated all of these changes.
-
28
The
modular
architecture greatly
Financial
Historical
Accounting
Database
MIS
Systems
Figure
6.
Current Data Flows
MIS
Database
The initial installed production system (on one VAX 11/780
end
VAX
one
and
11/785
front
back end) supported 30 users with database
query response times of 90 to 120 seconds, meaning that the decoupling
of queries and interaction was essential to the acceptability
the
of
The current software and hardware supports 160 simultaneous
system.
response
users with database
automation,
search
each
times
for
of
to
5
historical
seconds.
7
data
Before
required at least
15
minutes of an investigator's time.
The full cycle of
training,
and
development,
design,
testing,
installation,
live operation of the transaction investigation system
required about six months, with one
programmer
responsible
for
the
software development.
A
significant
number
small problems, and a few large ones,
of
were discovered in the course of development. Many of
could
have
been
anticipated
these
problems
by careful up-front studies.
However,
such studies would have required as much time as building
did.
the
system
Without the flexible and powerful environment used, the same
sorts of unanticipated
development
problems
disasters,
as
would
often
they
have
had
been
likely
to
cause
done in the past in the
experimental culture of the bank.
For example, it was discovered during the course
that
the ORACLE "money" data type would not hold large enough amounts
for the needs of a very large bank.
tape
It was
also discovered
that
the
recovery of very large tables would take much too long, and that
mirroring was not economically justifiable, while at
the
development
of
re-indexing
of
very
large
tables
was
taking
original analysis counted only the time required to
30
same
the
time,
too long (the
index
the
newly
each
data
appended
day,
not
the entire 20 gigabyte database.)
solve this problem, tables were partitioned by dates, and
To
application
code was written to permit searches across ranges of dates.
3.5.3 MIS Systems
phase
initial
The
of MIS system development and implementation
also required about six months, with three contract developers and one
About one third of this time was devoted to learning to read
manager.
non-integrated
correctly the tapes from the various
sources
of
MIS
and developing programs to map these tapes to relational tables
data,
without repeating
groups,
record
multiple
types,
variable
length
records, etc.
innovation of the system was the virtual elimination of
major
A
the printing of MIS reports: particular pages of reports
of
interest
to individual managers are viewed on terminals, and printed locally if
so desired.
system had intense advocates in the MIS department, who
MIS
The
specified exactly what they wanted to see in the system.
common
the
in
bank
delivered to them.
with
enriched,
for
The
it
is
this
more
users to ignore new systems until they are
system
constantly
been
has
the MIS group developing
use of SQL and ORACLE development tools
fact,
It is
enhanced
and
great deal of skill at the
a
for
applications.
small
In
system is not called the MIS system by most of the users,
called the ORACLE system.
used
The summary data from this system began to be
planning
sessions,
which
to
lead
of
a
strategic
MIS
desire
purposes.
forecasting and analytic modelling
creation
the
in
business
to use the data for
The
result
was
the
system, using the STRATAGEM modelling
31
which
language,
applications
and
analytic
of
receives
from
data
some
strategic
purposes,
the
highest
level
The use of this system for
ORACLE MIS system.
the
of
rather
reporting
purely
than
purposes, has grown slowly but steadily.
4.
CONCLUDING REMARKS
evolutionary
movement
The
blueprint.
integration
complete
described
model
conceptual
The
from
autonomy
complete
to
and will probably never occur in this
slow
is
in this paper has served as an
organization, nor any organization realistically.
By separating the
control,
and
external
interfaces,
message
control,
data
the database components from the application processing
components, the approach presented here provides for high
integration while preserving significant autonomy
-
degrees
of
and the ability to
evolve further in both directions.
Acknowledgements
Work
reported
herein
been
has
supported,
in
part,
Department of Transportation's Transportation System Center,
Force,
the
Space
and
Naval
Warfare
Development Center, and Citibank.
32
by
the
the
Air
Systems Command, the Rome Air
,
5.
1.
2.
REFERENCES
Appleton, D.S., "Very Large Projects," Datamation
January 1986.
Hsiao, D. K. and Madnick, S. E., "Database Machine Architecture
the Context of Information Technology Evolution," Proceedings
in
of the Third International Conference on VLDB pp. 63-84, October
6-8,
3.
,
1977.
Keen, P.W., Competing In Time:
Competitive Advantage, Ballinger,
Using
Telecommunications
for
1986.
4.
Madnick, S.E., Composite Information Systems - A
New Concept in Information Systems," CISR Working Paper # 35,
Sloan School of Management, MIT, 1981.
5.
Lipis, A., Electronic Banking, John Wiley and Sons,
6.
"Information Technology Changes the Way You
McFarlan,
F.W.,
Compete," Harvard Business Review, May-June 1984, pp. 98-103.
7.
Madnick,
S.
Information
1191-1199.
8.
Lam,
C.Y.
and
Computers and
"Trends
in
Utility,"
Vol.
Science,
185,
E.,
1985.
Computing:
March 1977,
The
pp.
"Evolution Towards
Strategic
and Wang,
Y.R.
Madnick,
S.E.
of
Very
Large Data Bases Through Composite
Applications
Sloan School of
Information Systems," Working Paper //1862-87,
Management, MIT, February 1987.
33
i^-953
058
V
Date Due
AUG 10
aw
Lib-26-67
MIT lISRARIf^
3
TDflD OD M
ebS 07M