V^^
HD28
.M414
DEWEY
an
"jFiA»l*3-'''
ALFRED
P.
WORKING PAPER
SLOAN SCHOOL OF MANAGEMENT
A CONCEPTUAL MODEL
FOR INTEGRATED AUTONOjMOUS PROCESSING;
AN INTERNATIONAL BANK'S EXPERIENCE
WITH LARGE DATABASES
William F. Frank
Stuart E. Madnick
Y. Richard VJanq
Revised
Seotember 1987
#VTP
1866-B7
MASSACHUSETTS
INSTITUTE OF TECHNOLOGY
50 MEMORIAL DRIVE
CAMBRIDGE, MASSACHUSETTS 02139
A CONCEPTUAL MODEL
FOR INTEGRATED AUTONOMOUS PROCESSING:
AN INTERNATIONAL RANK'S EXPERIENCE
WITH LARGE DATABASES
William F. Frank
Stuart E. Madnick
Y. Richard VJanq
Revised
Sentember 1987
#VTP
1866-87
To anoear in Proce edings of the International Confe r ence on Information
Systems December 1987.
,
A Conceptual Model
for Intesrated Autonomous Processing:
An International Bank's Experience
with Large Databases
William F. Frank
Integrated Information Systems Associates
Warren, Vermont
Stuart E. Madnick
Sloan School of Management
Massachusetts Institute of Technology
Y. Richard Wang
Department of Management Information Systems
College of Business and Public Administration
University of Arizona
Working Paper
1866-87
H.I.T.
Management,
of
School
Sloan
//
Report //CIS-87-04
Composite Information Systems Project
4
<|.^r.;
•
'
:\\\
M.IT. LIBRARIES
OCT 2 1 1987
RECBVED
.
TABLE OF CONTENTS
1
Banking Environment
1.1 Autonomy, Integration, and Evolution
1.2 Transaction Investigations
1.3 MIS Applications
Conceptual Model
Model Architecture
Autonomy Aspects of Model
Integration and Evolution Aspects of Model
Model Components
3. Implementation Experience
3.1 Summary of Results
3.2 System Design
3.3 Model Interpretation
3.4 Environmental and Systems Software Requirements
3.5 Development Histories
4. Concluding Remarks
5. References
2.
2.1
2.2
2.3
2.4
1
2
5
5
6
6
6
8
9
13
13
15
20
22
27
32
33
.
1.
BANKING ENVIRONMENT
The banking environment has experienced dramatic changes over the
past
and it continues to change at an accelerated pace.
decades
two
Competitive pressure is increasingly
industry
on
a
multitude
fronts.
of
imposed
being
There
banking
the
on
little to constrain
is
wholesale oriented institutions from crossing interstate barriers
encroaching
on
Competition is also crossing
local banks' territory.
countries
many
international frontiers; in
and
wholesale
international
banking is becoming much more aggressive as banks vie for the business
of multinational corporations.
technology
Information
1985]. For example,
[Lipis,
^
being
is
in
extensively
used
banking
in
1986 the value of computerized payments
House
processed by the New York Federal Reserve and New York Clearing
exceeded
often
critical
to
trillion
$1
keep
with
pace
in
a
day. Technology has been
single
volume
increased
the
of
financial
activities; payments processed through New York financial institutions
have
increased 50-fold in the past 20 years, to the point where every
four days an amount equal to the total annual US GNP is turned over.
During the 1990's, technological innovation
more
dominant
effect
on
financial
the
have
will
an
even
environment than
services
interest rate volatility. The development of sophisticated information
processing facilities
broader
range
of
are
enabling
institutions
products and services on
distributed
efficiency. Technologies such
as
machines
1977; Madnick,
[Hsiao
and Madnick,
by the industry to gain strategic
198A]
a
advantage
offer
to
a
much
global basis with great
systems
1977]
(Keen,
and
database
are being employed
1986;
McFarlan,
INTEGRATION, AND EVOLITION
1.1 AUTONO!fi',
This paper reviews and analyzes the development and deployment of
a
model
conceptual
for
The cultural
international bank.
non-integrated
autonomous
responsibility for getting
thus
job done is
itself
cope
with
constantly
the
satisfied
to
the
the
changing
it was recognized
designing
in
management)
is
funds transfer,
information
letter of credit,
autonomous ly
developed
Each product manager, in
general,
buy/modify
suitable application packages).
bank, this autonomy is critical since
responsible
for
has
complete
contract
freedom
(e.g.,
programmers,
In the culture of this
manager
is
held
solely
excuses such as "the data processing
his products:
people didn't do it right"
each
loans,
separate component
by
his hardware acquisition and software development choices
hire his own development staff, retain outside
or
which
in
an
autonomy, integration, and evolution.
personnel.
over
favored
distributed
and the distributed autonomous culture,
Each bank product (e.g.,
cash
bank
tradition
a
that three key goals reed to be
systems:
the
in
approach:
To
maximized.
always
environment
a
forces
major
a
level, and in which the independence of projects was
possible
lowest
integrated autonomous processing in
is not
acceptable. When information must be
exchanged, it was usually accomplished by "tape hand-offs", usually at
night, as depicted in Figure
1.
On the other hand, the needs for integrat ion have been increasing
rapidly both at the user level and database level. Since
had
its
own directly connected terminals,
user.':-
each
system
that required access
to multiple systems had to have multiple terminals in their office, or
walk to an area of the building that had
-
2
a
terminal tied to the system
.
/ MTZSk \
I
Terminals
and other
network
interfaces
External
Interface
External
Interface
Routines
Routines
Processing
Processing
Routines
Routines
Database
Database
Routines
Tape
Routines
"Hancj-off"
"Shadow"
"Original"
database
database
Database
LETTER OF CREDIT
LOAN
SYSTEM
SYSTEM
Figure
1
Independent Autonomous Systems
databases
needed. The tape hand-offs were used to create "shadow"
Since the shadow database diverges from
other's real databases.
each
of
the real database during the day, inconsistencies could result.
The problem of integration has been intensified by the
need
evolution in at least three areas: current products, new products,
technology.
new
the
As
tape
information
from
the
other
(e.g., cash management) are,
products
--
to
in
fact,
a
completely
produce
requirements
for
systems would be
new
a
dramatic
be
able
to
take
advantage
of
it
increase
Finally, to
tape hand-off information sharing.
maintain an efficient and cost-effective environment,
to
products
repackaging and combination of
expensive, time consuming, and impractical due to
in
systems.
hand-offs lead to processing complexities and do nor
address the need for up-to-date information. Many of the new
existing
ar.d
current products become more sophisticated,
there is need to acquire more
Increasing
for
is
important
new hardware components without
disrupting or discarding existing systems.
database
Traditional centralized
provide
integration,
a
The purpose of this paper is twofold:
1)
1987], that describes the architecture for an evolutionary,
integrated, and autonomous environment; and 2) report
the
experience
date in implementing this model by the institutional banking group
to
of
strategies
conceptual model, based upon [Lam and Madnick, 1981; Madnick
Vang,
and
system
but have limited capabilities for evolution and
reduce managerial autonomy.
present
management
a
major international bank.
five
During the past three years,
implemented
products/services
have
been
using this conceptual model. This paper focuses on two of
h
-
information
management
these systems: transaction investigations and
system (MIS) reporting.
1.2 TRANSACTION IN'VESTI CATIONS
Complex
international
financial
transactions performed in high
prone,
resulting
in
volume can be error
number
between customers' records and bank records.
discrepancies
inquire about the source
responsible
significant
a
of
reviewing
for
these
its
discrepencies
and
the
Customers
bank
is
records to establish what it did and
why, supplying these records to the customer, and making good
the
if
The bank's activity with respect to such an inquiry
bank is in error.
is
of
called an investigation.
are
Investigations
performed
They are usually
processing
transaction
the
job
with
systems,
of sizable staffs in a large bank.
the
aid
tracked
satisfaction,
to
allow
indicators
of
a
In
history, which
productivity,
customer
and the efficacy of the transaction processing systems.
The Historical Data Base (HDB), needed by the bank
investigations
the
on
microfiche, and printed reports.
addition, the investigation activity itself generates
should be
stored
data
of
and
to
support
these
other applications covering the previous 90 days,
must be able to hold at least 40 million records.
1.3 MIS APPLICATIONS
Each transaction
reports.
processing
system
generates
summary
various
Periodically, integrated MIS reporting for upper management
was accomplished by manually entering data from
a
variety
reports and other sources into an Apple computer spreadsheet.
of
these
Management
previously
integrated
presented
desired
new and more comprehensive MIS systems which
independently
In total,
data.
a
generated
database of at least
with over 250,000 additions per day, is needed to
and
inconsistently
12
million records,
hold
relevant
the
Furthermore, these new systems should facilitate the modelling
data.
of the effects of alternative decisions
(i.e., what-if analyses fed by
current real performance data).
2.
CONCEPTUAL MODEL
2.1 MODEL ARCHITECTURE
The model developed consists of seven major functional components
as
depicted in Figure
2.
These components are
into
five
with the integrating application-independent layers (external
layers,
interface, message control, data control, and
surrounding
separated
shared
data
resource)
application processing components (Madnick and Wang,
the
For this bank, these
1987].
separated
application
processing
components
are
into three classes of applications: transaction processing,
information processing, and administrative processing.
2.2 AUTONOMY ASPECTS OF MODEL
This architecture attempts to mediate the conflicts
goal
the
of
autonomy
except
layers,
themselves
to
between
the
and the goals of integration and evolution. All of
for
message
control
and
data
control,
lend
unlimited autonomy. Each product manager could acquire
and manage his own resources
including
-
6
-
1)
terminal/network
gateway
I
External
Interface
INPUT/
OUTPUT
Message Control
X
PROCESSING
Transaction
Processing
£
Information
Administrative
Processing
Support
Data Control
DATABASE
Shared
Data
Resource
Figure
2.
Conceptual Model
hardware,
application
2)
computers and software, and 3)
processing
database computers and software.
In the past,
were
bundled
as shown in Figure
together.
earlier, these three
1
In practice,
decisions
the primary concern for autonomy
involved the application processing, with
a
concern
lesser
the
for
database, and minimal concern for the external interface. It is in the
processing
application
manifest.
as
It
is
that
functionality
the
of
the
product is
important that enhancements and corrections, as
well
the initial development, be able to proceed with minimal needs for
coordination or delays due to the managers of other areas of the
bank
and other computer systems.
Given
the
architecture
Figure
in
2,
each manager has complete
control over his application processing system.
Furthermore, as
separate database systems as needed, or desired, can be created.
expected
that
initially
theoretically needed, due to
integrated
there
will
be
influence
the
of
many
databases
more
practice.
past
is
It
than
The
autonomous architecture provides access to these databases
by other applications as well as an evolutionary path
eventually
for
integrating these databases, as the needs for integration intensify.
2.3 INTEGRATIO.N AND EVOLUTION ASPECTS OF MODEL
There are several underlying concepts and components of the model
which address the issues of Integration and evolution.
Message
Control
the model. They are
coordinated.
application.
and Data Control perform unifying functions for
the
points
For example,
Furthermore,
which
at
in principle,
application
8
-
all
processing
will
be
any terminal can access any
subsystems
con
utilize
the
.
to manage and maintain data which is common to
Resource
Data
Shared
more than one component
Message Control and Data Control
The
entities.
components
other
both
are
single
conceptually
types of processing functions.
are
There may be many instances of each type (e.g.,
transaction
multiple
processing systems and multiple shared data resources).
l.U MODEL COMPONENTS
It
is
important to realize that the model components of Figure
They could be mapped to
are logically separate.
various
ways.
For instance,
actual
hardware
2
in
in the transaction investigation system,
each component resides on a separate processor; whereas,
in
the
MIS
system, many components residp. on a single processor.
2.A.1 External Interface
The external entities interfacing with these banking systems fall
into
3)
five categories:
1)
payment networks, 2) communication networks,
customer terminals, 4)
professional
workstations,
and
5)
other
intra-bank and/or inter-bank systems.
2.4.2 Message Control
Message
processing
coordinates the passage of messages between the
Control
components.
involves
This
routing,
translation,
sequencing, £md monitoring:
o
Routing
accepts
a
request
the
for
delivery
of
messages to
a
particular logical function and determines the appropriate physical
address to which the message should be delivered. Routing can
accommodate
thus
changes in the availability and location of functions.
-
9
-
Routing can also help coordinate requests for services that involve
several functions.
o Translation maps a
limited number of protocols from one standard
to
another.
determines
Sequencing
o
the
order
in
which
messages
are
to be
delivered to recipients on the basis of established priorities.
o
Monitoring determines the state of messages within the system at any
given time. Monitoring thus includes
integrity
of
message
a
from
the
time
the
responsibility
it
is
for
the
presented by one
component until it is accepted by another.
Transaction Processing
2. A. 3
Transaction Processing refers to the applications
the
customer's
retrieve and
update
balances).
The
amount
significant
a
These
instructions.
financial
sub-functions
of
execute
systems
data
Transaction
of
which
typically
(e.g.,
Processing
client
include
validation, risk management, accounting, and recording.
o Val idat ion functions are those which perform review of
instructions
to ensure that all information needed for processing is present and
that
for
the
data
"repaired"
is
elements.
Incomplete
by
either with
consistent with previously defined rules
information
augmenting
information
or
or
invalid
requests
may
be
clarifying the request information,
available
internally
or
from
external
sources such as the requestor.
o
Risk
Management
functions
those
are
which
verify
that
transactions being processed do not violate limits, conditions,
policies
regulatory
established
agencies.
by
the
customer,
these
Ideally,
10
-
the
or
the bank, or the various
functions
should
be
synchronized
where
this
with
is
processing
the
"after-the-fact"
feasible,
not
In some cases
of transactions.
Management
Risk
functions can be used to initiate corrective action,
o
Accounting
records
completed.
Accounting
cumulative
the
functions
impact
are
of
transactions
the
characterized
being
as
continuous over time, in contrast to the discrete events which take
place in most of the transaction processing functions. For example,
the banking environment, accounting takes place on two distinct
in
levels: customer accounting and organizational accounting.
2.^.4 Information Processing
Information Processing refers to all the subsystems that
analysis,
calculations,
or
restructuring
of
data
the
The sub-functions
consolidated financial statement).
perform
of
(e.g.,
Information
Processing include user interface, static reporting, ad hoc reporting,
and access to outside data resources.
Administrative Support
2. A. 5
Administrative Support provides facilities for the performance of
office
functions
by
administrative
or managerial personnel.
activity is required to maintain organization, procedural or
Example
information.
processing,
facilities
correspondence
files,
include
and
personal
mail,
word
controls.
The
electronic
inventory
This
sub- functions of Administrative Support provide
o Template Presentation
o
Editing
and
for the preparation of predefined documents.
Formatting
for
the
alteration
and
preparation of
documents and for automated checking of spelling and style,
o Mail Delivery and Storage for the transmission of documents
system to another.
11
from one
.
o Document Storage and Retrieval
for
maintaining
documents,
lists,
and passages from documents in an organization that allows
tables,
retrieval
the
on
basis
of
attributes
their
of
any
and
relationships
Functions
Control
o
to
monitor
administrative
activities
by
maintaining task related information including status, performance,
and activity data.
preparation
o Graphic Functions for the
of
diagrams,
flow
charts,
organization charts, and other illustrative communication tools.
2.A.6 Data Control
Data Control coordinates access, presentation, and the passage of
data
between
processing
functions and the Shared Data Resources. It
routes queries and updates to the appropriate component of the
Data
Resources,
Shared
security and priority functions, maintains
performs
concurrency control over the shared data, and returns responses to the
appropriate processing
Data
function.
Control
perform
must
the
following functions:
o
Security ensures that there is no unauthorized access to the Shared
Data Resource
and controls the view
permitted
data
the
of
to
different users,
o Presentation provides standard,
query
updates
and
data
flexible and simple means for making
definition
Control contains data manipulation
requests.
and
data
Therefore, Data
definition
language
processing functions.
o
Routing determines which segments of the shared data
access, and passes the
returns
the
request
to
those
segments,
a
request must
as
well
as
results of the requests to the appropriate processing
12
function. Alternate routing may be used if more than
one
copy
of
the data exists.
J
o
Sequencing
determines
the
priority
to ensure that
requests
of
response times for data requests are within acceptable limits,
o
Concurrency Control ensures that multiple, active
alter
the
requests
do
not
data so as to create an inconsistent state within
same
the Shared Data Resource.
l.h.l Shared Data Resources
Shared Data Resource is the component responsible for holding the
information common to one or
Although
this
activity
is
components
other
more
of
Model.
the
logically centralized in the Shared Data
Resource, it may contain multiple elements (storing different segments
of the shared data, or different organizations of
the
shared
data).
The Shared Data Resource performs two functions:
o
Information Management determines what information must actually be
stored
and
retrieved
transformations
to
satisfy
performs
request,
the
the
necessary to produce the required information, and
determines how the information
is
to be stored or retrieved.
o Storage Management determines physical
locations of data and
access
storage storage devices.
3.
IMPLEMENTATION EXPERIENCE
3.1 SUMMARY OF RESULTS
The
conceptual
model
Figure
of
2
has
provided
organizational guidelines for system development over the
years.
Two particular projects completed
-
13
-
in
that time,
last
general
three
including two
large databases (20 gigabytes and 1.5 gigabytes), will
technology
VAXCLUSTER
the ORACLE relational database management
and
system was used extensively to implement
portion
STRATEGIM
development
the
of
Most
language.
described.
be
one
of
of
of
other
the
conceptual
the
model.
A
applications used the
the
applications
were
either
programmed in the C language or used existing packages.
The goals which lead to the development of these two systems were
threefold
order
(in
importance):
of
applications for particular user
repository
further
and
dispatch
processing historical data; and
architecture
according to
which
in
a
groups;
MIS
point
provide
to
2)
data
of
central
a
for all banking transaction
application
to provide an
3)
effective
create
to
1)
various
was
kinds
partial order of increasing levels
systems
organized
abstraction
of
or
aggregation, so that higher levels of data would be created by batched
flows of data from lower levels.
The
were
design and implementation of the systems to meet these goals
greatly
considerations
affected
mentioned
functionality and
by
cultural
the
earlier.
performance
factors
business
and
Furthermore, experiences with the
characteristics
of
ORACLE
also
are
reported along the way.
The first and last of the goals (good applications and structured
aggregation)
have
been
largely achieved by the implementation.
second goal (shared historical
Technology
cautiously.
resource)
data
was
approaclied
The
very
makes this goal today more feasible than it
was in 198A.
Despite much skepticism about the
database
system
on
databases
as
K
-
performance
large
and
of
a
active
relational
as
these,
particularly on minicomputer technology, the applications
making
use
of these databases ultimately performed very well.
3.2 SYSTEM DESIGN
3.2.1 Application of the Conceptual Model
plans for new systems called for the gradual segregation
General
of
development
systems
efforts,
and
of
hardware
and
software
components, into specific classes of systems which would correspond to
components of the conceptual model.
important that this be done on an evolutionary basis since
It was
the inventory of e.xisting systems constitutes some 20 million lines of
There was (and is) insufficient business motivation to replace
code.
all
these working systems even to achieve such general goals
integration.
replaced,
with
Instead,
it
is
new
data
applications are built and old ones
intended that they be brought into greater
conformity
on an application-by-application basis.
This means
model,
the
as
as
that only those pieces of integrative components required to support
a
developing application may possibly be built.
In addition,
and
hardware
it was not
Instead,
it was
purchased commercially.
be
expected
At
Control
available.
software
and
Over time,
Data
it has
Control
that
components
these
the time the model was proposed
and accepted as the basis for new development
Message
integrative
required for Message Control and Data Control be custom
built for the bank.
would
practical that the
software
in
the
was
bank,
not
complete
commercially
increasingly become available. Thus, one
of the major values of the model has been positioning the bank for the
arrival of such new technology.
-
15
The following two sections describe the ways in which
of
the
conceptual
historical database
related
model
applied
were
system
to
ideas
development of the
the
transaction
for
the
investigation
and
the
system, first from the point of view of their functional
MIS
organization, and then
from
point
the
of
view
of
implementation
issues.
3.2.2 Role of
a
Historical Database
resource shared by
systems
database
historical
The
multiple
envisioned
was
as
providing
Transaction
applications:
a data
processing
would no longer have the responsibility of storing historical
data nor need to produce
the same
data
Instead,
all
for
a
variety of different summaries and views of
different
the
information
systems
systems.
transactions and proofed account balances of
completed
each type would simply be written to
processing
processing
would
a
Information
common database.
each extract the data they needed from the
historical database, and use that data independently from the uses put
to It by other systems.
The goals of such
of
transaction
a
historical database was to simplify the
work
processing systems, and more importantly, to simplify
inter-system data flows.
The current flows had
reached
a
level
of
complexity which were, in total, no longer known to any one person and
had
no discoverable principle of organization.
Of most urgency, dependencies
people several months to catalogue.
tape
They required several
on
hands-off between systems meant that each year more systems were
unable to complete their off-line
available,
especially
because
work
in
tlie
daily
time
periods
customers all ovrr the world demanded
that some systems be available on-line virtually all the
16
-
time.
The
TXN
Processing
Systems
Financial
Accounting
Figure
3.
Role of Historical
Database
to be provide by this new organization are shown in Figure
data-flows
3.
3.2.3 Structured Aggregation in MIS Systems
The
integrated
functional
HIS
discussed
benefits
in
addition
above,
was,
system,
providing
to
transaction
the
like
the
investigation system, intended to improve the overall organization
inter-system communications, by deciding in
of
systematic way which MIS
a
applications should communicate with which other applications.
to be accomplished by describing the information input
was
This
requirements
and
applications
in
output
information
the
desired
various
of
HIS
uniform way, and then identifying the lowest cost
a
(least transformation required) connections between such applications,
so that the output of some applications becomes the input to others.
Treating the output of each MIS application as
own
right,
regarding
and
primary
the
role
application (standard cost accounting) as
the result is
databases
partial order of aggregated
a
depend
ultimately
depicts the notion of
databases
a
a
on
database in
a
sort of MIS
this
of
its
data aggregation function,
databases
which
in
the raw historical data.
all
Figure A
partial order. The current collection
of
MIS
created consists of seven levels and the dependency diagram
barely fits on
large wall.
a
The general design of the MIS system is in fact to regard various
sets
of
tables
as
levels
aggregation,
of
created
by
specific
applications and supplying data for further applications. Although the
processing
is
quite
maintained
as
a
complex,
single
the
shared
fact
data
that
resource
all
has
simplified the operations of the MIS system components.
-
18
of
the data is
dramatically
Level 4
Aggregation
,
3.3 MODEL INTERPRETATION
"interpretation"
By
we
mean
components to hardware and software components.
information
For
instance,
processing
front-end
environments,
processors
model
of
The interpretation of
transaction
the model has varied considerably between the
and
mapping
functional
a
processing
and has varied over time.
now
are
distributed
being
geographically, which has caused some changes and extensions.
interpretation
The
provide
environment
the
investigation
system,
the
for
and
presented
model
the
of
development
TTie
layer,
since
the
was used to
the
transaction
later interface to the funds transfer
its
initial interpretation involved
system.
of
here
a
"null"
message
control
information processing applications did not
early
communicate with each other, and all screen management was done in the
same processor as the applications, although with
Only
when
later,
computers
and
capable
of
processors
applications
software
management
screen
communicating
via
software.
separate
running
on personal
datagrams
became commercially available, did
with
the
message
a
control layer become significant in information processing.
Tlie
in
five layers of the conceptual model are interpreted as
Figure
5:
(1)
an external
interface system consisting of terminal
controllers, wide area gateway boxes,
network
interfaces
a
terminal
to
for the application host machines,
host
of
the
application
software;
(3)
software
consists
message control, sharing
host-to-host Ethernet with data control; (^) data
of
ethernet
and the screen
management software on these hosts; (2) an application layer
only
shown
control
a
consisting
on the application hosts enabling communication with the
database processors over the host-to-host Ethernet, the communications
-
20
Terminal
?
Controllers
!
Network
Gateways
<
Terminal-to-Host Ethernet (lOMb/sec)
<
z
ec
Network
Interfaces
1
Application
Processors
(DEC VAX
<^
<
11/785)
Q.
C
2
<
Host-to-Hos! Ethernet (lOMb-sec)
co
o
< a
^/"
I/'
^ < (r
Z^ Z
^uOw
Database
Processors
(DEC VAX
8600.ORACLE)
t
UJ
U
oe
Host-to-Database
13
Ci
Bus (50Md sec)
o
>/\
<
t<
O
O
LU
intelligent
Storage
Controllers
<
X
en
i
Disks
Figure 5
Model interpretation
^
software on the front ends of the database processors, and
of
a
database management system, which includes query analysis and
the
concurrency control; and (5) shared data resources consisting
of
the
of the database management systems running on the database
ends
back
portion
processors, and the storage
systems
which
to
they
connected.
are
These components are described in more detail below.
3. A
ENTIRON.^NTAL AND SYSTEMS SOFTWARE REQUIREMENTS
bank's
The
development
emphasize individual projects
policies
personnel
that are as small as possible, both in number of
Ideally, projects should take less than
frame.
more than
5
year and require no
a
developers [Appleton, 1986].
The environmental software chosen to
applications
processing
time
and
applications
processing
transaction
from
differed
support
these
information
that chosen later to support
(such
as
the
transfer
funds
The major reasons for this difference are the technical and
system).
differences
performance
between
transaction
the
processing
and
information processing systems.
The
information
processing systems communicate with the outside
world largely interactively; their databases are the
must
which
addition,
this
components
fully recoverable, and the tolerable time to recovery
be
(While
may be as long as several hours.
systems
only
is
typically
information
a
for
transaction
processing
matter of minutes or even seconds).
processing
systems
are
highly
In
fluid:
requirements change from month to month.
The
primary
software required on the information
environmental
processing side were flexible screen managers and database
-
22
management
well
systems,
as
addition,
these
host-to-host
as
systems
themselves
lend
software.
In
prototyping
and
communications
to
non-procedural languages.
3.^.1 Operating Environment
hardware and software chosen to realize these components was
The
overwhelmingly based on the DEC VAX. The group's processors
include
thirty
about
VAXes
currently
two IBM mainframes. While the IBMs
and
exchange messages with the VAX systems, the two environments
are
not
Integrated into the same conceptual model: the IBM systems,
currently
developed earlier, perform high volume batch
running
work
purchased
packages which provide end-to-end support for these applications.
for this preponderance of VAXes were the experience
reasons
The
base of available developers and support
VAX
favoring
earlier,
as
the
computers
for
well
analysis
indicated
capable of handling projected workloads, and
were
systems
as
separate
preliminary
Thus in cases where
separate jobs.
that
noted
forces,
cultural
personnel
where specialized software was not immediately available off the shelf
for MVS systems, VAXes have invariably been chosen.
In
case
the
of
availability on the VAX of
tracking
was
since here the size of
As
a
a
system,
the
software package to support investigation
a
the
might
database
have
indicated
that
a
better prima facia choice.
result of this choice, VAXCLUSTER technology has been very
widely used in
a
variety of ways.
complexities have been omitted.
processors
investigation
most important factor in the selection of the VAX,
the
mainframe would be
transaction
the
are
To keep Figure
5
simple,
certain
For example, some of the applications
also connected to the Computer Inter-connect (CI) Bus
23
-
to allow them to take the place of
Furthermore,
for
increased
defective
a
database
processor.
each Ethernet shown has an
reliability,
additional backup Ethernet.
3.A.2 External Interface
system
All communications with the ultimate
data
Certain
based.
downloaded
are
manipulation, graphics, and report
communications
PC-to-host
physically
accomplished
to
users
PCs
terminal
is
spreadsheet
for
Terminal-to-host
printing.
and
(part of the External Interface system) is
by
Ethernet
an
fact,
(In
one
two:
for
production systems and another for development systems which serves as
backup for the production network).
Bridge
controllers (local terminal controllers, dial up-systems,
connection
and X.25 gateway controllers, which provides a
terminals
via
an
remote
to
in-house worldwide network) are used to serve both
They communicate with each other via
the terminals and the computers.
Sun Workstations are used on this network as network control and
XNS.
configuration management devises.
Screens are managed from
Viking
VAX
the
machines,
application
using
Forms Management software, which in turn communicates with the
application software.
3.A.3 Applications
The
transaction
application
investigation
software
largely
consists of calls to the screen management and the database management
systems.
In addition,
it performs any processing required to make the
necessary transformations between these
purchased
to
support
two
systems.
investigations,
these
considerably more than this.
-
24
-
in
The
package
actuality,
does
3.A.4 Data Control
The Data Control level consists largely
general
and
software
hardware.
purpose
Control could not be implemented, owing
to
management
of
database
All
the features of Data
the
lack
commercial
of
distributed database management systems at the time of implementation.
addition
In
to
development
productivity,
concerned performance, because of the large
as
criteria were:
important
Other
1)
relatively
2)
widely
databases.
the
of
preferences for relational systems
having the longest future and offering
gains,
size
issues
major
the
the
systems,
used
greatest
and
productivity
adherence
3)
to
standards and ad-hoc standards when possible, such as CODASYL or SQL.
the
At the start of
project,
there
was
mostly
considerable,
unsolicited, expression of opinion and concern that relational systems
poorly" and were therefore unsuitable for large databases.
"performed
Of course it is meaningless to talk
except
with
performance.
database
respect
good
performance
bad
or
to a specific use and to specific parameters of
In this case, both the historical database and
would
be written almost solely in batch mode.
volume of such writes was quite
night),
about
large
(300,000
512
the
MIS
Although the
byte
rows
per
the writes were all appends, no update; reads and writes were
never expected simultaneously against the same tables; all reads
were
predictable since very little access to the databases would be ad-hoc;
and
the
database
could
be
designed
in
such
a
way that most read
requests would require no joins between tables.
Performance projections were established by study of the
of
results
benchmarks performed by other institutions, by review of published
performance analyses, including
a
study
25
-
by
the
National
Bureau
of
.
Standards, and by analysis of the likely effects of design features of
the systems.
It
quickly
was
established
available
commercially
the
that
systems based on pointer chains (which happened to be coextensive with
nightly
systems)
CODASYL
the available
number
of
batch
while the systems based
permit
would
consideration)
even within
appends
on
incapable
were
indexes
(the
performing
of
period of 2A hours,
a
relational
systems
batch appends within
these
the
a
under
number of
hours
The two systems considered most seriously were INGRES and ORACLE.
Benchmarks
well
as
performance
in
opinion
general
as
performing
unlocked
suggested
that
ORACLE'S
against single tables in
reads
large databases was significantly superior to that of INGRES.
linear (and almost flat) increase in response times to such
exhibited
database
simple queries as the size of the
superior
ORACLE
to
ORACLE
INGRES
increased.
in the performance of complex joins,
was
and offered
superior fourth generation development tools.
For
appeared
the
to
type
of
production
be a better choice,
of
simultaneous
investigation system.
ORACLE
although neither system would be, in
its then current version for VAX hardware,
number
envisioned,
applications
capable
ultimately
users
of
handling
projected
for
the
the
Special design approaches were used to overcome
these problems. Furthermore, one could be confident that both hardware
and software would get even faster soon.
3.4.5 Shared Data Resource
The data resource used in all the VAXCLUSTER systems is supported
by DEC cluster storage, which consists of the very high speed
26
-
CI
Bus
.
(70
megabits
per
second)
controllers
number of intelligent
devices)
in
such
connecting
,
a
number of processors and
(currently
a
of
total
a
such
16
way that any processor can communicate with any
Controllers are
controller.
a
in
turn connected in pairs to dual ported
disks, allowing full redundancy of hardware components.
within the cluster, when
For instance,
processor fails, its work can be taken over
a
by another processor which has immediate access to
same
the
of
all
disks
3.5 DE\TLOPMENT HISTORIES
3.5.1 Changes in the Plans
In developing these systems,
certain
features
culture
bank
the
of
became clear that the strength of
it
was
greater
even
than
anticipated:
independence and competitiveness between managers, and
predilection
for
achieving
Integration
actually
called for.
In
tangible
fast
needed
be
to
building and supporting stand-alone applications, each
responsibility
the
application
and each group has
to users,
total
on cooperation from other groups, not
Only
a
Investigation
delivering
an
strong aversion to relying
equally
responsible
for
some
portion of the transaction processing data (that involved
had
was
application.
data from all the
aggregation
a
for
.
with funds transfers)
already
to
smaller than the original plans
development group has
deliverable
Steps
results.
a
required
required
processing
capability
by
support
the
needs
of
the
The MIS system, however, required summary
transaction
the
to
the
of
MIS
-
27
systems.
providing
the
applications,
systems
These
lowest
since
level
they
of
were
,
supplying
data
this
to current applications.
A
coordinated effort
would therefore provide theoretical future benefits, while
increasing
current development time, cost, and risk.
As
result,
a
investigation
transaction
the
HDB
/
and
MIS
projects, which were started at approximately the same time, wound
going
each
separate
its
aggregating data from
the
up
That is, instead of extracting and
way.
system
own
processing
and
Projections called for at least 100 simultaneous users using
the
aggregated
data
HIS
receives
its
partially
the
HDB,
transaction
from
feeds
financial accounting systems, as shown in Figure
6.
3.5.2 Transaction Investigation System (HDB)
investigation
system
and
read only access to the historical
having
database, with maximum rates of
database request each five seconds.
1
The applications and the database are supported on
as
earlier
shown
Figure
in
A VAX 8600 is used to support the
5.
database, while the investigation application
11/785
"front
end"
which
machines,
runs
communicate
machine via DECNET and the VMS mailbox facility.
application constructs a query in the form of
of parameters.
VAXCLUSTER
a
a
on
multiple
VAX
with the database
The
investigation
"query type' and
a set
The historical database system translates this into an
SQL query, and returns the response table to the application.
The
configuration has grown to the two front-end ll/785s and the
8600 "data base processor" shown in Figure 5, while
through
two major new releases.
performance
evolutionary
have
also
capability
been
ORACLE
has
gone
Several changes in design to improve
made
over
the
years.
The
modular
of the conceptual model architecture greatly
facilitated all of these changes.
28
-
Financial
Historical
Accounting
Database
T
MIS
Database
Investigation
System
Figure
6.
Current Data Flows
The initial installed production system (on one VAX 11/780
end
VAX
one
and
11/765
front
back end) supported 30 users with database
query response times of 90 to 120 seconds, meaning that the decoupling
of queries and interaction was essential to the acceptability
of
the
The current software and hardware supports 160 simultaneous
system.
users with database
automation,
response
search
each
times
for
of
to
5
historical
data
Before
seconds.
7
required at least
15
minutes of an investigator's time.
The full cycle of
training,
and
development,
design,
installation,
testing,
live operation of the transaction investigation system
programmer
required about six months, with one
responsible
for
the
software development.
A
number
significant
small problems, and a few large ones,
of
these
problems
by careful up-front studies.
However,
were discovered in the course of development. Mary of
could
have
been
anticipated
such studies would have required as much time as building
did.
the
system
Without the flexible and powerful environment used, the same
sorts of unanticipated
development
problems
disasters,
as
would
often
they
have
had
been
likely
to
cause
done in the past in the
experimental culture of the bank.
For example, it was discovered during the course
that
the ORACLE "money" data type would not hold large enough amounts
It was
for the needs of a very large bank.
tape
also discovered
that
the
recovery of very large tables would take much too long, and that
mirroring was not economically justifiable, while at
the
development
of
re-indexing
of
very
large
tables
was
taking
original analysis counted only the time required to
30
-
same
the
time,
too long (the
index
the
newly
appended
data
each
day,
not
the entire 20 gigabyte database.)
solve this problem, tables were partitioned by dates, and
To
application
code was written to permit searches across ranges of dates.
3.5.3 HIS Systems
initial
The
phase
of MIS system development and implementation
also required about six months, with three contract developers and one
About one third of this time was devoted to learning to read
manager.
correctly the tapes from the various
non-integrated
sources
of
MIS
and developing programs to map these tapes to relational tables
data,
without repeating
groups,
multiple
record
types,
variable
length
records, etc.
innovation of the system was the virtual elimination of
major
A
the printing of MIS reports: particular pages of reports
of
interest
to individual managers are viewed on terminals, and printed locally if
so desired.
The
system had intense advocates in the HIS department, who
MIS
specified e.xactly what they wanted to see in the system.
common
the
in
bank
delivered to them.
enriched,
with
for
The
it
is
this
is
more
users to ignore new systems until they are
system
constantly
been
has
the MIS group developing
use of SQL and ORACLE development tools
fact,
It
enhanced
and
great deal of skill at the
a
for
applications.
small
In
system is not called the MIS system by most of the users,
called the ORACLE system.
The summary data from this system began to be
planning
sessions,
which
to
lead
forecasting and analytic modelling
creation
of
a
strategic
MIS
the
desiro
purposes.
used
in
business
to use the data for
The
result
was
the
system, using the STRATAGEM modelling
31
which
language,
applications
and
analytic
of
receives
data
from
some
strategic
purposes,
the
highest
level
The use of this system for
ORACLE MIS system.
the
of
rather
purely
than
reporting
purposes, has grown slowly but steadily.
U.
CO.NCLTOING REMARKS
conceptual
The
evolutionary
movement
The
blueprint.
integration
complete
described
model
from
autonomy
complete
to
and will probably never occur in this
slow
is
in this paper has served as an
organization, nor any organization realistically.
By separating the
control,
and
external
interfaces,
message
control,
data
the database components from the application processing
components, the approach presented here provides for high
integration while preserving significant autonomy
-
degrees
of
and the ability to
evolve further in both directions.
Acknowledgements
Work
reported
herein
been
has
supported,
in
part,
Department of Transportation's Transportation System Center,
Force,
the
Space
and
Naval
Warfare
Development Center, and Citibank.
32
by
the
the
Air
Systems Command, the Rome Air
,
5.
1.
,
REFERENCES
Appleton, D.S., "Very Large Projects," Datamation
January 1986.
2.
Hsiao, D. K. and Madnick, S. £., "Database Machine Architecture
the Context of Information Technology Evolution," Proceedings
in
of the TTiird International Conference on VLDB pp. 63-8'i, October
6-8, 1977.
3.
Using
Keen, P.W., Competing In Time:
Competitive Advantage, Ballinger, 1986.
4.
Madnick, S.E., Composite Information Systems - A
New Concept in Information Systems," CISR Working Paper # 35,
Sloan School of Management, MIT, 1981.
5.
Lipis
6.
"Information Technology Changes the Way You
McFarlan, F.W.
Compete," Harvard Business Review, May-June 1984, pp. 98-103.
7.
Madnick,
S.
Information
1191-1199.
8.
C.Y.
Lam,
,
Telecommunications
for
and
A., Electronic Banking,
John Wiley and Sons,
1985.
,
Computers and
"Trends
in
Utility,"
Vol.
Science,
185,
E.,
Computing:
March 1977,
The
pp.
"Evolution Towards
Madnick,
and Wang,
Y.R.
Strategic
S.E.
Applications
Very
Large Data Bases Through Composite
of
Information Systems," Working Paper //1862-87,
Sloan School of
Management, MIT, February 1987.
33
-
k 53 k
k.
\
Date Due
ikfi)<^ '^^
JUL J4.
I-ib-2fi-67
I
3 TDflD
Hill
mil
111
DOS 131 D3T