EVALUATING TIHE-SHARED COMPUTIZR USAGE'>
by
Michael M. Gold
19S-65
May,
^o
1365
MASSACHUSETTS
INSTITUTE OF TECHNOLOGY
50 MEMORIAL DRIVE
CAMBRIDGE, MASSACHUSETTS 02139
'
•
ImaSS. INST. TECH.
JUN
EVALUATING
T ME
I
28
1971
-SHARED COiMPUTlIR USAGE'>
by
Michael
198-66
f>'u
Gold
May,
1355
Alfred P. Sloan School of MancEoment
Massachusetts l-nstltute of Tcchnolojiy
Cambridge, Massachusetts
Professors
reported herein Is beng conducted under
zenbaum.
V.'el
end
Josepli
Marquis,
Donald
G.
Donald C. Carroll/
Foundation
Ford
by
the
part)
(In
Support Is being provided
and Project MAC, an M.I.T. research proeram spo:!sored by the
Department of Defense,
Advanced Research Projects Agency,
Number
Contract
Research
Naval
of
under
Office
In
part
Is
or
In
v/hole
Reproduction
N0nr-til02(01)
Government.
permitted for any purpose of the United States
*
Woric
.
PAGE
2
Pi.
.'clopment of
to
1
ti
me-shered computer
syste=T-:s
major technical and philosophic changes
cc
puter field during this decade. 1/
ho
3
1/
wit:-, in
has
the
Computer manuif acturers
available both computer hardware
software
^\-\(\
specially designed for time-sharing usage.
systems
The Adv,anced
Research Projects Agency (ARPA) of the Pentagon
given
h5:i
Increasing amounts of support to M.KT. and other ^^.-ademic
(and non-profit) communities for research
into
t
i
The Joint Computer Confei'ences sponsored by AF PS
I
devoted an Increasing number of sessions at their
to
i
^/
has
r.cetlngs
tims-shar ng.
i
•
The primary concern voiced
published literature appears
the design and development of
to
these meetings
in
^\^
nd
r-
the
center around proL-e;ns of
the
ti
me- slir^red
system -- little or no interest seems to
for
-shar ng.
mu-
hr.ve
comp-ter
been generated
evaluation of these systems as to their usefulness
for present or future users.
The need for such
a
formal
usefulness of time -sharing
is
investigation into
hi ghl
I
gli tr.rl
of
Information concerning the value of the
of
time-sharing computer systems
v>'hich
by:
(1)
the
:;ie
lack
rJ.'iM*iLflI.iXLsJtjjr-.5.
mc'!;es
these systems
differ from the more traditional ba tch-pi'cccr.s ng computer
I
systems; and (2) uncertainty as to the urcfulncrs o?
time-sharing systems or the premium (If any)
tliat
mtist be
paid for this service. 1l/ This proposed research will
evaluate time-shared computer system usage by means of
a
PAGE
;
-rn
rf
^
c
b^T'h
i
nv
ostl sat Ion
:/
tiie
5
Into the relationship between user
behavloi*/
and Cittltudes and the characteristics
time-shared computer systems and the comparable
nventlonal computer systems.
PAGE
R,
M.
"The notion of machine-aided
Fano has stated/
cognition implies an intimate collaboration
user and
Q?
a
a
computer in
problem,
betv^/een
a
human
real-=time dialogue on the solution
a
which the two parties contribute their best
in
Others with considerable
capabilities.
.
experience
this area have also added
in
k
".
.
(16)
their support to
the
development of interactive time-shared computer systems to
aid In this man-machine col labora
tioiio
J./
from
Indeed,
a
reading of the published literature concerning both
maclii
ne-a dcd cognition and time-shoring, one gets the
i
impression that there
use of such systems for
a
human user and
almost unanimous agreement that the
is
tills
'intimate collaboration between
computer' produces far more pi'oductive,
a
efficient, and effective results than could be obtained with
any other type of system.
major developments
believe that they
in
Those persons responsible for
the field of
v/ill,
time-shared systems
among other things;
.open up
Improve the ability to program.
several new forms of computer usage 6,/
.
increase productivity of 'computer catalyzed
research' that results from close man-machine
Interface.
2/
.
.
Unfortunately, no systematic studies arc available to
support the above statements.
There appears to be an
emotional commitment to these systems rather than
investigation of existing conditions.
low priority level.
in
fact,
formal
Most of the people in
the field have apparently relegated formal
a
a
investigation to
the first published
PAGE
ref>^ren'~e
ui..
:
.
5
problem (and one which did not appear
to this
miost four years after work began
the field)
in
quBtloned v;hether:
The promoter-scientists of time-sharing are systematically, scientifically, and economically
expl or ng it ; r,r whet;iier t ime-shar
js ROo d
w hat purposes
,or bad and f
All they seem
to be concer
v/ith is, which kind to design,
build, or bu;
pj (underlining added)
i
i
'
In
addition, at
n;:^,
.
.
tlic:
October 1965 meeting of the Los Angeles
ACM chapter three advocates of time-sharing conceded the
absence of any comparative evaluation of time-shared systems
and batch"process ng systems
I
9_/
Some of those persons developing time-shared computer
systems have attacked proposals for formal
Into the value of
investigation
time-sharing as both unwarranted and
They advance the following reasons for their
incorrect.
attitude: first,
time-shared systems are so newly developed
that the users do not yet know how to mal;e use of these
systems; second, present time-shared systems are
experimental
in
nature and sliould not be advanced as
representative of such systems of the future; and third,
Is
alleged
tliat
it
time-sharing applications are qualitatively
different than batch-processing applications and therefore
cannot be compared with them.
There
is
no doubt
that these statements are
substantially correct •- time-shared systems are new,
developing, and different than the more traditional
batch-processing systems.
true
However,
It
is
not necessarily
that the stated conclusions follow from
the
facts.
At
PAGE
to
point out that although some
ng applications will
have qualitative differences
this pointy
tlme-'i^har
I
it
must suffice
when compared with batch-processing applications^
majority of the applications are expected
with both types of systems.
v/ho
Jcnow
In
addition^
to
6
tlio
be coinpotlble
there are people
how to use these systems, and who have enough
knovHedge of the characteristics of future eppl en t ions,
I
design an Investigation incorporating at least some of
These aspects
v.'I
1
1
be
discussed
In
detail
belov.'.
to
tliem.
PACK
of Evaluatin g
.
7
r
Interactiv e System Usa ge
\^__
The Interactive time-shored systems arc described as
'more efficient'^
'faster'/
termS/ as used/
'more productive'.
and
are both ambiguous and not amenable to
measurement unless made more preci?c.
could refer to the total elapsed
problem to its completion/ but
total
These
I.Q./
For example/
it
amount of the man's time,
the
froin
t
'.
a
the
to
amount of the
otal
th
start of
also refer
d
'faster'
computer's time/ or even the computer's response time.
problem
of
v.'hich
Is
evaluator can have no notion
that the potential
meaning of 'faster'
The
relevant
is
the persons
to
using the term.
As another example of
investigation of
use
cf
an
ojjji
interactive
this problem/
f^
ime-sliar ng system for
p.
t
i
for
1.1/
in
However/
this
six- fold
(When compared with the
the same problem done on a non-i
system).
a
is
met
rite
acceptable re
Inefficiency' of interactive
t ime-£.lit'M
i
nary
prel
measure of 'efficiency'
v;orking FORTRAN program resulted
computer time used
a
i
that
cing
ve computer
'evidence of the
ng usage only
agreement between the relevant cvaluatorn
above
is
the one valid measure of
if
tliot
tlic
determination of whom the relevant evaluators arc.
It
that any group formed solely of programmers/
as evaluators
to
those persons
to
the
'efficiency'.
Another problem that appears here concerns
the computer
in
lime required
Is
or people paying for
a
i.iciease
there
doubtful
the
is
users/
system v/ould be acceptable
whom an evaluation might
PAGE
be addressed.
-- mixed groups would doubtless find
8
it
difficult to agree on acceptable measures.
The methodological
differences between computer
applications using interactive time-shared systems and the
more traditional computer systems result
For example,
problems.
in
further
problem on
the programming of a
these two types of systems can be expected to result in
program
different allocation of time for problem definition,
writing,
statement declaration, debugging, editing,
compiling,
time to complete the problem.
addition,
In
en evaluation could be furtlier complicated
by attempting to compare
is
v/fiich
reacqua ntance
think time,
analysis, waste time^
and total
tima,
a
two systems by using
the
not fully defined on the traditional
a
problem
system.
This
could result from an application v/hich uses features of an
Interactive time-shared system as an integral
For example,
tasi'v.
a
in
tas!<
v/h
i
ch
computer v/ould converse frequently
this category because
tyiv:?
Imprec
t
An
Is
a
the
i
v/i
th
the
each other fits
time-shared system.
of worl^ could be done on
system,
the user and
into
depends greatly on the rapid
it
response available with
feature of the
large amount of
a
traditional
Although this
ba tch- process
interaction required m.akes
i
ng
it
cal
integral
factor
in
this
lack of formal
investigation
probably prompted by the (usually found) electrical
engineering and computer science background and orientation
of
those persons responsible for the design and development
PAGE
;f
;,ac;
Interactive time-shared systems.
i-.'-'C
;
behavioral
Few,
any,
hove
science orientation or the training
necessary for deslcnlng and conducting formal
to evaluate
If
Investigation
man-machine interactive aspects of the usage of
these systems.
It
would thus appear that both the commitment to
time-sharing concept and the
laclc
of
a
behavioral
tlie
science
orientation of the designers and developers of such systems
are the primary factors responsible for this
lack of
scientific investigation into the supposed advantages of
Interactive time-sharing.
9
PAGE 10
Sc:iipjg._ /iM.
Pur P.Q-S^
The above suggests four possible sets of variables
relevant to an investigation of this area/
wlilch:
(1)
are characteristic of
study;
(2)
concern the attitudes
systems;
(3)
namely those
the computer systems under
of
the users of
these
are measures of the user's behavior and usage
of the computer system; and
('})
are measures of the results
(output) produced by the man-computer system.
FIffure,
J
ICompui
Syst(
User
I
\/
I
\
2,
Char;.;,
hQristlQ
_
I
i
s User Behavior
I
I
.
.J
Usage
Measures
Figure
2
I
Evaluated
Performance
Measu res
..
displays the conjectured relationships between
these four sets of variables.
More specifically,
preliminary experimentation and the viewed literature
provides support for the following major conjecture:
a result of the characteristics of a time-shared
computer system, the user of such a system will have
more favorable attitudes toward both the use of this
computer system and the results he produces wliile using
it than
he would if using a traditional computer
system.
Both the characteristics and the ensuing more
favorable user attitude will result In the user
producing results faster and more efficiently than he
could If using the traditional computer system.
Finally, the results produced by the user exhibit a
quantitatively higher performance level than the
results produced if a traditional computer system had
been used.
As
I
L
PAG£ 11
Before discussing each of the above sets of variables in
detail/
shoLild be noted
it
that
the cor.jec tiTe specifically
refrains from stating that either computer system
proven 'better'
only
if
than
'better'
v/as
the other "-
interpreted by eac
However/ each
exactly the same way.
values and tre.de-offs
tliic
to
ii
i
s
1
be
1
bo attempted
coi'lc!
every user in
nd
'
v/i
sslgns different
l!:
requircmen
for speed of
solution, cost/ access speed/ and system size for each type
The scope of this study will
of application.
limited
be
Investigating the usage of both time-shsred end the
traditional computer systems and providing
to
tnore
means of
a
evaluating the use of either type of sys tcru (or some
combination of each) over
a
range
Oi'
applications.
£omPu.tiir_S^sj;.eni £.h3.ra.c ie.ni s.tlci3.
The major
independent vari abler- to be conr dcred in
!
this research are those which rcprosciit the dii
between Interactive time-shared
more tradi
t
ionej
conveniently categorized as those
Th:;-"
wiilcli
v;;r!al
rciiilt
differing feedback delays; and (2) c'iffcrinfi
Interaction between the
addition/
it
liser
and
tlic.
an
be
'
/i
(1)
f
o^
c!
Cviivutcr
alleged that proper itc
is
the
ryf-tcmc
comrti'icr
computer systems.
-iccr.
r;
'erne
In
bo made of these
characteristics of interactive timo-sIiCrcu r-ystcmn to
produce qualitatively different rcsponr-e to
The elapsed
his program to
time
tlr..
tircr.
the user must wait from submission of
the return of
results
is
a
measure of the
PAGE 12
feedback
Interactive
(^r-lns/_.
tl
me-slicr ng systems would be
i
expected to have normal feedback delays measurable
systems would measure this in
seconds while the traditional
The decree of
hours or days.
in
1
between the user
v ^•^'•pc,t..i.,on
and the computer can be measured by both the delay involved
before
request for service
a
difficulties
a
user encounters in making one extra computer
a
Althouch admittedly
run.
recognized and the
is
i?
gross measure^ we shall
computer system as Interactive
Input
response to
the
a
normal
expected rapidly enough that the user does not turn
is
work while waiting for the response
to otiier
The rapid
^12/
if
consider
interaction
betv.'een
input.
the computer
the user and
large time-shared computer allows the
available only with
a
Implementation of
programming system
a
his
to
which the user can
in
take an active part In the computing-problem solving process
--
tlie
U^en
AtJiIiud.e.s.
As
of
man-machine symbiosis so often discussed,
a
result of these differences
the computer systems under
that a user's attitude
oi:d
the results
differ.
In
p:
r,
nvest igat on^
i
botIi
is
expected
the computer system used
it
is
expected the rapid feedback
,aring system will
ti.
it
while using this system will
''iced
parti
feature of the
toward
i
the characteristics
in
increase
a
user's
fulness of both the computer system
evaluation of the
used and the results of any given interaction with the
computer system.
In
addition,
correct errors and re-run
a
the ability
to
Immediately
program available with
PAGE 13
:ln
i
ng should result
in
smaller change in
a
level of satisfaction after an unsuccessful
than if the user had used a
f'or
the same reason.
It
is
a
user's
computer run
traditional computer system,
expected that
a
time-sharing
user's estimate of progress toward completion of the project
would be less affected by an unsuccessful
estimate of
a
user of
than would the
run
traditional computer system.
a
The user of a time-sharing system v'll rate
and
the usefulness of both the computer sy:
i/i
the results of a computer run higher t'
the user of a traditional system.
(1)
1
1
The ne/7;ative influence of an unsuccessful
be stronger in a
computer run on a user v;i
batch-processing than a time-sharing
envi ronment
(2)
1
lJ.Siir_Bj£hiivloj: a.nd. iisa.g£.
lle^su.r£.s
P.ejlfG.i'OiaiLce
When evaluating usap:e of
1
man-computer system^, we are
a
concerned with both the amount of elapsed time required
obtain
a
solution to
a
problem and
this solution could be ob tal ned
.
tlie
to
efficiency with which
Experience with both
j_3/
time-sharing and the more traditional computer usage leads
us
to expect
tlie
faster fcedbacJi available with
time-sharing system will
requiring
a
result in
a
a
problem solution
shorter elapsed time for completion.
The man- computer systems under study will also be
evaluated by measuring both components of the cost
efficiency and the methods of usage of the two systems.
cost efficiency
Is
The
determined by calculating the cost of
both the man's time and the computer's processing time for
each system.
We would expect that usage of
a
time-sharing
PAGE 14
system will
re;
higher cost
o:
will depend up.
from
it
in
lower cost of the man's time and
a
computer's time.
the balance between
The aq s
t
.ef f
the savings
i
ci
a
encv
resulting
reduced number of man-hours and both the increased
a
number of hours of computer processing time consumed and the
higher cost of time-sharing usage.
The method of usage is
a
measure of the degree to which
the user relegated programmable problem solving
computer system»
i
the
result of the user's attitude
As a
concerning the access
to
I.)
i
we would expect the user
1
i
ty and
to
usesbility of time-sharing,
relegate
a
larger number and
percentage of the programmable tasks, such as checking for
errors,
to
the computer while concentrating his attention on
non-programmable tasks, such as analysis of logical errors
and deci slon-malcing.
usage
Is
This does not really mean that such
more or less efficient than usage of
computer system.
different
Rather it indicates differential modes of
behavior while using these systems -- behavior
influence the final
ilQJiig/:
a
v.'liich
may
results produced.
The use of
a time-sharing computer system
result in a task solution sooner than the
solution could be obtained with a
batch-processing computer system.
(1)
will
The use of a time-sharing computer system
result in a greater amount of computer
processing (and higher computer processing
cost) tiian corresponding usage of a
batch-processing computer system.
However,
this usage of a time-sharing computer system
v;ill require fewer man-hours than for a
corresponding ba tcli-process ng computer system.
(2)
will
i
PAGE 15
The differential user attitudes toward the
two types of computer systems will result in
greater use of the time-shared system -however, this greater use will result, in part,
from a greater reliance, by the user, on the
computer for routine tasks.
(3)
£xl.ejin^l_E^alu^tioa
The purpose of
Is
to
af_Pje.t*£oj:ma.n£.e
evaluation of performance
the external
make available
a
measure of the output of the
man- computer system which
is
arrived at independently of the
user's methods or the computer system used.
this objective both
the results
produced by the user on the
computer system and an evaluation (by
of the user's understanding of
Implementation of
a
To accomplish
competent evaluator)
a
the problem and
solution will bo used to judge
performance.
The published literature suggests that time-sharing
usage will: "improve the ability to program"; "open up
several
now forms of computer usage"; and "increase (the)
productivity of 'cotnputer catalyzed research,..".
preliminary experimentation
it
appears that.
In
From
fact,
the
use of time-sharing .does allow the user to produce 'better
results' "" If the task In question
and produce
a
decision-rule,
system will produce
to
a
produce
a
a
is
to
the user of
analyze
the
better decision rulr.
workxing computer program,
problem
a
time-sharing
If
task
the
the user will
Is
produce
program In which results are more often correct, have done
more checking of inconsistencies
completely documented than
a
in
the data,
or is more
program produced by
batch-processing computer system.
a
user of
a
PAGE 15
The results produced by the user of the
tinie-sliorl ng system will exhibit a higher level
of perfcrmance than the user of a traditional
computer system.
PAGE
Expe rimental
17
DosUn
l'U.r(aducI.ian
The selection of an appropriate research setting,
experimental
tasi;,
and experimental
every investigation of
research
is
to
tli
i
type.
s
subjects
important in
is
Since one purpose of thhs
provide answers to questions concerninj;^ the
value of time-sharing usage,
the research settings and
selected should have relevance for both
'
rea -v;or d
1
taslcs
'
1
programmers and managers in terms of their potential uses of
time-sharing systems.
be conducted with
In
subjects
addition,
v/ho
the experimentation must
are competent users of these
types of computer systems.
Three research settings at
appropriate
course
in
in
terms of
the above ci'iteria:
(2)
a
graduate
an advanced undergraduate programming
course, given by the Electrical
an
(1)
industrial Dynamics, given by the Sloan School of
Management;
(3)
were selected as
i-i.l.Tc
Engineering Department; and
introduction course In management information systems
also given by the Sloan School of flanagement
.
The first and
second courses require the students to program, debug, and
analyze problems using the MIT Computation Center's IBM
computer which
is
used for both time-sharing and
batch-processing applications.
The ttilrd course makes use
of the Sloan School's non-time shared
the same
The
709^1
IBM
1620 computer for
typo of work.
Industrial Dynamics course was
Included in this
study because industrial Dynamics requires the use of
a
PAGE IS
system for the primary purpose of analyzing the
.e-.y
or...
..haracteri
s t
(Vianagement
i
cs of computer simulation models and dosli^ning
policies based on their analysis.
undergraduate programming course was included
The
to
investigate
the behavior of experienced programmers working on
a
fairly
well defined problem which primarily required programming
The management
effort.
information systems course
v;as
included because it would allow an investigation of the
effect of providing
of computer response
feedback delay
-•-
a
'better' quality and different amounts
a
to
the user while maintaining
feature not found
the other
in
the same
two
setti ngs.
£x2.eii mejD. t a
i
The
1
_T a.sJis
tasic
to be
assignment will
used
In
I
in
the course will
small-scale Independent
a
builder within this industry who
in
Dynamics course
market dynamics in
'f
student
Eacli
the role of
talce
average profits
ial
nc'
involve the analysi;
'Construction Industry'.
required to
the
is
trying to maximize his
light of certain market dynamics.
Through his analysis of the market and its behavior^
student
is
expected to arrive at
a
the
decision rule for
starting apartment construction that will accomplish this
goal
The task for the industrial
administered through the use of
homework problem.
The actual
and higiily constrained,
v/i
th
Dynamics course will be
a
'case'
assigned as
problem will be
botli
the construction of
a
simple
the
a
be
PAGE 19
Jnorpendent builder's decision rule to maximize average
prcvlts the primary area for creative v^orlu
will be required to complete the programminc of
'case'
a
simulation
(using the DYNAMO programming language
model
of
IJi/)/
debug the program,
this
students
Tlie
analyse the results, end propose
and test a decision rule to be used by the independent
bul Ider.
The assignment for the advanced programming course will
theorem proving algorithm in the
be the programming of
a
field of mathematical
logic using the MAD-SLIP programming
system available on the IBM
7091^.
The management information systems course will
the programming of an accounting and
assignment for
a
inventory control
manufacturing company^
The student will
assigned to one of the two FORTRAN-compat
languages available on
require
i
b
the non-time shared
1
e
I
Bf'1
be
programming
1620 computer
and will be allowed to program the problem in any way
des red.
I
The students'
will
be with
primary concern In the three assignments
the course project required by the
-" the data collection questionnaires will
an outside research pi"oject having no
grade for the assignment.
be primarily motivatc-d by
motivation
to
As
a
be explained as
Influence upon their
result,
tlie
students should
the class assignment "- a stronger
participate than that provided by most
laboratory experiments which use
pr imary mot vator.
I
Instructor
a
small
sum of money as the
PAGE 20
MethouolQ>;y
tasks will
three experimental
methodology for all
The general
be described
in
two distinct phases:
the method of collecting
a
the
the data
procedure for conducting the research; and (2)
be collected aiid
(J.)
It.
in
to
addition,
description of the Industrial Dynamics pilot study
v;i
1
1
be
discussed.
£xj2e£.l men t^l _Pi:o^eciure
The experimental
tasks will
be
presented
to
the
students as part of the normally required course assignment
for all
instructor
tlie
While assigning the problem^
three courses.
v.'i
1
notify the class that
1
Industrial
case of the
Dynamics Researcli Group)
is
a
research group (in
Dynamics course,
involved in
the
a
the
Industrial
study of some
aspects of computer usage and will be collecting data
concerning the methods used by the students
assignments of the type being assigned
the assignment
aslced
to fill
presented
is
to
a
the
the students,
out pre~exper imental
next section for
to
completing
in
class.
they will
through the use of this questionnaire).
to
be collected
Based on their
answers^ students with similar backgrounds will
a
this assignment will
be
different experimental
done on
a
be
questionnaires (see the
discussion of the data
and each assigned to
Before
be paired
treatment -•
random basis.
Each student's assignment to either the time-sharing or
batch-processing computer systems (or,
in
the case of
the
management information systems course, either of the two
PAGE 21
FORTRAN compatible programining systems)
v/il
1
be handed out
During the
along with the course project material.
assignment, and before the students begin using the
computer/
they will
have several
questionnaires
addition, after each use of the computer,
In
will be required to complete
complete.
to
the students
short questionnaires
a
pertaining to that interaction with the computer.
The students using the batch-processing system will
be
able to submit their computer runs once or twice each day --
depending on the course they are
the
to
The students assigned
in.
time-shared system will be able to use the computer
at almost any hour of
the day or evening.
Industrial Dynamics course,
the
For
the
students will have four
consoles available for their exclusive use -- all
consoles will be continuously 'logged
system and the students
w-I
1
in'
have access
1
to
the
able to 'log in'
t:.
;
to
M.I.T. campus.
during the day
in
purpose..!.^./
the students will
be
the computer from a console anywhere on
In
the case of
information systems course,
1G20 computer will
time-shared
limited only by the
other students using the consoles for the same
For the advanced programming course,
four
time on
the
management
the Sloan School's
have been reserved for several
addition
to
IBM
periods
normally schedul cd .servl ce
runs.
D.a_t.a_C(ilie£.tioa
The data will
be
collected by both the questionnaires
completed by the students during the assignment and/or of
PAGE 22
conij-'Uter
th'"
(1)
to:
dor.i.
This will
Input and output of each student.
discover how each student works;
(2)
ancJ
be
cross
check the data obtained from the ques ionna res the students
i
compl eted.
These questionnaires will
pre-exper Imen
t
iorma re administered before
i
the computer will
they will
a
Second,
a
questionnaire to gather background
tal
information before any manipulations take place.
ques
First
forms:
take several
investigate
tfie
the students make use of
influence of knowing that
be using one or another computer
system.
Third,
a
questionnaire completed after their analysis of each
computer run
to
investigate the students' attitudcis and
behavior during the course of the experitnent and
time expended,
measures of progress/
and
The fourth type of questionnaire,
done.
the students have completed
the above factors on
a
the
comments concerning the assignment.
obtain
type of work
administered after
the assignment,
overall basis as
to
v/cll
will
as
investigate
requesting
The final
questionnaires will be administered three weeks after the
experiment and will measure the degree of interaction
betv/een
the students while solving
this problem.
The data to be collected through questionnaire and/or
content analysis of the printed output will be:
amount of time expended on the experimental
(1)
the
task and the
distribution of that time over the period the experiment
in
progress;
(2)
the categorization of
this expend
into analyzing results, making changes, v;aiting for
is
'e
rn
PAGE 23
ofrc-Lults^ etc.;
submitted;
ih)
(3)
the number of computer
analysis of types of changes made and the
reasons for these changes;
(6)
runs
analysis of types of errors;
(5)
student's evaluation of the 'usefulness' of each run's
results; and (7) student's evaluation of 'effectiveness' of
each run'
£llol
resul
s
tr>.
Stiidji
One pilot study has been conducted to investigate the
use of the particular Industrial
Dynamics task
study of the Construction Industry)/
collection devices.
(
th(?
the use of
and
For the pilot experiment,
'case'
the data
several
students who had previously taken the Industral Dynamics
were asked to participate
cours'
/
experi
t.
in
a
pre-test of the
After completing the experimental
task
they
were asked for individual comments concerning the
presentation of the experiment,
the
task/
the
interest generated by
and the data collection devices used.
All
pro-test subjects were requested to refrain from discussing
the experiment with anyone
v/as
honored.
—
it
appears as
if
this
request
PAGE 2U
Spm '
Po
'-,
?
I
b
1
e
E x
i^
pr
i
n'l
ontfll
Difficulti
The following are several
in
es.
potent.ii'l
Tir^t,
the design of the proposed researcli:
appropriateness of
v/Iio
the experimental
inherent
tlio
settings and the
tasks could be questioned ~- especially by
experimental
those
botli
problems
believe that time-sharing applications have not
yet been developed and,
when
be qualitatively different
have been developed, will
tfiey
applications.
than traditional
This assumes that the area of
time-sharing and
batch-processing applications do not and will not overlap.
In
fact, experience to data on several operating systems has
shown many areas where this overlap is substantial.
The possibility of conrnuni ca
future subject Is
experimental
a
potential
tasks,
Wliether
communication had value
the other subjects
affected
of
in
in
some way.
independent work
to
o,r
t
i
1
on from one subject
the content of
not,
the
the conduct or actions of
the experimental
situation might be
Although emphasizing the importance
the students may not
insure they work
ndently of one another, measures taken after the
s:
ts
:
to a
problem in the conduct of the
Ir
p
/
receive their grades
v/i
1
1
allow for control of this
um
The last
Important area of potential problems concerns
data collection and the filling out of the
the subjects
computer run.
participating
in
the first
'Data Sheet'
tas'k
after each
There may bo some problems connected with
requiring the subjects to remember the necessary
dat-'.
by
PAGE
However/
the only alternative
collection
is
the use of a
to
2i>
the proposed method of data
time record,
divided into ten or
fifteen minute intervals, which the subjects would fill out
during each interval.
The
'Data Sheet' method was chosen
to
eliminate the possibility of subject antagonism resulting
from the large amount of time required and both Intentional
and unintentional errors and biases
of using the second method.
introduced as
a
result
PAGE 2o
Summarv and Implicati ons
To briefly SL'mmai' ize the major hypotheses of
research,
it
has been predicted
that
the user of a
time-sharing computer system will demonstrate
of performance than will
computer system.
of performance will
level
the
the user of a more
Furtlier/
two
it
this
higher level
a
traditional
was predicted that this higher
result from differential usage of
types of computer systems and that
the
characteristics of each computer system affects the user's
attitudes
tov.'ard
the computer system.
The major potential
pe'rsons
and
implication of this study is that
responsible for the design and selection of computer
information systems should consider
tiie
their requirements for speed of solution,
speed,
trade-offs among
access
cost,
and size of the accessible system before determining
the type of computer system to be used.
In
addition,
it
is
hoped
ti<at
this study
v/i
1
provide
1
a
basis for evaluating time-siiar ng and/or interactive
i
computer systems whicli have been designed to:
ni'-.sdiate
'debugging'
the researcher with
tool
for
tlie
the ability to
hunches at the time they occur;
a
tool
provide
for
a
immediate response
system to
tiie
to
(>)
programmer;
(1)
'de an
!-
(2)
,
de
test conjectures
provide the tea-'ier with
student inquiries; and
('t)
manager for the purpose of
experimenting with several different policies
in a
environment so the policy exhibiting the desired
characteristics can be implemented.
simulated
PAGE
It
is
Important to realize that
requirements for
a
computer or infon
such that implementation of
a
tima"s!
only increase the total cost of
a
cc
downgrade the service previously pro
computer system.
•
2 7
organization's
"on rystem may be
g
'
system would not
system,
with
a
but also
traditional
PAGE
23
FOOTNOTES
1
definitions of time-sharing by experts in the field
shall mean one
are: "i^y a time-sharing computer sy5>tem
tlirough a
users
that interacts with many simultaneous
Such a system will look to
number of remote consoles.
McCar thy 32)
eacli user like a large private computer".
Also/ "I3y time-sharing we moan those systems in which the
facilities of a computer comijlex are rapidly commutated
among Independent users who are eacli on-line at a remote
The terms
console." Glaser and CorbatcS (20).
will
be used
and
'time-shared'
'Interactive'
discussion.
It should
following
Interchangeably in the
sys'
computer
time-shared
be realized, that although
true.
We
necessarily
are Interactive, the reverse Is not
systems
are cons der nr:.. in this discussion, comi:iULcr
which are botli interactive and time-shared.
Tv;o
1
(
•;:,
I
2
I
See McCarthy (32), Corbati (G), Fano ( U. , and Licklidor
(27) for a discussion of the chatiges that have resulted
from th s deve lopment
)
.
i
3
The Arnorican £ederatlon of Information £roco5sing
jSocI Cities Is composed of three primary groups; tlie
Association for Computing Machinery, the liZlir, aiid
American Ooc""''Tita t on Institute.
the
i
l»
ad>Jitional "out of pocl;ct" expense required for
time-sharing results from the extra computer h.jrdwdre and
software required, the time-sharing accounting, tiie
transmission lines and user consoles, and the system
Iloweve-r, since the metliod of usoi^e of
tna ntal nance.
tliese systems is presently unknown, any costs or savin./.,
resulting from this usage cannot be calculated.
Tlie
I
5
Gee LIcklider (27),
(
i»
:i
Corbati (G), Schwartz CiO), and
Slot/.
)
and Daley (7).
C
Corbat6, Daggett,
7
Dennis (11).
8
f-ein
(13),
9
(!j),
p.
10
asked to compare interactive time-sliarod systems
with ba tcli-process ng computer systems, both part-time
and professional programmers were almost unanliiious in
tlieir agreement that Interactive time-shared -..ystems are
When pressed to define 'better', tlicy say
'better'.
first that they are able to do a 'better' job on sucli
p.
I's.
10 3.
IVhen
I
PAGE 29
Tlicy also
-systems, thus increasing their satisfaction.
point out that the time to complete this job is less than
It would have been had they used a batch-processing
These programmers do not explicitly cite other
system.
measures of productivity/ efficiency, or effectiveness.
(These observations resulted from informal contacts v>'I th
professional time-sharing programmers at M.I.T. and
Systems Development Corporation.)
the end products (the working programs) were
evaluated, there v^as no apparent difference in the final
There was, however, a difference in
programs produced.
There
elapsed
time
to produce these progranis.
total
the
first,
Increase;
this
causing
two
components
seem to be
the use of the Project MAC time-sharing system required
twice as much processor time for printing and editing
programs as for compiling them -- a three-fold increase
In processor usage; and second, the behavior of the users
apparently changed as a result of the greater
availability of the computer with time-shared systcins.
This resulted In a greater number of compilations (and
the accompanying Increase in printing and editing) than
vsfould have occurred with the traditional batch-processing
"system -- this increase have been variously estimated
with estimates ranging bet\' n 50?o and i^OO!^.
11 K'hen
12
should be noted that th
terms of on-line system
batch-processing systems a.
design, non-interactive.
It
in
'eflnltion Is relevant only
traditional
y their very nature and
the simplifyi,
assumption that the
We could. If
solution could, in fact, i^e ubtalned.
necessary, treat this as another variable.
13 We arc mailing
l'>
The DYNAMO programming language is described in Pugh (3 7)
Industrial Dynamics is described in
and Forrester (17).
Forres tor ( 17';
15
Preliminary experimentation indicates this number of
available consoles to be sufficient to handle tlie
students' requirements of access.
16
We are assuming
that these students have the same
qualifications as those students who will participate in
with the exception of having
the actual experiment
completed a ful course in Industrial Dynamics.
—
referring to the experiences of two experimental
time-sharing systems.. Project MAC at M.I.T. and TSS at
System Development Corporation, and two commercial
time-sharing systems, the IBM QUIKTRAN system and the
KeyData system.
17 We are
PAGE 30
REFERENCES
Bauer/ W. F., "On-Line Systems--The r Characteristics
and Motivation"/ On-L ine C ornputi n.p Sys tems/ American
Data Processing/ Detroit, 1955.
1.
i
;
Berleson, B. and G. A. Steiner, Human B ehavior An
nventory q± Sci entifi c F ndi n;!:s,/ New York, Harcourt,
Brace and World, lOBU.
2.
:
i
I
3.
Carroll, D. C, "Implications of On-Line, Real-Time
Systems for Managerial Decision Malting", (''ul.T. Sloan
School of Management v\'orking Paper 165--65, March, 1955.
h,
Carroll, D. C, "Man-Machine Cooperation on Planning and
Control Problems", M.I.T. Sloan School of (''lanagement
Working Paper l'i5~66, 1965.
5.
Collilla, R. A., "Time-Sharing and Mul ti -Process ng
Terminology", Da tama t on. Vol. 11, No. 11, November,
i
i
1965.
6.
Corbat6, F. J.,
Time-Sharing",
(Chairman),
Da
tama
ti
on
.
"A Panel
Vol. 10,
Discussion on
8, August,
No.
196it.
7.
Corbati, F. J., M. M. Daggett, and R. C. Daley, "An
Experimental Time-Sharing System", AF PS Conference
PjlQceedjjng s , Washington, D.C., Spartan Books, Vol. 21
(1962).
1
8.
Corbat6, F. J., "System Requirements for Time-Shared
Computers", M.I.T. Technical Report MAC-TPs~3,
Cambri dge,., Massachusetts, June, 1955.
9.
Da tama
10.
David, E. E., and R. M. Fano, "Some Thoughts about the
Social Implications of Accessible Computing", AF PS
.Conference Proceedings ^ V^ashington, D. C, Spartan
Books, Vol. 26, 1965.
ti
on ^
Vol.
11,
No.
11,
November,
1955.
I
11.
12.
Dennis, J. C, "Program Structure in a Multi-Access
Computer", M.I.T. Technical Report MAC-TR-11,
Cambridge, Massachusetts, May, IGG';.
'
^smonde, vl. M., Real-Time Data Process ing Sy s tems ^
ntrodoc torv Concepts , Prentice-Mall, Englev/ood Cliffs,
J., 1965.
'
13.
old, J., i^.evond Automatio n^
York, 196(4 .
McGraw-Hill Book Co.,
PAGE 31
Ik.
J. C.^ "The Impact of Information Technology on
Organization"/ P roceed n^s of. Xbil ,1 Q 6 Annual Meet! ng
Qf the Acadomv of Mana^tement^ Chicago, 195tt.
Emory,
i
15.
16.
"A Corporate Time-Shared Computing System",
ndustr a Mana;!;&ment Revi_e_v^, Vol. 6, No. 2, 19 65.
Ever,
I
'»
.
J.,
]
i
M., "The MAC System: A Progress Report",
Spectrum Vol. 2, No. 1, January, 1955.
Fano,
R.
I
EEE
..
17.
Feigenbaum, E. A., and Feldman, d., (Editors), Computers
and Thoufrht . McGraw-Hill, New York, 1963.
18.
Fein, L., "Letters
No. 11, November,
19.
the Editor",
Datamation
.
Vol.
11,
1965.
Forrester, J. V,'., J nd ustr al Dypamics, The MIT Press and
John V.'iloy and Sons, Cambridge, Massachusetts and New
.i
York,
20.
to
1952.
E. L., and F. J. Corbatd, "Introduction to
Time-Sharing", Datamatio n. Vol. 10, No. 11, November,
Glaser,
1954.
21.
Greenberger, M., "The Computers of Tomorrow", Atlantic
Monthly . Vol. 213, May 196iu
22.
Greenberger, M., "The Two Sides of Time-Sharing", AF PS
Conr>:res s .1.965/ Vol. 11, Washington, D. C, Spartan
Books, 19 65.
23.
Greenberger, M., M. M. Jones, J. H. Morris, Jr., end D.
N. Ness, On-l.ine Computation and Simul ation: Tho OPS -3
System, Cambridge, Massachusetts, The M..I.T. Press,
January, 19 66.
21'..
I
Greenberger, M. (Edi tor). C omputers and ^tie World. q± the
Future . Cambridge, Massachusetts, The M.I.T. Press,
19 62.
:5.
R., S. Strong, and J. B racket t, MAP; A System
M.I.T. Technical
for On-l-ine Mathematcal Analysis
Report MAC-TR-2'>, Cambridge, Massachusetts, 1965.
Kc'plow,
.
26.
2 7.
Krech, D.,
Individual
R. S. Crutchfield, and
in Societ y. New York,
L.
E.
Ballachey,
McGraw-Hill',
1962.
Licklider, J. C. R., "Man-Computer Partnership",
nternat ional Sci once and Technolor:y. No. 41, May,
I
1965.
28.
G., and H. A. Simon, Organ za tlons . New York,
March, J
John Wiley and Sons, 1958.
i
PAGE 32
J.^ Prop;rainmi nr. Roa -Time ComputQi- Systems /
Prentice-Hall, Engelwood CllffS/ N. J., 1965.
2
."l.-Jitin,
30.
MIT Computation Center, The Compatible Time Shr-^rinK
Sys torn
A Progr amme r s Gu| d e, (Second edition),
Cambridge, Massaciiuset ts. The MIT Press, 1965.
]
.
.
'
,
31.
MIT Preliminary Study Committee Reports, Quarter! y
Progres s Reports of tho. Real -Time Time-Shari ng Project,
Cambridge, Massachusetts, MIT Computation Center, 1961
(Unpublished Report).
32.
in
McCarthy, J., "Time-Sharing Computer Systems",
Greenberger (Edi tor). Com puters and the iloriii Q± the
Future , Cambridge, Massachusetts, The MIT Press, 1952.
33.
Morrissey, J., "The QUIKTkAN System", Datamation,
No. 2, February, 19 55.
5h
Myers,
.
35.
Vol.11,
"The Impact of EDP on Management
Organization and Managerial Work", M.I.T. Sloan School
of Management Working Paper 139-65, Cambridge, 1965.
A.,
C.
K'eisser, U., "MAC and Its Users", M.
Memo 185, Cambridge, Massachusetts,
Project MAC
(Unpublished).
I.T.
1965
Cambridge,
36.
Project MAC, Prof^.ress.. Report
Massachusetts, 1965.
37.
Pugh, A., Ill, DYNAM O User's Matiual . (Second Edition),
The MilT Press, Cambridge, Massachusetts, 1963.
38.
Samuel, A. L., "Time-Sharing on a Mul t -Conso
Computer", MIT Technical Report MAC-TR-17, Cambridge,
Massachusetts, June^ 1965.
39'
Scherr, A. L., "An Analysis of Time-Shared Computer
Systems", MIT Technical Report MAC-T(<-18, Cambridge,
Massachusetts, 1965.
I|0.
Schwartz, J., "A Genera --Purpose Time-Shared Computer
System", API PS Conference Proceed npis , iVashington, D.
C, Spartan Books, Vol. 25, 196U.
,1
I
,
i
1
1
i
kl,
Schwartz,
Da tama
U2.
t
!
I.,
Vol.
"The SDC Time-Sharing System",
10,
Shultz, G.
P., and T.
o n and the
of Glencoe, 19 50.
Or;^ani za
4 3.
J.
on.
t
i
No.
11,
fJovember
19
6 ^t.
L. Whisler, Manar,ement,
Computer , Glencoe, The Free Press
Simon, H. A., JhQ, cvj Sci ence. p_£. Mana/tomen
New York, Harper and Row, 1960.
i
'
t
Deci si on .
PAGE 33
kk,
Sprague, R. E., "The Information ULllitics",
Automation ^ Vol. 12, No. 3, 1955.
f*5.
Stotz,
Busipe.s^
R., "Man-Machine Console Facilities for
Computer-Alded Design", AJILP?. r.onf f^r^nce Proceedings.
Vol.
23,
1953.
U5.
Strachey, C, "Time Sharing in Large, Fast Computers",
Information Processing / (Proceedings of the
International Conference on Information Processing),
UNESCO, Paris, 1959,1960.
k7
Sutherland, I. E., "The Future of On-Line Systems",
On- Line Computing Systems , American Data Processing,
Detroit, Michigan, 1955.
.
kS.
'i9.
"Time Sharing at MIT and SDC",
No. 2, February, 19 5t|..
Da tamatio n,
Vol.
10,
Zannetos, Z. S., "Toward Intelligent Management
Information Systems", MIT Sloan School of Management
Working Paper HiS-GS, 1955.