THE HUMAN INTERFACE
IN THREE DIMENSIONAL COMPUTER ART SPACE
By
Jennifer A. Hall
B.F.A.
Kansas City Art Institute
1980
Submitted to the Department of Architecture
in partial fulfillment of the requirements for the degree of
Master Of Science in Visual Studies
at the
Massachusetts Institute Of Technology
October 1985
Copyright 1985.
JAHall. All rights reserved.
The author hereby grants MIT permission to reproduce and to
distribute publicly copies of this thesis document
in whole or in part.
1
/
I/
..
. ..
Signature of author......en..
r Hall
epartment.of Architecture
October 16, 1985
"
Certified by.
..............................................
Professor Otto Piene
Director, Center For Advanced Visual Studies
Thesis Supervisor
.......... ....................
Professor Nicholas Negroponte
Chairman, Department Committee for Graduate Student
Accepted by...............
OCT
Sc
Acknowledgement
I
am grateful
Advanced
to The Center For
Visual
The Archit.ecture Machine Group, and the patience
the
involved individuals who allowed me
of
Studies,
the
many
freedom to construct my
own special space at MIT.
My
thanks,
also,
Aesthetic Confidant, and
to
Patrick
friend,
Purcell,
and
to
my
my
editor,
Roberts, who saved not only me, but countless inocent
hours
of tedious paper crunching.
ii
:)
Professor,
Shelly
victims,
Table of Contents
Section I
.
Introduction .................
.
- ---
.
-.
-.-- -Pg
1
.Pg
3
Defining 3D Space............
..
3
Physical Computer Space....
Logical Programming Space..
.Pg
4
Impressed Graphics Space...
.Pg
5
Neurological Representations. ...
.
Computer as Artifact............. ....
......
.......
.....
..
Machine Vision...................
...
Pg 8
...
Pg
11
.Pg
18
Qualitive
Considerations
of Machine Vision.............
...
Pg 18
.Pg20
Gesture Analysis..............
Computer Constructed Motion...
21
Computer Controlled Tracking..
Pg 23
iii
....
.Pg
.....
Body Tracking.................
.Pg
25
Body Tracking for Imagery..
.Pg
25
Hardware/LEDware Overview..
.Pg
26
Graphical Marionettes/Scrip ting
by Enactment............... .....0 ..................
Pg30
Signing as Computer Input
.Pg
Concept............
-Pg 37
34
Pg38
Conclusion..... ...................................---
Pg 39
Illustrations.. ............................................
Pg 46
...
.-----
References..... .......................
Section II
Appendix............... ..
General Comparison..
.
Pg
- -.
. .-
.
.
.
.
.
.
.
.
.
.
.
52
Pg 52
...
Op-Eye..............
.Pg56
Watsmart............
.Pg59
Bibliography...........
.Pg61
iv
Section I
V
Introduction
The interactive bridge cannot be reasoned with, cannot be
communicated with in human terms, and by its very
is
nature
It is a link,
capable only of repetitive rigid responses.
location between two situations. An Interface
is
a
surface
a
regarded as the common boundary of two bodies or spaces.
It
is the study of these bonded bodies or spaces that enables us
to understand what happens to each when they interact.
We use our sensory impressions that
reach
us
from
the
external spatial world surrounding us to construct the mental
model of our universe.
This includes all that we think,
same
that we create. Human output spawns from the
sensory pool.
universal
It is this pool of human perception that
that
relationships
together
relatively un-associated.
at
appear,
first
all
ties
encounter,
With this raw material we compile,
associate and communicate.
Our most identifiable source for
locating our physical
second
is
the
beings
psychic
in
a
self-identification
spatial
understanding
Page 1
of
context.
that
is
The
context.
Physical constituents which link us to another person, object
or idea,
are not,
in
and
of
themselves,
interactive,
merely the structural bond or surface interface.
the passage of the creative
psyche
create what we have come to
know
Page 2
across
as
The
this
Human
but
It requires
bridge
to
Interface.
Defining 3D Space
Physical Computer Space
The spatial qualities of a computer begin with the system
itself.
of
electrons
boards
inserted
slots
into
machine.
the
of
on
rows
in
placed
clippers, and signal boosters,
resistors,
of
banks
through
flow
Internal
law.
machine and, as such, cannot violate natural
rivers
embodied
physically
a
The computer is, of course,
Memory
is
bit
of
organized as a system of cells which represents each
data points.
There can be
no
space
word has a number associated with it,
memory
in
byte
or
every
machines,
In most
The
matter.
without
machine not only occupies space, but also formats
even geometric patterns.
discrete
of
data, making computer space ultimately composed
circuit
which
is
called
of
as
a
its
address.
thought
Computer memory can also be
The
randomly accessible and variable information.
needed information.
In
case,
either
workspace or region of storage that
free of its physical or spatial
can
origins.
processor
identifies
need not know which portion of data it wants, but
computer
the
never
It
quite
is
Page 3
is
a
break
therefore
only natural to think of a computer as having a logical
that has plasticity and dimension.
the
locate
to
with a simple vector style coordinate system
of
bank
area
Logical Programming Space
The addressing, or numbering of
as
analytical geometry, and is as old
centuries.
three
His
space
least
at
Descartes,
spatial
awoke
system
numbering
from
derived
is
state
perception from a bodiless, finite, culturally defined
with numbers, and numbers associated with ideas.
and
terms
abstractions,
The duality
think
of computer space requires that a programmer
physical
associated
be
Ideas could
into the pure extension of idea.
both
in
and
pragmatically
philosophically.
"The computer programmer lives in two
worlds simultaneously: his space is a
logical entity (like the space of the
topologist), and yet it is logically
realized in the transistors of storage.
it resembles Einstein's
In this way...
space which is mathematical and abstract
claims to be the space of the
and still
world of experience."
<Bolter 84>
physical
The
space
of
to
subject
machine. is
the
terrestrial limitations, yet the logical properties remove it
from its mechanical origins.
Mathematics,
physics, allowed the mechanics of Newtonian
put
into
Euclidean
correspondence
and
Riemannian
algebraic
with
spaces,
evaluated in relation to each other
terms.
numbers
in
with
intertwined
geometry
numbers.
could
linear
and
to
be
As
in
then
be
graphic
Mathematical space had become a coordinate system.
Page 4
Impressed Graphic Space
"Elements
of
data
cannot
be
manipulated endlessly; they cannot remain
forever in the central processor.
At
sometime, they must be deposited."
<Bolter 84>
As long as computers use devices such as hardcopy and the
cathode-ray tube, they will continue to output
all
graphics
onto the two dimensional world of paper and silvered surface.
Here an image is virtually denotated.
by the apparent convergence of space
Dimension
taught
to
is
implied
us
by
the
Renaissance masters Piero Della Francesca and Michelangelo.
Figure 1.
@wpm
Page 5
It
on
is
designed
this
interpolation occurs.
<Benton 84>
inherent
an
not
Monocular cueing is
learned
easily
function of human optical perception, but is
at an early age.
optical
that
surface
type
The creation of this
of
impressed exhibition space is the melding of apparent optical
depth and human understanding of those laws.
the
Perhaps
genuine
spatial
vector
As in
graphics lies, literally, behind the tube.
computer
3D
of
reality
or
raster scan, the convergence of rays upon the screen emanates
from the tube's yoke.
from the yoke at the same time as the viewers
own ablility
to
receive
and
shot
Beams of light are
dimension as an optical phenomenon.
perceive
their
project
the
onto
to
sculptural
any
obtain
reception, when the information will
prior
time
This will be the only
surface,
meeting and causing a common interface plane.
One of the only
display
devices
exploits the potential of 3D graphics is the
procedure
for
acquiring
surprisingly simple.
the
that
hologram.
The
computer
from' the
data
is
A display screen can be thought of as a
porthole or viewing zone into
computer's graphic
today
available
reality.
the
digitized
First,
world
viewing
the
of
zone
the
is
algorithmically fixed, much like choosing the proper lens for
a camera.
When this area has been
can place computer-defined objects
arena.
defined,
into
the
the
programmer
pre-designated
This is only one of an infinite number of angles that
Page 6
By moving
these objects can be viewed from.
viewport,
and
recording
dimensionality of
new
each
the space can be
tapped.
here
is
instead
that
consecutive frame in time, all
the
onto the same piece of emulsion.
in
the holographic
product is
of
technique is
This
The
animation.
each
displaying
is
information
This emulsion is
technique instead of a real
a display
or
true
the
perspective,
known as "keyframing", and is used often in
difference
camera
the
recorded
then
used
object.
The
that takes advantage of our
ability
to
known
as
see with two eyes and works on an optical function
visual disparity.
This is a fine technique for
reality.
Basically, it is
a
creating
a
3D
of
a
collection
reality
patterns and process, the latter being capable of
artificial
of
producing,
The
destroying, and modifying the former <Newell/Simon 82>.
most
important properties of these patterns are that they can
designate objects, process,
or other patterns, and
they designate process,
they can be
microcosm unto itself.
The question
this
when
that
is
It
interpreted.
does
is,
that' erises
a
finite interpretation of reality have the same intrinsic
capabilities as the world in which we exist? Has
of the world within the creator's mind been captured
computer
process
perhaps
through
universal law?
Page 7
some
nature
the
by
underlying
the
and
Neurological Representations
The instruments that we make are,
naturally,
extensions
of ourselves both physically and psychologically.
When using
instruments skillfully, we internalize aspects of them in the
form of kinesthetic and perceptual habits.
they not only
modify
them,
the
altering
thus
of
extensions
become
In
sense,
perceptions,
our
basis
this
of
our
effective
The creation and rendering of
relationship to ourselves.
artificial reality has not only ornamental significance,
symbolic as well.
an
but
We make in our own image and likeness, and
holding
looking at what we make is like
mirror.
but
we
From these representations,
internal representations or
models
in
conceptions which continue to alter our
up
the
proverbial
modified
construct
our
brains,
revise
to
making
approach
sense of the world.
"The basic idea of cognitive science
is that intelligent beings are semantic
engines
-
in
other
words,
automatic
formal systems with interpretations under
which they consistently make sense."
<Haugeland 82>
Given this premise, it is then possible to understand the
physical
correlation
between
humans
and-
machines,
somehow, it is difficult to accept that a human as a
Page 8
yet
machine
Simply put, we have not yet made
is a reasoned principle.
a
machine that comes anywhere close to bioelectrical potential.
The
model
as
human
of
with people.
the
In Biology,
for
sophisticated methods
theory
compartment
reactions
where
cases
conjunction
in
used
is
works as model system only when it
Machine
processes.
making
decision
the
in
part
ourselves better only when we take a conscious
act of the computer's
understand
us
helps
machine
provides
open
of
Open
systems are compared with conventional closed systems.
systems
show
contradict the usual
quantum leaps in logic.
These gaps are often
considered
to
only
by
explainable
life,
be the vital characteristics of
huge
produce
they
as
laws
physical
to
seem
which
characteristics
inherent
introducing soul-like or entelechial factors into the organic
happening.
Artificial intelligence does not have the vital force
It.
principle directing growth and life.
is
or
in people can be perceived as being irrational
A
because it systematically claims
psychol,
a
Intelligence
misrepresentation
is
not
theory
effort
only,
it
Intelligence is not merely a real Software
Page 9
is
illusion
or
is
creative
Without
information.
best
human interface, the computer's
illogical
be
never
can
computer
once
Moreover, creativity
removed by algorithm to the real thing.
behavior to a machine.
model
a
or
also
String
the
inherently
of
life.
action.
Searcher,
which is apparently conceptually
but
smart,
a
more than a jukebox, that is, no more than
finding
matching
appropriately.
electrical
and
mechanism
outputing
no
for
them
Dualism is the concept that mental states are
states of the soul, i.e.
nature.
impulses
physically
the
human's
moral
and
emotional
Feelings like anger and elation come from the
And the physical manifestation of these emotions is
soul interacts with the body.
Once again,
the laws of physics.
Page 10
you
how
cannot
soul.
the
relax
Computer as Artifact
We can
think
of
constructed
socially
memory
computer
space
an
as
usually
that
artifact,
functions
a
in
predictable ways in order to serve our technological society.
Artifacts are
almost
always
less
sophisticated
than
the
leading researches of science because they must be built upon
In
this
ornamental
and
ground that is already intellectually quite simple.
sense,
electronic
space
is
used
for
predictable symbolic significance.
Figure 2.
V
93~~~--
X
Y2
Xt
X
Y1
17U7S
873841831
Y
U1 71
74
739 813
V47
K8 ?9
96287i M73874344 M83 M18 V75 ft
136-3
87-1
974 42 M83 06i 9Mt IL
13 O9 474 U738
5688
C4781
Z&28478
9 0743 8404 OM3 oet in?
1825 IM7 6M- U7 482
UK3 M118
18139H 847673 8738 8476
I565MR
228
477
102S SM7 O7UR7 87+1M83 06
01-M 84674 I7M IM7 M
Page 11
V71t
ag71
We can also see how computer memory has properties
that,
collectively, can take us back to even more intuitive notions
of what an artifact means.
For instance, numbers in computer
terms are as discrete and corporeal as even a pythagorean
of
aspects
of
ancient times would have
them.
Moreover,
all
most
internal computer space is finite, like the universe in
cosmologies.
Yet through all the universal attributes, the process
and
computing is usually experienced a-s a relatively private
personal affair.
The act of personal
intellectual and
spatial
seclusion.
computing
It
is
is
during
of
one
of
these
computer intimacies, that an emotional bonding to the machine
occurs.
Page 12
Figures 3-5.
I
Page 13
)
During this bonding period, our
bodies
along
minds, are blended together, and move towards
experience of space <Lewin
74>.
Here
we
a
with
our
hodological
connect
to
our
machines through sensoral techniques by creating a mental map
of our destination.
computer
as
surrounding.
object,
In this case,
as
This is a
surface,
cognitive
offers the participant a metaphoric
transmission procedures.
Figure 6.
Page 14
the
destination
and
mapping
state
as
is
the
philisophical
technique
of
which
multi-layered
The
language
through
which
a
machine is selectively opaque; it
but
allows
representing
the
or
exploitation
addressing
masks
of
the
workspace in which memory is stored
becomes virtually transparent.
programmer
absolute
variable
subject.
is
commands
an
a
addresses
names
The
region
enigma,
In other words, the
for
for
or
it
physical
shell of a computer and its peripherals are designed to be as
invisible as possible.
Figure 7.
Page 15
This duality between
opaque
and
transparent
makes a computer workstation a wonderous locale.
structure
The CRT can
become the visible expression of an intellectual or emotional
condition while the physical presence of the hardware remains
a symbol in and of itself.
Figure 8.
Page 16
We
have
physical
remaining
structure
of
as
a
only
information.
programmed
carefully
In
apparatus
display
the
make
to
transluscent,
conceptual
frame
for
frames
become
less
our
time,
ourselves
outgoing
less
and
obvious, leaving us conscious of only half the icon.
But
in
fact, each aspect of the artifact, the invisible tool and the
visible manifestation
of
that
experience of these machines.
Page 17
tool,
makes
up
the
total
Machine Vision
Qualitative Considerations Of Machine Vision
acquisition
the
literally,
Machine vision, which is,
information external to the computer through
of
sensors,
optical
is one sensory technique that bridges computer language and the
of
This type
world outside the computer's reality.
interface
modeling
of
human behavior, and, instead, uses the acquisition of real,
or
outside, information,
own
their
helps remove computers from
internal
producing
ultimately
machine
terms,
physical
In
account of spatial perspective.
accurate
more
a
vision helps create a more dynamic or sensitive system.
work
In only a gross technical analogy does machine vision
to
focus
image,
the
three-dimensional
world,
disparity
and,
perhaps
lenses
Both use
the same as the vertebrate's organs of sight.
interpolate
to
crucial,
most
the
are
peripheral to a more important central processing area.
But at closer comparison, the
We find that there is
no
machine
end
similarities
technique
abruptly.
available
which
accurately simulates the spectacular craftsmanship of the human
eye.
as
The computer has no external associated
the
eyelids,
and
eyelashes,
to
structures
screen
qualities of light falling on the retina, which
in the overall
function
of
"seeing."
Page 18
Within
such
quantities
and
together,
aid
the
internal
function
in the overall
internal
the
Within
"seeing."
of
structure of the eye, light converges upon the thin wall of the
retina
through
the
fluid
vitreous
body.
device
detects
multi-layered photosensitive
sorting
radiation,
electro-magnetic
neuro-electronic pulsed information.
to
wavelengths
converting
and
delicate,
This
Today's machine vision is
as
process
the
equipped with no such subtle features, nor is
elaborate.
The use
of
photosensitive
relatively crudely ground lenses and
devices
eventually
will
equipment of finer properties.
inadequate
with
replaced
be
High quality translators could,
to
speed
in theory, produce machines with sufficient
examine
dictionaries
and
that
the
quantitative factors of refined hardware, sufficient speed
and
detailed syntactic rules by installing large
look-up tables.
Many machine
vision
feel
experts
sufficient detailed syntactic rules make true machine vision an
unrealistic endeavor.
Even if a technically equivalent machine vision
interpretation
could eventually be produced, the machine's
the outside reality would, compared to
information
from
external
bio-mechanical
Though
fundamentally remain inadequate.
sources,
data
processed or understood from the context of
natural language.
peripheral
it
would
its
optics,
draw
still
would
own
of
its
be
internal
The computer will never truly understand the
difference between synthetic and externally derived data.
Page 19
When
understands
the
visual scene in purely syntactic structure, and relates to
the
exercises
its
a computer receives optical
information,
The
process in the same linear logic.
it
machine
intelligence in problem-solving by modifying symbol
until
it
produces
a
structure.
solution
structures
will
Solutions
therefore remain as represented symbolic constructs.
The
conclusion
is
therefore
the
First,
three-fold.
creation of realistic simulated worlds is within the reach of a
machine's ability.
Secondly, the accuracy of
artificially
an
intelligent machine to adequately mirror the world in which
Thirdly, optical
exist remains that of a heuristic hypothesis.
hardware for data acquisition is
used when
we
the
simulation
or
facsimile of real world space is not acceptable.
Gesture Analysis
People/Machine relationships are empirical kinships
the
naturally, we expect the computer to become
where,
extension
In turn, we
additional compiler for our own bodies and psyche.
are, however, also extending the physical space of computer
well.
raw
If the computer could externally obtain
the
Page 20
from
physical
graphics.
Optical
peripherals help the computer literally reach
own hardware boundaries making
data
as
the
the physical world, there could be a blending of
world into the digital reality of computer
or
machine's
out
beyond
physical
its
space
potentially as large as its field of vision.
body tracking,
be
the performer can
In
becoming part of the computer, i.e. of the
as
of
thought
case
the
the hardware, but the programmed living machine.
actually
just
not
machine,
of
In this case,
the living machine is an extension of both the human's and
the
sense,
it
computer's
capabilities.
a
In
philosophical
certainly is appropriate that machines look to
their
creators
for the acquisition of their data.
Computer Constructed Motion
An
attempt
early
to
as
such
movement
human
record
This
Labanotation used a pictogram method for recording dance.
computer
was the predecessor to techniques used in
simulation
today.
The Dynamic Technique starts with a simulation of force
muscle and the mass which
it
moves
81>.
<Hatze
A
representation reacts to a pre-designed set of rules
addressable by changing programmed
variables
graphic
are
that
as
The product
speed and theoretical physical limitations.
reality of simulated constructs.
such
of
force,
is
a
The better the constructs the
better the simulated reality.
Another
widely
used
technique
is
keyframing.
The general packaging of this
is
to
a
line
line
function.
Page 21
one
Literally
commonly
simulation
frame
by
called
concept
frame,
interpolation between two pre-defined postures gives the
model
Used
in
conjunction with this, analytical movement descriptors such
as
a route to travel in a predetermined amount of time.
Cutting's walking algorithm <Cutting 78> may be used.
example, motion is determined by describing fifteen key
on a walking figure as a vector-valued oscillation.
Figure 9.
Page 22
In
this
points
by
digitally
three-dimensional
locations
In 1980 movement on film was first
grabbing frame by frame to deduce
the
Highlights found on
on key points <O'Rourke 80>.
constitute a mapping from object space
information <Maxwell 74>.
analyzed
Pattern
as
known
technique,
This
positional
freedom
to
subject
Recognition, as used in the Moving Light Display technique, was
a first attempt to draw raw data from a real world source.
The
onto
the
obvious flaw is that once the information is decoded
surface of film, all true
been stripped away.
Its strength
the
within
lies
have
definitions
three-dimensional
computers
ability to interpret changes in line and two-dimensional form.
Computer Controlled Tracking
specification
the
Towards the late 70's research began in
all
of
these techniques use stereoscopic triangulation from a pair
of
or tracking of articulated
motion
directly.
Almost
digitizing devices to determine spatial location.
Most of these techniques were
in tracking human motion.
<Maxwell 82>
for
developed specifically
Now intermediate media such as
use
film
accurate
rendering
One of these, the Coda-3 System monitors up to 8
landmarks
could be bypassed producing a closer, more
of the subject.
or locations on the body at one time.
3 fan-shaped beams of light across the field of view
reflection.
Another, the Selspot System
Page 23
sweeps
Optical scanning
uses
light
to
sense
emitting
diodes worn on the human body to locate each critical limb in a
spatial context.
The lights are
connected to the computer.
then
picked
up
by
cameras
It is from this technique that
Op-Eye system for tracking the body was developed.
Page 24
The
Body Tracking
Body Tracking For Imagery
human
Generally, most three-dimensional
systems
have
been
for
designed
sports
biomechanics, neurology and
in
use
are
of
fields
the
mechanics.
systems
research in robotics, tracking
digitizing
scale
current
With
considered
being
and
displacements
within the study of high resolution
static
attitude in wind tunnel operations.
The prime purpose of these
researches is
to acquire data for the statistical
analysis
of
human motion.
The following project
uses
similar
mentioned above with one fundamental difference:
body
tracking is an imagery-data acquisition tool
those
as
techniques
act
the
which
of
helps
produce an imagery-based product.
Optical Body Tracking research is an
MIT's Graphical
Marionettes
Project
Ginsburg 83/ Purcell 84/ Lewis 84>.
Page 25
of
Maxwell
82/
Its prime purpose is
the
<Bolt
creation of trackdata for the animation of
figures.
project
offshoot
81/
computer
generated
Hardware/LEDware Overview
The Op-Eye
Body
Tracking
alternative to the Coda
3
or
System
Selspot
offers
providing
systems,
similar three dimensional coordinate system
The
suit which has light
fabric
(LEDware).
emitting
Each
band
diodes
of
diodes
position
performer
(LEDs)
a
signature pulse which differentiates and locates each
free space.
Page 26
wears
into
sewn
emits
a
computer.
the
to
In this case, the system tracks the corresponding
the joints (nodes) in the human body.
inexpensive
an
of
a
the
specific
limb
in
Figures 10-11.
Page 27
Figures 12-13.
Page 28
Optical hardware consists of four two-axis
diodes made by
United
Detectors
Technology.
detect the position of a spot of light on
the tracked LEDs and two-dimensional
each
detector.
Using
LEDs
and
These -sensors
its
surface.
position
then obtained via four electrode connections
sensors
effect
lateral
Both
information
at
the
edge
sensitive
to
is
of
the
infrared range avoids optical interference (optical noise) with
existing or ambient light.
a camera
system
that
This also avoids the complexity
would
demand
further
elaborate
of
data
processing and expense.
Figure 14.
The sensors do not require precise focus as
the
detectors
sense the centroid of the light spot on its surface.
The signals from each
detector
are
fed
interface module, Op-Eye, where they are first
Page 29
to
the
central
amplified
then
digitized for processing in a manner compatible with the IBM PC
series of micro-processors.
Graphical Marionettes/Scripting By Enactment
The Tracking Suit is used as
motion of
the
achieving
fluid
human
body.
motion
an
It
without
analysis and tedious programming.
is
the
alternative
the
to
animator's
difficulty
The user
modeling
wears
way
of
motion
LEDware
create motions that are optically received and stored as
within the IBM-PC.
A real-time stick figure body is
of
to
files
displayed
to check continuity.
Figure 15.
These files can later be filtered and used, in conjunction with
larger
databases
on
mainframes,
animated sequences..
Page 30
to
create
more
developed
Figure 16-17.
One long-term project is to create a filing system full
of
used
at
trackdata motions much like the cell animation system
most animation houses.
be stored,
The difference is that all motions
tree-structure
memory for use.
fashion,
and
If the movement of a hand
delicate object is
needed,
the
animator
Page 31
easily
called
grasping
a
goes
the
to
can
into
small,
HAND
Directory
and
for
checks
names
Ideally, there would be a number of
back
found that can be loaded
looked at in the real-time
into
like
different
files
gra-sping
and
micro-processor
the
stick-figure
delicate.
or
Grasp
resolution.
choice then made, the animator need only call up this
With
a
movement
by name, joint or node location, and speed-within the scripting
of the animation.
Each movement can be used as many times,
as many locations, and
actually
to
create
speeds
the
as
movement
developing a computer animation then
cut-and-paste, a more natural form
sequence.
Figure 18.
Page 32
desired
again.
becomes
for
without
The
a
creating
the
process
job
an
of
in
need
of
data
animated
Another possible use for trackdata in
total
animation
system
is
as
an
of
a
85>.
software
gives power to certain hierarcical
to
addition
technique called adaptive motion <Zeltzer
potential configurations
the development of
In
motion.
a
environment.
the
throughout
Such examples of
other
words,
automatic
can
of
avoid
to
object
of -the
has -any
kinematic and behavioral complexities of a person, it might
If the
better to have help from an example.
problem
a
object
collision, information about the physical nature of the
If the
that
detection
collision
But
a
object
an
constraints
are currently being implemented in software.
in motion is also important.
the
This object or figure
or figure existing within dataland.
maneuver
process
drives
certain amount of spatial knowledge can be given to
be trained to
feedback
This
that
a
of,
be
for
example, the performance of joint motion under a certain set of
involved
conditions
exceeds
the
knowledge
base
of
implemented software, trackdata could help establish the
the
rules
of behavior.
Theoretically, the
allocation
of
searching
through
and
finding the appropriate trackdata file could be all done in the
task-level of the software.
By checking
between the motion in software, and
the
similar
trejectories
trackdata
file,
the
animation could try to repair an awkward or incorrect motion by
altering it with a trackdata alternative.
The ultimate goal would be to
Page 33
have
hierarchical
software
to
guiding-level a few alternatives
motion
questionable
all
the
offer
task-level
the
having
by
This
motions.
making possible the simulation of more complex
would be accomplished
system
the
of
portions
other
to
that would allocate tasks
before the data reaches the scripting or animator-level.
Motion and Computer Controlled Tracking lies
final product
looks.
opposite
positions.
to retain
is attempting to
Constructed
the initial
starting
are
motion
simulate the workings of the body while
attempts
in how natural the
Each of these techniques
arrive at the same naturalness, but
form of motion
to begin with the natural
polar
to
best
strives
of
The assumption is
is digitally processed.
from
technique
the tracking
qualities
life
Constructed
Computer
between
difference
The fundamental
to
the data as
that it
it
is
better
a
better
produce
product.
Signing As Computer Input
Personal computation by
definition
A goal in developing these systems
process.
element while enabling the system's
mature.
should
be
a
private
is to retain
this
interactive capabilities
to
The use of a gestural glove can help minimize physical
inhibition
with
the
hope
of
seeing
interchange.
Page 34
a
more
personalized
This Body Tracking tool is geared toward the handicapped as
both a learning and a communications
been designed to help
teach
especially those with
fine
simple and intimate setup.
the
student
motor
The
enhancer.
of
Software
sign
student
wears
language,
by
dysfunctions,
the
has
using
glove
practice hand positions in front of a graphic monitor
a
to
equipped
with sensors.
Figure 19.
A Real-time
stick
figure
representation
of
position simply mirrors the student's attempts.
position closely resembles a letter of
the
When
alphabet
stored in the software look-up table, that letter
printed
on
the
screen.
In
this
the
fashion,
or
hand's
a
hand
or
word
word
sentences
is
and
paragraphs can be constructed as fast as the student can sign.
Page 35
Figures 20-22.
Page 36
Concept
Series
devices are used
to
a
make
electrical elements produce
These currents and
voltages
variables of the body
and
a
abstracted.
of
set
behave
its
Both
the
that
such
isomorphically
In
this
human
a
IMAGE /'IM-IJ/
N 1: A likeness or
limitation of a
person or thing. 2: A visual counterpart
of an object formed by a device
(as
a
mirror or lens).
3: A mental
picture or
conception; impression, idea, concept.
VB 1: To create a representation
to bring up before the imagination.
Figure 23.
Page 37
case,
the
general
"felt" analogy.
2:
the
and
body
electronically reflected image is preferred in
with
element
conceptual
the
qualities.
electrical
motion.
and
electronic
other
model
physical
relation between physical element
explicitly
and
capacitors,
resistors,
of
of.
is
its
and
Conclusion
and
physical
a
as
Generally, we can think of all we make
phychic extension of ourselves. Being the territorial creatures
our
validate
as
(such
symbolic
utilitarian or
a
bowl
cooking
aesthetic design, respectively), their function is to
we
be
tools
our
Whether
existences.
own
more
the
has obvious logical advantages. The more we make
can
territories
collective
and
we seem to be, extending personal
its
and
transmit
our own nature into our surrounding reality.
Computers not only carry the logical formats found
but
mathamatics,
to
appear
display
exclusively to living organisms. We expect
attributed
been
have
these
machines
now,
has
thinking
to
exclusively
attributed
been
to
also
which
receive, assimilate, and output correct solutions,
until
of
levels
discrete
interactive responses which, until now,
pure
in
organisms. We have given them artificial vision, speech, touch,
smell, taste, and motor functions. They are able
to
reproduce
their own kind. They have even produced reasonable attempts
their
between
correlations
making intelligent
internal
and
our
own
external worlds. We continue to build these tools
image
and
characteristics that
Ironically,
simutaneously
we
have
mirror
they
yet
likeness,
attribute
we
given
and
to
true
computers
transcend
existance.
Page 38
the
in
have
don't
still
the
qualities.
life
which
attributes
logic
at
of
our
own
Illustrations
Figure 1.
Mirror Cell
Frame from a computer animation.
30 Seconds.
Inside
a
computer
three-dimensional
surfaces spin in free space.
these
Reflected from
through
surfaces rotate, the images
the
and
environment
reflected
in
the
surfaces
camera.
the
is the landscape sceen from the viepoint of
the camera travels
flat
environment
As
flat
the
are
mirrors
constantly updated.
Figure 2.
Data Display
Data
monitor
displaying
two
separate
two-dimensional locations, received by two separate
optical
sensors.
approximately
40
The
sensors
hertz,
are
therefore,
receiving
the
screen
of
sets
sets
data
is
of
at
being
refreshed with a new set of data points about once every 30th
of a second. This data will be saved into a
file
later drive objects and figures for animation.
Page 39
that
will
Figure 3-5.
Pulsating Orifice
3 frames from a computer animation
5 Second loop
The wall of the animated orifice
is
driven
by
scripted
software. Each individual 'clot' rotates in X
and
their own origins. The
location
camera
receives
its
Y
around
from
trackdata. The trackdata was created by the author by placing
her chest against the face
of
a
monitor
located
next
to
optical sensors. The final product is a kenetic wall
of
red
polygons and a camera that
in
the
heaves
into
the
space
middle, and then out.
Figure 6.
Computer Pulsed Island.
detail, computer installation
Orifice Animation on a gold-leaf monitor reflects onto the
chest of a stylized women's body.
Page 40
Figure 7.
Computer Pulsed Island
detail, computer installation
Computers painted red, black and white, sit on
of white sand. Graphics which pulse from the
an
center
island
of
the
screen outwards, change from black to red, and back again.
Figure 8.
Computer Pulsed Island
computer installation
.
White monitor in foreground displays a
of a women's body which echos features from
stylised body towards the
left,
and
beneath it.
Page 41
the
digitized.
both
pure
picture
the
white
black
sand
Figure 9.
Stick Figures
Two overlayed
frames
from a test
animation.
Figure 10.
Model in LEDware
Collecting
upper
body
trackdata
for
Marching
Band
Animation.
Figure 11.
Model in LEDware
Collecting upper and lower body trackdata archives.
Figure 12-13.
Dancer in LEDware
Collecting upper and lower body trackdata for animation.
Page 42
Figure 14.
Model in LEDware
Checking stick-figure for continuity of motion..
Figure 15.
Stick figure in pieces
Real-time stick figure is separated
into
four
sections.
The top two squares are what each seperate sensor is tracking
from the uppper body. The lower two squares are what they are
tracking of the legs.
Figure 16.
Frame from Marching Band Animation
20 seconds
Each of the animated
band
members
trackdata in the scripting level.
Page 43
were
assembled
from
Figure 17.
Frame from Eggman Animation
Eggman was
assembled
from
trackdata
in
the
scripting
level, and then produced through the keyframing technique.
Figure 18.
Frame from Cloudpeople Cell animation
30 seconds
Both software and
trackdata
driven
figures
share
this
animated space where gravity is non-existant.
Figure 19.
LEDware/Glove
There are 15 critical joints in the human hand. Each joint
has an associated strip of LEDs sewn into the glove.
Page 44
Figure 20-22.
From real-world to dataland
The individual nodes of the hand are tracked and displayed
on the screen with or without the connecting lines. From this
information alone, the modeling of a more refined hand can be
executed.
Figure 23.
Electronic window
Page 45
References
<Bolter 84>
D.J. Bolter
Turings Man: Western Culture In The Computer Age
Carolina Press
1984 pg 97
<Bolter 84>
D.J. Bolter
IBID
1984 pg 81
<Benton 84>
S.
Benton
Lecture, MIT
1984
<Newell/Simon 82>
A. Newell and H.A. Simon
Computer Science as Empirical Inquiry
Mind Design
MIT Press
1982
Page 46
<Havgeland 82>
J. Havgeland
Semantic Engines
Mind Design
MIT Press
1982
<Lewin 84>
K. Lewin
Space Newtworks: Towards a Hodological Space Design
Man, Starting with a Cognitive/Perceptual notation
PHD Thesis, Mit Mitropoulos, University of Edinburgh
1974
<Hatze 81>
H. Hatze
Musocybernetic Control Models of Skeletal Muscle
University of South Africa Press
1981
Page 47
for
Urban
<Cutting 78>
J.
E. Cutting
Behavior Research Methods and Instrumentations
10(1):91-94
1978
<O'Rourke 80>
J. O'Rourke
Image Analysis of Human Motion
PHD Thesis,
University Of Pennsylvania
1980
<Maxwell 82>
D. Maxwell
Graphical Marrionette:
Master Thesis,
A Modern Day Pinocchio
MIT
1982
<Bolt 81>
R. Bolt
Proposal for the Development of a Graphical Marrionette
Techinical Report,
MIT Architecture Machine Group
1981
Page 48
<Ginsburg 83>
C. Ginsburg
Human Body Motion as Input to an Animated Graphical Display
Master Thesis,
MIT
1983
<Purcell 84>
P. Purcell
Graphical Marionettes Project
Progress Report to NHK Corporation
1984
<Lewis 84>
J. Lewis
Computer Animation of Human Figures in Conversation and Action
Master Thesis,
MIT
1984
Page 49
<Zeltzer>
D.
Zeltzer
Towards
an
Integrated View of 3D
Paper for AMT,
Computer Animation
MIT
1985
Page 50
Section II
Page 51
Appendix
General Comparison
The following
is
a
comparative
evaluation
between
the
existing Op-Eye Tracking System (MIT), and The WATSMART Spatial
Motion Analysis and Recording Technique available from Northern
Digital, Waterloo, Ontario. The method of
systems are quite similar as observed in
operation
difference lies within
technique is
making
how
clearer
both
non-contact
devices.
controlled
the
trackdata
The
data
files.
3D
important
acquisition
Is
it
upgrading existing Body Tracking hardware and software,
it advantageous
both
the general f'unctional
overviews below. Fundamentally, they are
digitizing, state of the art tracking
for
worth
or
to buy into a pre-existing package such as
is
the
WATSMART.
Good trackdata is
a
function
of
proper
triggering
and
synchronization within the data collection technique. What
has
been overlooked with the current system is
the
importance
camera setup and proper hardware calibration. Whenever
Page 52
one
of
or
the system must have
more cameras/s-ensors are physically moved,
the
proper
facilities
constructs. There is
the
OP-Eye
system.
automatically by
to
camera
program
calibration
cameras
tubular steel
all
of
view
in
frame containing 22 pre-surveyed LED markers
run.
to
provided with the system and takes only a few minutes
The alignment procedure requires the placing of a
done
is
registration
the Watsmart by use of a
with
existing
no such updating technique
This
mathematical
3D
its
reconstruct
in use. The calibration file produced by this procedure
must be used as an input to
all data collected by
. This calibration
and is conceivable to
the 3D linear transform process
for
that particular camera setup.
technique was considered for
the
though,
some extent. There is,
Op-eye,
major
one
drawback. There are already too many arbitrary hardware devices
it virtually
built into the system, which makes
traditional
and
installation. Existing trackdata
critical
files
and has to be low-passed filtered
no
any
to
are,
These
optic
extremely
is
to such excess
that the
product is not accurate for analysis and barely acceptable
use in animation. The final evaluation must be made
Page 53
lab
only
therefore,
accurate for gross motor functions. The raw data
noisy
were
accuracy.
standards established for relative/absolute
are
to
there
correctly calibrate. In its initial conception
standards
impossible
by
for
direct
0
comparison of trackdata made from each system.
The Watsmart also includes
software
accuracy for modeling purposes.
It
to
is
increase
easy
to
the
obtain
pitch, and roll of models as well as compute valid 3D
3D
yaw,
data
in
situations where there are few or no markers within common view
of at least two
cameras.
This
is
extremely
important,
for
visual obscurity is one of the prime reasons for excess optical
noise in the current system.
Unfortunatly, all optical tracking devices have the similar
problem of background lighting and reflections.
Many
surfaces
that appear as black in visible wavelengths are actually
quite
reflective in the near infrared light red region. Care must
be
taken in setting up an
in
experiment
so
that
position to reflect directly from markers
all
surfaces
a
into
anti-reflective. Unobvious problems occur' because
camera
the
are
camera
receives both the true marker optical signal and its reflection
at the same time
causing
an
apparent
position measured. This is due to
the
shift
fact
in
the
marker
that
the
camera
averages all infrared light it sees, and therefore a reflection
cannot be separated out from the objective image.
The Op-Eye Package was constructed over
Page 54
a
period
of
two
years.
All
standards
technical
featured
The
the
in
specifications obtained
Machine Group,
specifications
in lab
are
owner's
manual
findings
of
and
the
actual
Architecture
MIT.
Watsmart
is
a
total
specifications are averages between
package.
All
standards
featured
preliminary technical description manual and
The Center
between
averages
for Bioelectrical Engineering, MIT.
Page 55
the
technical
in
findings
a
of
Op-Eye Package
Hardware
1) 4 bi-lateral effect diode cameras
2) LEDware
3) A camera controller and power chasis
4) -
5) A computer and controller board
6)
-
7) Adequate cables to interconnect all the above
Software
1)
-
2) Define the number of nodes and their location
3) Datacollection and storage of raw data
4) Raw data to 3D information using 3D linear
tecniques
5) Produce 2D stick figure graphics
6) -
Page
56
transformation
Op-Eye Package
Technical Specifications
A/D Resolution: 12 bit. 1 part in 4096
Spectral Range: 350-1100 nano meters
Approximate Saturation Level: lOmW/cm2
Typical Position Linearity: 0.5% in central 25% of
detector/
3% in central 75% of detector
Minimum Detectable Intensity at Detector Surface:
(10
to the 7th ohms)
2uW =10v (full scale)
Programmble Gains 1,10,100,500
Page 57
High
gain
Watsmart Package
Hardware
1) One or more high resolution infrared cameras
2) Variety of infrared diodes (IRED)
3) Camera controller and power chasis
4) Marker strobing controllers (8 markers each)
5) A computer and controller board
6) A
pre-surveyed
calibration
frame
to
facilitate
quick
camera setup and ensure positional accuracy
7) Adequate cables to interconnect all the above
Software
1) 3D re-construction/calibration used when the
sensors
are
moved
2) Parameter software to define an experiment in terms of the
number of markers used, the frequency (frame
rate)
of
data
collection, duration of collection, etc.
3) Data collection and disk storage
4) 3D linear transformation software that changes raw data to
a more useable 3D format.
Page 58
Watsmart Package
Software (Cont.)
5)Produces 2D stick figure graphics in real time
6) Perform general functions such as
Butterworth
of
Spline
filtering, velocity and acceleration derivation and graphical
examination of both data and its derivatives.
Technical Specifications
Camera resolution: 1:406 or 0.00025 of field
Absolute accuracy: Corrected to 0.002 of full range (optional
to .00025)
Field of view:35 degrees w/ supplied 22 mm custom lens (vert.
and hor.)
45
degree
diagonal
angle.
Optional
90
degree
diagonal custon lens.
Distance: .75 to 10 meters, dependent on
emission
power
if
LED marker
.Sensitivity:
Infrared light in spectrum.range 800
nanometers
to 1050
Aggregate max.sampling rate: 8,000 markers/second/camera
Page 59
Op-Eye Optional Position Indicator
United Detector Technology
3939 Landmark Street
Culver City, Ca. 90230
(213) 240-2250
Watsmart Spatial Motion Analysis and Recording Technique
Northern Digital
415 Phillip Street
Waterloo,
Ontario N2L3X2
(519) 884-5142
Page 60
Bibliography
Badler, N.,
"Digital Representations Of
and Smoliar, S.,
Movement", Computing
Sureys,
Vol.
11,
No.
1,
March
Human
1979,
pp19-38.
Baker,
Kenneth(editor),
Brave
New
World?.
Pergamon
Press,
1982.
Elmsford, N.Y.,
Bet-telheim, Bruno. The Empty Fortress:
Infantile Autism amd the
Birth of the Self. The Free Press, New York, 1967.
Bolt,
Spatial Data-Management. Technical Report, MIT
Richard A.
Architecture Machine Group, March, 1979.
Bolt, Richard
For
"Proposal
A.
The
Graphical
Marionette".
Technical Report. MIT Architecture Machine Group Report, 1981
Bolt,
Richard
A.
The
Human
Interface,
Where
Computers Meet. Lifetime Learning Publications, A
Wadsworth,
Inc.,
Belmont Ca.,
1984.
Page 61
People
Division
And
Of
Bolter,
David J.
Turing's Man: Western Culture In The
Press,
University Of North Carolina
Age.
Computer
Hill,
Chapel
N.C..
1984.
the Computer State. Vintage Books, A
Burham, David. The Rise Of
Division Of Random House.
Cutting, J.E. "A
New York, 1984.
Program
to
Generate
Dynamic Point-light Displays". Behavior
Synthetic
Methods
Research
as
Walkers
and
Instrumentation 10(1):91-94, 1978.
Danzinger, James N.,
L.
Williams H.
Dutton. Rob Kling, and Kenneth
Kraemer. Computers And Politics: High Technology
Local Governments. New York:
Dechert,
Charles
R.
The
in American
Columbia University Press, 1981.
Social
Impact
Of
Cybernetics.
University Of Notre Dame Press. London 1966.
De Fleur, Melvin L.,
and Ball-Rokeach, Sandra. Theories Of Mass
Communications. New York, New York: Logman Inc.,
Page 62
1982.
Dertouzos, Michael L.,
and Joel
Moses.
The
Computer
Age:
A
Twenty-year View. Cambridge, Ma.: MIT Press. 1979.
The
Evans Christopher,
Micro
Millennium.
New
York:
Viking
Press, 1979.
Federal Information
Issued
By
The
Sources
And
Systems
Sourcebook Series. Washington, D.C.,
Inc. Berkley, Ca.,
Fink, D.
G.
Doubleday Co.,
A
Directory
Congressional
1980.
Confessions Of
An
Infomaniac.
Sybex
1984.
Computers
Inc.
1980
General.
Comptroller
Ferrarini, Elizabeth M.,
1980,
And
The
Human
Garden City, N.J.
Ginsberg, Carol. "Human Body
as
Anchor
Books,
1966.
Imput
Display". Master Thesis, MIT. May, 1983.
Page 63
Mind.
To
Animate
Graphical
Goethals, T.
Gregor. The TV Ritual.
Universalist
Association.
Beacon Press Books, 1981.
Hatz, H.,
Mycrocybernetic Control Models
of
Skeletal
Muscle.
University Of South Africa Press, 1981
Haugeland, John. Mind Design. MIT Press, Cambrige, Ma. 1981.
Higgins, Joseph R.
C.V.
Human Movement: An Intergrated Approach. The
Mosby Company, St. Louis, Mo.,
Hutchinson,
A.
Labanotation:
The
1977.
System
Of
Analyzing
And
Recording Movement. London, 1954.
Hulteen,
Eric
Andrew.
"The
Human
Interface
To
Personal
Computers". Master Thesis, MIT, June, 1980.
Kidder, Tracy. The Soul Of A New Machine.
1981.
Page 64
Avon
Books,
N.Y.C.
Krippendorff, Klaus.
And
Communication
N.Y.,
N.Y.,
Gordon And Breach Science Publishers,
In
Control
1979.
Notation.
Movement
Laban, Rudolf von. Principles Of Dance And
Society.
Bristol, Great Britan: J.W. Arrowsmith Ltd. 1975.
Lewis,
"Computer
John.
Figures
Human
of
Animation
in
Conversation and Action". Masters Thesis, MIT, June, 1984.
Maxwell,
Marionette:
"Graphical
Delle.
Rae
A
Modern
Day
Pinocchio". Master Thesis, MIT, June, 1983.
Mitropoulos,
Space
Efitmos G.
Cognitive/Perceptual
Notation".
PhD
Towards
Hodologocal
With
A
University
of
Starting
Man,
Urban
For
Design
Networks:
"Space
Thesis,
Edinburgh, 1974.
Andrew
Negroponte, Nicholas and Lippman,
"Transmission
Cybernetics
Of
Presence".
Technology
A
proposal
Division,
Defense
and
Bolt,
submitted
Advanced
Richard.
to
Research
Projects Agency. The Architecture Machine Group, MIT, 1983.
Page 65
the
Nora;
Simon,
Cambridge,
O'Rourke,
Ma.: MIT Press,
J.
Of
Computerization
The
and Alain Minc.
Society.
1980.
Image "Analysis
Of
Motion".
Human
PHD
Thesis,
University Of Pennsylvania, 1980.
Pheiffer, John.
The
Thinking
Machine.
J.B.
Lippincott
Co.
Phildelphia, 1962.
Purcell,
Patrick.
"Graphical
Report To NHK Corporation".
Project:
Marionettes
Architecture
Machine
Progress
Group,
MIT
Cambrige Ma., Febuary, 1984.
Roberts, Shelly.
I'm Sorry, But
I
Don't
Speak
Hexidecimal.
Write Protect Publishing. New York, NY. 1985.
Rose,
Frank.
"Wired To God: Accolytes
Of
High
Fair, August 1984, Volume 47 Number 8, pg. 40.
Page 66
Tech".
Vanity
Rybczynski, Witold. Taming The Tiger:
A
Struggle
To
Control
Technology. Viking Penguin Inc. N.Y.C. 1983.
Smith,
Brian
Reffin.
Soft
Art
Computing:
And
Design.
Addison-Wesley Publishing Company, Workingham, England, 1984.
Sussman, Lenard R. "Glossary For International Communications".
The Media Institute, Washingtion D.C., 1983.
Turkle, Sherry. The Second Self Computers And The Human Spirit.
Simon And Schuster, N.Y.C., 1984
Weiner, Norbert. The Human Use Of Human Beings: Cybernetics And
Society. Garden City, N.J.: Doubleday, 1954.
Winner, Langdon. "Mythinformation:
Romantic
Computer Revolution". A paper delivered
Technology Department, U.C. Santa Cruz.
Page 67
In
The
Politics
And
Politics
to. the
Zeltzer,
David.
"3D
Movement
Simulation",
Siggraph
1982
Tutorial, July 1982.
Zeltzer, David. "Towards and Integrated
Animation", Paper Delivered for AMT, MIT.
Page 68
View
of
3D
Computer
September 1985.