Ambient Displays:
Information Visualization through Physical Interfaces
Sandia Ren
Submitted to the Department of Electrical Engineering and Computer Science
in Partial Fulfillment of the Requirements for the Degrees of
Bachelor of Science in Computer Science and Engineering
and Master of Engineering in Electrical Engineering and Computer Science
at the Massachusetts Institute of Technology
May 19, 2000
C 2000 Sandia Ren. All rights reserved.
The author hereby grants to M.I.T. permission to reproduce and
distribute publicly paper and electronic copies of this thesis
and to grant others the right to do so.
Va
MASSACHUSE T INSTITUTE
OF TECH NOLOGY
JUL 2 7 2000
LIBRJARIES
Author
Department of Elec6fcal Engineering and Computer Science
May 19, 2000
Certifie d by
Hiroshi Ishii
Thesis Supervisor
Accepted by
Arthur C. Smith
Chairman, Department Committee on Graduate Theses
Ambient Displays:
Information Visualization through Physical Interfaces
Sandia Ren
Submitted to the
Department of Electrical Engineering and Computer Science
May 19, 2000
In Partial Fulfillment of the Requirements for the Degrees of
Bachelor of Science in Computer Science and Engineering
and Master of Engineering in Electrical Engineering and Computer Science
Abstract
Ambient Displays are a means of interfacing humans with digital information in our peripheral
perception. They present information through subtle changes in movement, light, and sound that can
be processed in the periphery of our attention. This thesis documents the various steps taken
towards making some headway into the potential use of these informative physical interfaces. The
first was a creation of a toolkit with which ambient displays can be quickly developed and configured.
Displays were then designed, implemented, and evaluated for four application domains: stock market
activity, website visitor tracking, natural wind movement, and network activity. The process of
creating the displays led to a number of initial conclusions about the design of ambient displays. The
work described here is meant to be a starting point for user evaluations that will complete the
evolution of ambient displays.
Thesis Supervisor: Hiroshi Ishii
Title: Associate Professor of Media Arts and Sciences, MIT Media Laboratory
2
Acknowledgements
This thesis would not have come together without the help of many to whom I owe my deepest
thanks and appreciation.
Hiroshi Ishii
For giving me the chance to venture into this unfamiliar area and for his strong guidance,
support, and belief in me along the way.
Phil Frei, Jay Lee, Ali Mazalek, James Patten, Brygg Ulmer
For the ideas, the brilliance, and the friendship that have made them a wonderful group of
people I am blessed with the chance to work with.
Phil Frei
For all his work in bringing the pinwheels to life, for his creativity and good sense of design,
and for being tall enough to reach the pinwheels for me.
Brygg Ulmer
For providing me with many of the resources I needed to create the displays and for the
long discussions that have shone much light on my research.
Jay Lee
For his sketches that are in this document and for bringing me into TMG.
Victor Su
For opening the door to this opportunity for me, for his unending patience to my equally
unending list of questions, and for his friendship and all his "yes sandia?"s over the past two
years.
Rujira Hongladaromp
For picking up where Victor left off, for his diligence and good work in all of my projects,
and for his sense of humor.
Seye Ojumu
For doing the mundane job of assembling all the pinwheels without complaint.
Ali Pearlman
For her work in creating the pinwheel and various display structures.
Muriel Waldvogel, Jeffrey Huang
For their enthusiasm in bringing the pinwheels into the Swisshouse and their renderings and
documents that are in this thesis.
Rob Poor
For all his help and suggestions in the creation of the pinwheels. The pinwheels would not
have come about so successfully without him.
Andy Wheeler
For helping a frustrated Sandia put together the first pinwheel board and for teaching me to
solder.
3
0
Laurie Qian, Matt Debski, Duncan Bryce, Charatpong Chotigavanich, Tim Lu, Greg
Bollrud, Andy Wheeler, John Gavenonis, Tim Shiau, and all my friends
For their unconditional friendship, care, and encouragement that has gotten me through this
hectic year.
Charles Lee, Lawrence Wang, Jeff Hsing, Alex Shing, and all of Eyeshake
For putting up with my schedule and for being such a fun group of people.
Dad, Mom, and Abraham
For creating a warm nest of love, strength, trust, and understanding, and giving me a happy
and loving home.
4
Table of Contents
Title
1
Abstract
2
Acknowledgements
3
Table of Contents
5
Table of Figures
6
1
Introduction
8
2
Background
10
3
Pinwheel Development Toolkit
15
15
24
3.1
3.2
4
Hardware & Structure
Software API
Applications
4.1
4.2
4.3
4.4
4.5
4.6
Interface Design Criteria
Graphical User Interface
Application I: Stock Market Activity
Application II: Website Visitor Activity
Application III: Natural Wind Movement
Application IV: Network Activity
30
30
31
32
43
55
65
5
Discussion
73
6
Future Work
83
7
Conclusion
88
Appendices
A
B
C
D
89
89
91
96
111
Circuit Schematics
Pinwheel Structure Components
Pinwheel API
Swisshouse Floorplan
References
112
5
Table of Figures
2
3
4
5
Background
ambientROOM media
2.1
2.2
WaterLamp
2.3
Pinwheels
10
12
12
Pinwheel Development Toolkit
3.1
Overall System Architecture
3.2
Block Diagram of Pinwheel Circuitry
3.3
Motor Driver Truth Table
3.4
Sample Motor Behavior
Pinwheel Assembly Process
3.5
Modular Dependency Diagram of API classes
3.6
16
18
19
19
21
25
Applications
4.1
Stock Price Fluctuations in a day
Possible Stock Portfolio Ambient Display
4.2
4.3
Volume Activity in a day
4.4
Possible Timeline Arrangements
Mapping volume changes to pinwheel speeds
4.5
Initial Stock Tracker GUI
4.6
Individual Stock Tracker GUI
4.7
4.8
Close up of Single Array
4.9
Stock Market Ambient Display
Directionally Arranged Pinwheels
4.10
4.11
Time Zone Arrangement
4.12
Tree Structure Arrangement
Website Tracker System Architecture
4.13
Mapping number of visitors to pinwheel speeds
4.14
4.15
Initial Website Tracker GUI
4.16
GUI for Configuring Website Tracker
GUI Displaying Visitor Information
4.17
Rendering of Ambient Display in Swisshouse
4.18
4.19
Ambient display showing wind direction at various time points
4.20
Vector Calculations for Wave Effect Algorithm
Counter Increment Time vs. Natural Wind Speed
4.21
Numerical Angles of Directions
4.22
4.23
Conversions for Area Rotations
4.24
Horizontally Oriented Matrix Display
4.25
Vertically Oriented Matrix Display
Scattered Clusters Display
4.26
'Daily' Router Activity
4.27
Initial Network Activity GUI
4.28
4.29
Tracking Router Activity
4.30
Ambient Display for Network Activity
33
33
34
35
37
38
39
40
41
44
45
47
48
50
51
51
52
53
56
58
59
60
61
62
62
63
66
69
70
71
Discussion
5.1
Users in Ambient Display Environments
75
6
A
B
D
Circuit Schematics
Pinwheel Board
A.1
A.2
Serial Port Interface Circuit Board
89
90
Pinwheel Structure Components
B.1
Pinwheel Structure Components
B.2
Finished pinwheel circuit board
Front view of Pinwheel Container
B.3
Side view of Pinwheel Container
B.4
Back view of Pinwheel Container
B.5
B.6
Front view of container with motor
B.7
Back view of container with motor
Fully stuffed pinwheel container
B.8
Front view of pinwheel
B.9
Side view of pinwheel
B.10
91
92
92
92
93
93
93
94
94
95
Swisshouse Floorplan
Swisshouse floorplan showing tentative location of ambient display
D.1
7
111
1
Introduction
Technology these days presents most of its digital information as "painted bits" on rectangular flat
screens. This is restrictive in two ways; it confines the user's focus to a certain space and is only
informative if given a significant amount of attention. In this thesis, we present ambient displays,
physical interfaces that overcome these restrictions by presenting information in our architectural
space through subtle changes in movement, light, and sound that do not require as much focused
attention. They are derived from natural ambient displays that awaken our senses and make us aware
of events occurring that we are not directly involved in. For example, the sound of rain informs us of
the weather; it does so without requiring us to focus our attention on it; rather it is in the periphery
of our attention and subconsciously makes us aware of the rain. Whether or not we want to act upon
this awareness is then up to us.
Ambient displays function in a similar manner. They were designed to take advantage of the human
ability to process events in the background of our awareness [2]. With ambient displays, users can be
aware of states of things that are important to them without having to devote constant attention to
them. While they normally reside in the background of a user's attention, ambient displays also have
the ability to move to the foreground of attention when the information they are presenting is in
some extreme state, thereby alerting the users when these extremes occur.
Ambient displays are made up of kinetic physical objects called ambient fixtures, which can be used
either individually or as a group. One such ambient fixture is the pinwheel, which, metaphorically
speaking, spins in a "bit wind". This is the fixture currently being used in our ambient displays. By
laying out pinwheels in various arrangements, we create displays that use the entire physical
environment as an interface between humans and online digital information. With ambient displays,
the various sources of information that normally compete for space on a small screen can present
themselves throughout one's architectural space. The space can then convey information that
smoothly moves into the user's center of attention when needed and fades into the periphery when
not in use [3].
Ambient displays can be all around us and are suited for many different environments. They fit well
in a home environment where people feel the need to be connected to loved ones that are
geographically distant from them. They are also suitable for environments where many streams of
information need to be constantly monitored. Because of their ability to move to the foreground of a
8
user's attention, they can be used, for example, in aviation cockpits or power plant control rooms, to
signal when something is wrong [1].
Using these ambient physical interfaces to visualize information is a largely unexplored area. The
general direction of this thesis was to make some headway into this area by studying how these
displays may be used. Three specific research goals were identified. The first was to create a
technically sound infrastructure for the pinwheels, to allow for quick and flexible development of
robust ambient displays. The second was to study what kinds of contents were appropriate for these
displays, i.e. which application domains it would make sense to use them for. The last goal was to
study how the pinwheels may be used to map information in these different domains. This involves
looking at how the combination of the kinetic motion and layout of the pinwheels may create
interfaces that are meaningful and informative. To accomplish these goals, a number of displays were
designed, implemented and evaluated. A toolkit was first created that can be used to quickly develop
and configure ambient displays. This toolkit was then used to develop ambient displays for four
specific application domains: 1) Stock Market Activity, 2) Website Visitor Activity, 3) Natural Wind
Movement, and 4) Network Activity. Through the process of creating these displays, we were able to
draw some conclusions about the use of ambient displays.
This thesis will explain the steps taken to achieve the three goals and discuss the lessons learned in
the process. Section 2 discusses the background research and ideas among which ambient displays
were born. Section 3 describes the different components of the toolkit and how they fit together.
Section 4 looks at each of the ambient displays that were built and discusses both the design
considerations behind each of them and their implementations. Section 5 presents some conclusions
drawn based on the building of the displays. Section 6 presents a next step for research in ambient
displays. Section 7 concludes the thesis by summarizing the findings.
9
2
Background
Ambient displays grew out of the research being done at the Tangible Media Group at the MIT
Media Lab. This next section will discuss this research and the projects that eventually led to the
current ambient displays. This section will also look at the research that has been done in the area of
peripheral awareness and information visualization for both are important characteristics of the
displays.
2.1
Tangible Bits
The idea of using physical objects to convey information evolves from a "Tangible Bits" vision
where tangible interfaces are used for human computer interaction [4]. They bridge the digital world
with the physical world by coupling bits with everyday physical objects. In this way, information can
be brought off the flat rectangular screens and into our physical environments. With tangible user
interfaces, it is expected that people can use the skills that they have developed and that feel natural
to them to physically manipulate digital information. The other key idea of tangible bits is to make
users aware of information at the periphery of their perception. Ambient Displays were born out of
these two key concepts [1].
2.2
ambientROOM
The ambientROOM developed previously by the Tangible
Media Group explored the ideas behind the Tangible Bits
water
ripples
-natural
concept. It was developed as a platform for expressing
digital information with ambient media. Ambient media
patcles
includes environmental light, sound, and movement that can
be processed in the background of a person's peripheral
bottles
awareness [1]. It can be used to inform people of events and
to give people a sense of community. People have a need to
feel connected to loved ones especially when they are apart;
Figure 2.1: ambientROOM media
ambient media can be used to create a persistent yet non-intrusive connection with others. The
following are some of the ambient media that were a part of the original ambientROOM [1].
10
M
Water Ripples
The actions of another can be manifested as the motion of a solenoid in a shallow water tank. This
motion causes water ripples whose shadows are then reflected onto the ceiling with a lamp. A
constant rippling gives people the feeling that another is active and around them.
Active Wallpaper
Movement of illuminated spots on the wallpaper can represent activity in a remote work area. The
motion of the spots is directly correlated to the level of activity in the work area, providing a display
of remote activity. This sort of ambient media unobtrusively makes people aware of things that are
going on outside the confines of the ambientROOM.
Ambient Sound
When sounds of dry-erase pens rubbing against a whiteboard are heard, people become
subconsciously aware that there is activity in a work area. This is a common sound in a workplace, so
transmitting it into the room at a low volume does not disturb the inhabitant and also conveys
remote activity.
2.3
Ambient Fixtures
Ambient fixtures are an extension to the ambient media of the ambientROOM. They are standalone
kinetic ambient media. While the ones in the ambientROOM were restricted to being inside, ambient
fixtures allowed the externalization of the media into open space, thereby allowing many people to
benefit from them at once [2].
The Water Lamp (Figure 2.2) is an extension of the water ripples of the ambientROOM. It consists
of a light that shines up through a pan of water, which is set in motion by changing information. As
the water moves, different patterns of light are projected onto the ceiling above the lamp. The
pinwheels (Figure 2.3) evolved from the idea of using airflow in the ambientROOM. The flow of air
itself is difficult to control, but visualizing the airflow turned out to be something that could be used
to convey information. Thus, the pinwheels are spun by a "bit wind" of digital information that
controls the rate and direction of the pinwheels. Compared to something like the WaterLamp, the
pinwheels are good fixtures to build large displays out of because of their size and the wide range of
possibilities for interfaces of multiple pinwheels. The "bit wind" is also representative of information;
the wind flows through the pinwheels similar to the way digital information does. The displays built
for this thesis were all made up of pinwheels.
11
Figure 2.2
Figure 2.3
WaterLamp
Pinwheels
photo by
Andrew Dahley
photo by
Webb Chapell
Through the design of ambient fixtures, three main lessons were learned that can be applied to
ambient displays as well [1,2]. One was that ambient displays must translate digital information into a
pattern that successfully presents the information. Secondly, location of ambient fixtures is important
because of their persistency. People learn to look for them in a certain place so placing them well in
the architectural space is essential. Lastly, the threshold at which attention transitions from
background to foreground is an open design issue that should be looked into. This thesis continues
the research in these three areas.
2.4
Information Visualization
Information visualization is defined as the use of computer-supported, interactive, visual
representation of abstract data to amplify cognition [8]. Quite a bit of research has been done in the
area of graphical information visualization. All sorts of frameworks have been developed for
visualizing data. COOL, for example, translates textual objects and relations into pictorial
representations [9]. With the advent of the Internet, much research has been done in visualizing vast
amounts of data for the web [11]. Evaluations have been done on the effectiveness of text, 2D, and
3D visualizations [10].
Not as much research has been done in regards to physical interfaces for information visualization.
However, some of the ideas for graphical interfaces can be applied to physical interfaces as well. For
example, metaphors are essential in both types of interface design [6]. Awareness of the use of
metaphors can make displays more effective in communicating information to a diverse group of
users because they make data easily recognizable, understood, and remembered. They achieve
effectiveness through associations of organization or operation.
12
Research in the area of architectural spaces as information spaces has shown that the placement of
"information objects" in relation to other physical objects depends on many factors [5]. One is the
size and area needed to display all the information. A second is the need for persistency and visual
accessibility to the objects in the space. A third is the various ways to convey relationships between
multiple objects. One last factor is the ambient information in the space that can provide background
and context cues.
2.5
Awareness Studies
An essential characteristic of ambient displays is their reliance on people's peripheral awareness.
Awareness is essential in coordinating collaboration in shared workspaces; people need to know
events and activities of other people in the workspace. Our concept of ambient media was motivated
by the Fields and Thresholds work of Dunne and Raby (1994) and the Live Wire of Jeremijenko [1].
With our ambient displays, awareness is not limited to other people's activities but also extends into
awareness of events that are happening at a more global level - for example, stock market activity
and wind movement.
The AROMA project (Pederson & Sokoler, 1997) researches ideas around peripheral awareness [13].
They believe that people have developed skills in reading their environment, especially those for
everyday needs. For example, the urban citizen maintains awareness of the neighbors by the
soundscape of the apartment building and neighborhood. People in office spaces make good use
peripheral awareness - we are aware of the comings and goings and activities of our colleagues
without paying focused attention on them.
There are also various cognitive science theories related to ambient displays. Many were studied
during the development of these displays [1]. One is the ecological perspective of psychologists like
J.J. Gibson[15]
and Ulric Neisser [16]. They describe perception as an active task that humans
participate in inside their environment [1]. Our design of ambient displays has also been heavily
influenced by attention studies on notions like the "cocktail party effect" [14]. This is a person's
ability to selectively move his or her attention around a busy environment. Ambient displays aim to
take advantage of this ability. Other attention studies include those by Treisman [18] and Rensink
[17]. Treisman showed that information does not necessarily have to be in the foreground of a
person's attention in order to be processed. Rensink and others have performed experiments to show
how perception is influenced by attention. They have been successful in showing how attention
works at a basic level but not with more complex surroundings [1].
13
2.6
Related Work
There have been a few related works in the area of ambient media. Perhaps one of the most noted
ones is Natalie Jeremijenko's "Live Wire" that she created when she was an artist in residence at
Xerox PARC. It consists of a single string that moves in response to network activity. Thus, it
alternately waves calmly or shakes violently throughout the day, depending on the traffic load on the
local computer network. The Live Wire has many of the same characteristics as our ambient displays.
The AROMA project by Elin Pedersen and Tomas Sokoler attempts to mediate the same awareness
that people who are located physically close are effortlessly able to maintain among people who are
geographically distant [13]. They explore people's ability to peripherally update a sense of our social
and physical context. Towards their goal, they have developed a set of technologies that they call "an
augmentation of existing technologies" [13]. They explore the use of abstract representations to
peripherally convey a sense of remote presence. Their initial findings suggest that it is possible to
make sense of remote activity even with primitive activity displays and that symbolic representations
can be processed peripherally to convey a sense of remote presence [13, 19]. This project also
explores many of the same ideas behind our ambient displays and its results are encouraging for the
use of ambient media.
Audio Aura is a project whose goal is to provide serendipitous information via background auditory
cues that are tied to people's physical actions in the workplace [7]. A project by Elizabeth Mynatt,
Maribeth Back, and Roy Want at Xerox PARC, it explores the usage of audio cues to connect a
person's activity in the physical world to information from the virtual world. It aims to leverage the
human ability to peripherally process audio to be aware of the state of things around us and thus
aimed to develop periphery displays that are not physically distracting. Towards this goal, a number
of sonic landscapes were prototyped in VRML worlds. Three different scenarios were explored: 1)
entering a bistro gives audio cues about new email, 2) an audio cue for visitors to indicate how long
the person they intended to visit has been out of the office, and 3) a group pulse that represents the
collective activity of individuals in a group. Initial user reactions were positive. Users found the
sound choice to be good in general and that the sounds generally remained nicely in the periphery [7].
Again, this project ties in nicely with the goals of our ambient displays, the main difference being that
its focus is more on audio effects while ours is more on visual ones.
14
3
Pinwheel Development Toolkit
We envision having ambient displays everywhere and used by everyone. Thus, it would be ideal if
users could create displays easily and quickly without having to understand the low-level workings of
the pinwheels. To that end, we created a toolkit with which users can develop ambient displays. The
toolkit comprises a hardware component that consists of the circuitry needed to run the pinwheels, a
structure that is used to hold all the pinwheel components, and a software API that abstracts away
the low-level implementation of the pinwheels and allows developers to work at a higher and more
understandable level. This section describes the different components of the toolkit, discusses how
well they work, and suggests some possible future additions to them.
3.1
Hardware & Structure
The hardware and structure component of the toolkit involves circuitry that gives the pinwheels the
ability to spin at different speeds and a container and mechanism for assembling the pinwheels and
connecting them to the data. Two circuit boards were implemented, one to run an individual
pinwheel and another to interface between the computer serial port and the pinwheels. A container
was created that snugly fit the motor and circuit board. The hardware and pinwheels structure were
designed with fellow Tangible Media Group grad student Phil Frei with the help of Rob Poor of the
Personal Information Architecture (PIA) Group at the Media Lab.
3.1.1
Design Considerations
To create ambient displays that serve their purpose well, the following were design criteria that were
felt to be necessary for the hardware and structure to fulfill.
0
Robustness
An important aspect of ambient displays is their persistency. Thus, the circuitry behind the pinwheels
needed to most importantly be robust. The usefulness of a display is quite dependent on its accuracy
-
*
a constantly failing circuit board would defeat the purposes of the display.
Small and Contained
Regular pinwheels are attached to a thin rod that is almost invisible, so what one mostly sees is the
spinning pinwheel. It was important that adding a motor and a circuit board did not change this
15
effect by adding too much bulk to the pinwheel. This required the board to be small and for the
pinwheel container to hold the parts compactly.
*
Flexible
Many interfaces are possible with pinwheels, so it was necessary for the overall system to be flexible
enough to allow the pinwheels to be arranged in any manner. The most form-limiting factor of an
ambient display is how the pinwheels are connected to the data. Thus, an unrestrictive method for
doing so was needed.
9
General
To allow for future development of other ambient fixtures, we aimed to design the board and
structure to be general enough so other ambient fixtures could be implemented with the same
electrical and mechanical base.
3.1.2
Implementation
3.1.2.1
System Overview
Given the design criteria listed above, the following system structure was chosen:
LI
Serial Data
6
6
Serial Port / Pinwheels
interface circuitry
Serial
Power
Ground
Ground
Power
LSerial
6
Pinwheel
Circuit
Board
Figure 3.1: Overall System Architecture
The pinwheels are driven by instructions that are generated by the Java program and sent through the
serial port. The first circuit board takes in the serial data and bundles it with power and ground
signals to create a six-signal bus that serves as the data source for the pinwheels. The pinwheel circuit
boards are connected to the bus via telephone cables that connect them to each other and connect
16
the interfacing circuit board to the first pinwheel. Each pinwheel listens to the bus and acts only
when an instruction specific to itself passes by on the bus. All other instructions are ignored.
As can be seen from figure 3.1, the bus consists of six signals but only has three distinct signals.
Since this bus is the only source of signals for each pinwheel board, doubling each signal gives each
board a stronger signal, increasing the robustness and stability of each pinwheel. Making the bus
symmetrical prevents human errors such as plugging cables in incorrectly that may cause damage to
the boards.
With this set up, numerous pinwheels can be daisy chained together to form a large ambient display
since a pinwheel merely has to be connected to one other pinwheel to receive data. This set up also
allows for flexibility in design because it imposes little restriction on the way pinwheels are laid out.
3.1.2.2 Serial Port Interface Circuitry
The serial port interface circuitry is the board that resides between the computer and the first
pinwheel. Its main function is to convert the +/-10V serial signals from the computer to the
0-5V
signals used by the microcontroller on the pinwheel circuit board. After doing so, it bundles the serial
signal with power and ground to create the six signal bus described in the previous section.
All communication with the pinwheel boards is done using the RS-232C standard for serial
communication. The most common data format of the RS-232C standard, known as 8N1, is the one
used in this system. It has a start bit, then has 8 bits of data, and ends with a stop bit. There is also no
parity checking. The start bit is a transition from a logical HIGH to a logical LOW while the stop bit
is the opposite, with a transition from a logical LOW to a logical HIGH. Other data formats have a
different number of data bits and more than one stop bit. The other parameter for serial
communication is the baud rate. The pinwheels use a baud rate of 2400, meaning that the time
needed to send one bit is approximately 417gsec. With 8N1, 10 bits are sent for one byte of data so
one byte would take 4.17msec to transmit.
On the microcontroller side, all serial communication signals are in the range of 0-5V. On the RS232 side, the signals range from -10 to 1OV. To perform the conversion between +/-10V and 0-5V,
a MAXIM Max233 chip is used. Usually, a charge pump is used to produce the +/-10V for the RS232 specs. The Max233 includes the whole charge pump in the chip, so no external capacitors are
needed, making it a good solution for converting between the two voltage levels. The reason the RS-
17
232 standard has increased voltages is because of the amount of noise which may result from having
long cables. They are not necessarily needed in all applications, but since it is quite possible that the
pinwheels may be situated quite a ways away from its source computer, having the Max233 will help
reduce the chance of noise causing any problems.
The amount of power used by a display is dependent on the number of pinwheels being driven by
the power source. A minimum of 9V is needed to drive one pinwheel, 10V have successfully run 10
pinwheels, and 12V was required for 20. The original plan was to use a 9V battery for power but that
did not work well because the battery drained out quite quickly. A DC power supply is currently
being used, so the amount of power being fed to a display can easily be adjusted.
3.1.2.3 Pinwheel Circuitry
The pinwheel circuit board is 3.8cmx2.2cm in size and contains three main components. One is a
microcontroller pic chip that reads, interprets, and acts on data from the bus. The second is a bidirectional motor driver that controls the pinwheel motor. Its last component is a voltage regulator
that converts the power signal from the bus, which is of at least 9V, into the appropriate 5V signal
needed by the pic chip and motor driver. The following is a block diagram of the board. The
schematic can be found in appendix A. A picture of the finished board can be found in appendix B.
6/
/Bus0
5V
Rj2Vsrc
Data 05V5VM
Voltage
Gnd
6/
/Bus
RJ12
Re ulator
Gn
VsrC-
INlout-:
IN1 ouT I
:IN2 OUT2
IN20u
PIC12C672 Gn-MLB1638M
Motor
5L
Figure 3.2: Block Diagram of Pinwheel Circuitry
The telephone cable ends plug into the RJ12 connectors, giving the board the necessary bus signals.
Motor Driver
The motor driver we are using is an LB1638 low-saturation bi-directional motor driver for lowvoltage applications. This driver is a good choice in many ways. It is small and its bi-directionality
allows the pinwheel to be spun in both directions, giving an extra dimension that can be used for
representing information. It also has low saturation voltage and drains little current when in standby
mode, which is advantageous since the pinwheels are often not moving.
18
The motor driver takes in two digital input signals and outputs two signals that set the motor in the
mode determined by the following table:
IN 1
H
L
H
L
IN 2
L
H
H
L
OUT 1
H
L
L
OFF
OUT 2
L
H
L
OFF
Mode
Forward
Reverse
Brake
Standby
Figure 3.3: Motor Driver Truth Table
A method called pulse width modulation (PWM) is used on the inputs to vary the motor speeds. It
creates pulses in the input signals. The pulse widths, also known as the duty cycle, can vary from
zero to 100 percent of the waveform period of the signal. The following figure shows IN1 with a
50% duty cycle and IN2 with a 0% duty cycle.
IN 1
IN 2
Mode
Forw
Stdby
Forw
Stdby
Forw
Stdby
Forw
Stdby
Figure 3.4: Sample Motor Behavior
With these inputs, the motor would appear to be spinning at half speed because it is only spinning
intermittently, in this case, only half the time. If IN1 had a 100% duty cycle instead, the motor would
spin forward at full speed. By varying the duty cycle, the motor can be made to spin at different
speeds.
12C672 MicrocontrollerPic Chip
The function of the pic is to read in the serial data, determine if it is an instruction for the particular
pinwheel it resides on and if so, output signals to the motor driver according to the instruction. If the
instruction does not pertain to its pinwheel, the pic takes no action. The pic code was based on the
code Rob Poor wrote for Nami, a project of the PIA group.
Instructions are in the form "**pffbb" where
*
leader character for start of instruction (there may be more than one)
unique character id of the pinwheel which this instruction is for
p
ff
two hex chars representing the speed to associate with IN1 to the motor driver
bb
two hex chars representing the speed to associate with IN2 to the motor driver
19
To distinguish between the pinwheels, each one is given a unique id, which is used to indicate
instructions that are for it. Speed is represented with two hex characters, making the minimum speed
'00' and the maximum speed 'ff'. Setting the speed to be a number x, where 0
x
5 256, will give the
signal a duty cycle of (x/256)(100)%. In other words, the length of the input pulse is directly
proportional to the speed specified in the instruction. This form of instruction does not perform any
error checking, thus it is possible that an instruction will be sent that causes both IN1 and IN2 to be
high, sending the motor into lock mode. It is important that no such instruction is ever sent by the
computer. The software API created as the other half of the pinwheel development toolkit prevents
this from happening by layering the API so that developers do not have to deal with low-level
instructions.
The pinwheels receive instructions through serial communication. Specific details about the serial
communication are described in the serial port interface circuitry section. Here, the method the pic
uses to read in the data will be discussed. This particular microcontroller does not possess serial
hardware so the serial data is read in bit by bit. The code first looks for a start bit, which signifies that
there is data on the serial port. Once a start bit is detected, the data characters are pieced together bit
by bit (8 bits/character). Once a character has been assembled, the pic either checks to see if it is a
leader character '*' or, if it is already in the middle of an instruction, stores the character in its
relevant variable. This serial sampling code is called once every 69.444 gsec.
Once the id part of an instruction has been assembled, the pic immediately checks to see if it is the
same as its own id. If it is, instruction assembly continues and variables are set to the new values
specified by the instruction. If not, the pic goes back to waiting for a start bit.
To create pulsed outputs that become inputs to the motor driver, two counters are kept, one for IN1
and another for IN2. If the counter is greater than zero, then the output is high, otherwise it is low.
The initial values of the counters are the speeds specified by the instructions. They are decremented
every 34.7 psec (if the current value is greater than zero) and reset every (34.7gsec)(256) = 8.89msec.
For example, if the instruction were **A2fOO, INicount would start at 32 and IN2count would start
at 0. INlout is high for (32)(34.7) = 1.11msec, at which time it reaches zero and remains low for the
next (256-32)(34.7) = 7.78msec. INicount is reset to 32 at that time and the whole cycle occurs
again. The resulting INlout is a pulsed signal with a (32/256)(100) = 12.5% duty cycle. IN2count is
always zero, making IN2out always low. INlout and IN2out are fed directly into the motor driver's
IN1 and IN2 inputs, which spin the motor at, in this case, a relatively low speed.
20
Through the combined methods of sampling the serial line, assembling the instructions, setting,
decrementing, and resetting the counters, the microcontroller can successfully read in data from the
serial port and produce outputs to the motor driver that spin the motor at speeds specified by the
instructions.
Voltage Regulator
The components on the board require power signals of two different voltages, one of at least 9V to
run the motor (used by the motor driver) and another of 5V to power the pic and the motor driver.
If both signals were passed in through the bus, then either the bus size would have to increase or the
symmetry of the bus would be lost. Since neither of these should happen, a voltage regulator was
added which converts the power signal from the bus to a 5V signal.
3.1.3
Pinwheel Structure & Assembly
As mentioned above, a small and contained method of assembling the pinwheels was needed. Phil
Frei, with the help of Ali Pearlman designed a 4.28cm x 3.35cm x 2.29cm rectangular container
which compactly holds the motor and the pinwheel circuit board and provides a mechanism for
fastening the paper pinwheel onto the shaft of the motor.
Figure 3.5: Pinwheel Assembly Process
The motor slides into a cylindrical hole in the bottom half of the container and the board fits in on
the top. The wires of the motor fit into a ridge in the back of container and are soldered into two
holes at the back edge of the board. Two plastic screws hold the board in place. When the motor is
inside the container, its shaft protrudes out of a hole in the front of the container. The paper
21
pinwheel slides onto the shaft and is fastened with a shaft collar. Pictures of the pinwheel
components can be found in appendix B.
Given the rectangular shape of the container, the pinwheels can easily be mounted in many
orientations. Other ambient displays can use this same structure because different shapes of different
types of material can be mounted onto the motor shaft.
3.1.4
Evaluation
Overall, a robust and flexible hardware system was built for creating ambient displays. The pinwheel
circuitry has shown to be pretty robust, some pinwheels have been running for over seven months
now and only one breakdown has occurred. The structure for the pinwheels holds all its components
compactly and does not aesthetically add any weight to the pinwheel.
The only non-trivial problem has to do with varying the speed of the motor. Setting the pulse width
to be directly proportional to the speed specified in the instruction does not give 256 distinct speeds.
There are cluster of speeds that seem to make the pinwheel spin at the same speed. There is a clear
distinction between one speed and the one after it up to approximately speed 16, after which the
numbers of speeds in a cluster gradually increase. After speed 128, there is no noticeable difference
between the speeds anymore. One solution to this would be to not map speeds directly to pulse
width, but to map more speeds to shorter pulse widths since those are more distinct from each other.
Spinning generated by longer pulse widths all appear to be the same, so fewer speeds should be
mapped to it. Adding a function in the pic code that maps instruction speeds to an appropriate
counter value, rather than setting the counter value directly to the specified speed, would be a way to
implement this solution.
3.1.5
Future Work
Using the telephone cable as a bus between the pinwheels still imposes some restrictions to the
displays. It requires the pinwheels to be fixed to one position. This will probably be true of most
displays built, but if, for some reason, there was a need to spin the whole pinwheel on its vertical
axis, the telephone cable would eventually get tangled around that axis. The telephone cable also
requires the pinwheels to be relatively near each other so they can be connected together. This would
prevent displays from, for example, being on different floors.
22
A possible replacement for the telephone cables would be to have no cables at all. A wireless ambient
display would not only allow the pinwheels to be placed anywhere but make them even more like the
regular undigitized pinwheels. This solution could be implemented by having the base component
(the serial port interface circuit board) communicate with the pinwheels through RF communication.
Each pinwheel would again listen for instructions for itself and act accordingly.
23
M
3.2
Software API
The software component of the toolkit is an API for pinwheel development. It presents the pinwheel
as a black box to the developer so he/she will not need to have low-level knowledge of how the
pinwheels work. At the simplest level, a developer can control a pinwheel by simply specifying the
speed. The API developed allows users to develop the system at a high level, without having to worry
about the underlying details. It also allows developers to work with sets of pinwheels, giving them yet
an even higher level of abstraction to work with. Additionally, it prevents human error from causing
hardware errors because it keeps developers from working at the level where software bugs could
cause hardware failure.
3.2.1
Design Considerations
To be a truly useful part of the toolkit, the following were design criteria the API needed to fulfill:
0
Reliable
A standard requirement for APIs, reliability means that developers should be able to trust that the
API does what it claims to do. In other words, it should be bug free, or bugs should be clearly
pointed out to users.
*
Configurable
The API should be easy to use and should allow developers to build applications quickly. Having a
hard to use API defeats the purposes of having an API, developers would probably rather work with
low-level details than figure out how to use the API.
0
Platform-independent
Its is very likely that ambient displays will be developed on a range of operating system platforms,
not just on Windows, or on Unix, or on any other specific platform. The API should be usable on
any platform the developer chooses to use.
0
Layered
Developers should be able to work at different levels of implementation. They should be able to
work with a single pinwheel at a time or with a whole set of pinwheels. They should be able to use
the predefined speeds of the pinwheels or create their own set of speeds that is more fitting for their
particular application. The API should allow for working at these various levels.
24
0
Open
There should be no limit on adding functionality. The API should keep users from dealing with lowlevel instructions to the pinwheels but should not impose any restrictions on adding functionality on
top of the already existing API.
*
Flexible
The API should not restrict developers to certain ambient display configurations. In fact, it should
not be dependent on any configuration and should give users full creativity in that aspect.
*
Thread-safe
Threads are now a common occurrence in many applications. Thus, the API should actively support
threads and not cause any threading issues if used in a program that is thread dependent.
3.2.2
The Pinwheel API
Java was used to develop the API. It is platform independent and has a familiar API structure that
the Pinwheel API easily fits into. The Pinwheel API consists of three main classes: 1) SpeedMapping,
2) Pinwheel, and 3) PinwheelSet. This section will discuss the functionality of and motivation behind
each one in turn. The following is a modular dependency diagram of the classes in the API.
Documentation for the API can be found in appendix C.
PinwheelSet
b,
Pinwheel
SpeedMapping
Figure 3.6: Modular Dependency Diagram of API classes
25
M
3.2.2.1
SpeedMapping
As described in the hardware section, the pinwheel can be set to spin at 256 different speeds. These
will be referred to as basic pinwheel speeds. A SpeedMapping allows developers to map speeds to
basic speeds. In other words, a SpeedMapping object acts as a function, f(x), where x = (1.. .s} and
f(x)
= basic pinwheel speed user would like to map to x. For example, a user may want a pinwheel to
spin at speeds 5, 10, 15, and 20 only. A SpeedMapping could then be created that maps speed 1 to
basic speed 5, speed 2 to basic speed 10, 3 to 15, and 4 to 20. Giving a pinwheel this SpeedMapping
allows the user to use incSpeed and decSpeed functions on the pinwheel. Instead of incrementing by
one, it increments up to the basic speed the next speed is mapped to. Having this class also adds
modularity since the developer can easily change the basic speeds the pinwheels spin at without going
into the rest of the code.
The SpeedMapping class is implemented with a java.util.Vector that holds the speeds in increasing
order. It has functions to add and remove speeds, get mapped speeds, and get the mapped speeds
before and after a certain speed.
To cut out complications, speeds must map to basic pinwheel speeds in increasing order, i.e. Vi,j
where i<j and f is the SpeedMapping function, f(i)
f(j). The class takes care of sorting of the
speeds, so users only have to decide which speeds they would like to use. At creation, the set of basic
pinwheel speeds that is passed in is first sorted with an insertion sort algorithm. Inserted speeds are
inserted in the correct position. This may pose to be a limitation but it makes things easier for
developers - they need only to worry about choosing which speeds they'd like to use. The class
currently does not allow users to set a certain speed to be mapped to a certain basic speed because it
might cause the mapped speeds to be out of order. This is not necessarily a bad thing, but in the
interest of simplicity, this functionality was not added to the class. If users wish to create an
unordered SpeedMapping, something that probably will not happen often, there are definitely ways
for them to do it.
3.2.2.2 Pinwheel
The Pinwheel class represents one pinwheel. Each Pinwheel object has to have a unique id, a speed, a
direction, and a java.io.OutputStream that is a handle to the serial port. Additionally, it may have a
SpeedMapping and a name.
26
At the simplest level, a pinwheel object can be created with just a unique char id and an
OutputStream. The pinwheel starts with speed zero, no speed mapping, and default direction
forward. Three other constructors exist with which a Pinwheel can be created with more
information. Get methods exist for getting the id, current speed, SpeedMapping, direction, and name.
Set methods exist for the speed, SpeedMapping, direction and name. In setting the speed, the
argument is an unmapped speed. If a SpeedMapping exists, the argument is mapped to a basic
pinwheel speed that the pinwheel is then set to spin at. If no SpeedMapping exists, the pinwheel is
set directly to spin at the specified speed. If no new direction is specified in setSpeedO, the pinwheel
continues to spin in the same direction. The class also contains incSpeedQ and decSpeedO methods.
Similar to the setSpeedO methods, if a SpeedMapping exists, the speed is incremented or
decremented to the next or previous mapped speed. If no SpeedMapping exists, the speed is
incremented or decremented by one.
The Pinwheel class is a level above direct serial communication with the pinwheels. It was created
mainly to keep the users from having to follow the instruction format for communicating with the
pinwheels. As mentioned in the hardware section, if an instruction was sent that caused both lines for
the motor to be high, the motor would lock and the pinwheel would have to be reset. Since
pinwheels are often situated in a space that is not easily accessible, resetting them is not an easy task.
Encapsulating everything in a Pinwheel object prevents this sort of error.
3.2.2.3
PinwheelSet
A PinwheelSet holds a set of pinwheels. It may optionally have a name and a SpeedMapping which
all pinwheels in the set use, unless a particular Pinwheel's SpeedMapping is set to something else
afterwards. All Pinwheel objects in a set must have an id different from the rest of the Pinwheels in
the set.
A PinwheelSet can be created from a Vector of Pinwheel objects or from none at all. It can also be
created with a start char id, an end char id, and an OutputStream. This constructor creates Pinwheel
objects with ids from the start char to the end char and creates the PinwheelSet with those new
objects. The class acts very much like the Vector class. It has methods to add and remove Pinwheel
objects from a set. As in a Vector, Pinwheels are added one after another to a PinwheelSet, giving the
set an order. They may be removed by id, name, or by object. They may be retrieved from a set by
those three attributes as well. Because the PinwheelSet is ordered, there are methods for retrieving its
first and last Pinwheels and a method to retrieve the next Pinwheel after a given one. The
27
PinwheelSet also provides an easy way for manipulating multiple Pinwheel objects, i.e. they can all be
started and stopped at the same time with one command. The start method can be used to set all
Pinwheels to a certain speed.
In building ambient displays, grouping pinwheels together or having them relate to each other in
some way is a common occurrence. The PinwheelSet class gives developers the ability to work at this
higher level of abstraction. For example, in the stock market application, five pinwheels are allocated
to one stock. Every five seconds, the speed of a pinwheel changes to the speed of the pinwheel
before it. Having the PinwheelSet makes performing that sort of transition much easier.
3.2.3
Evaluation
The goal of the Pinwheel API was to provide developers with an easy way to program contents for
and control ambient displays. It was designed with the criteria listed previously in mind. Standard
verification methods, black box and white box testing, were used to test each class of the API. As
applications were built, the API continued to go through development as more functionality, such as
different methods for retrieving Pinwheels from a PinwheelSet and replacing a Pinwheel in a set, was
added to each class. Regression testing was used to ensure that the additions did not break already
existing functionality. Through the applications that were built, the resulting API has shown to be
reliable and easy to use. The use of Java makes it platform independent and it allows developers to
work at different levels of implementation. Additionally, it places no restrictions on the displays,
either in form or functionality. Developers can build on top of the existing API to create classes for,
for example, arrays or matrices of pinwheels. It has also been used in conjunction with Threads and
has not caused any problems. To conclude, all the criteria set for creating a software toolkit to aid in
the development of ambient displays have been met.
3.2.4
Future Work
The use of Java makes it possible to port this API onto devices other than the computer. If these
devices also support serial communication, running of the ambient displays can be moved entirely off
the computer and onto a much smaller device. An example of such a device is the TINI board. TINI
stands for Tiny Internet Interface and allows one to web-enable just about any piece of electrical
equipment. With the TINI board, all control of the pinwheels can reside on just this board, adding to
the portability of the system. The TINI board has not yet been used because it does not have an
easily visible graphical interface that is needed for the GUI portion of an ambient display. Another
example of a smaller device that can be used and also has a visual interface is the Palm. It supports
28
Java and can also perform serial communication, so ambient displays could be run off of this small
handheld device. A simple GUI component could easily fit onto the Palm interface, completing the
display.
29
4
Applications
The core of this thesis was to study possible application domains for the ambient displays and
investigate the kind of contents and interfaces appropriate for the pinwheels. Several possible
application domains were discussed and ambient displays were built for four of them: 1) Stock
Market Activity, 2) Website Visitor Activity, 3) Natural Wind Movement, and 4) Network Activity.
The following section will first describe the design criteria for an interface of an ambient display.
Then for each application, it will discuss possible contents and interfaces for that domain, the
specific contents that were chosen, the interface that was built, and the implementation and
evaluation of that display.
4.1
Interface Design Criteria
There are many ways pinwheels can be situated in a display. For example, one could use a single
pinwheel to display one bit of information. The pinwheels could also be set up in an array where
each pinwheel represents a point in time, making the whole array a timeline. They could also be set
up as a matrix, where movement in the x and y directions together portray movement over a certain
area. Other possible arrangements include concentric circles or tree structures. Arrangements vary
depending on the application domain. However, all the interfaces of the displays should have the
following characteristics:
0
Representative
The interfaces should most importantly be representative of the data. Since they give information
through subtle changes in movement, sound, and light, there should be a clear difference in its
motions if its data is different. For example, a sudden large difference in data could be reflected as a
sudden large change in the speed of a pinwheel.
0
Understandable
By just looking at a display, most people cannot tell what information is being presented. The
interfaces need to be put in context; given this context, users need to be able to make some
conclusions about the information being portrayed with little explanation. They might not necessarily
get the full picture but the display should at least intuitively make sense to them.
30
*
Aggregately Informative
Each individual ambient fixture represents one bit of information. All together, the interface should
show how each of these bits relates to each other. Thus, each display should not only give disjointed
information but also tie together all the bits and portray the aggregate information as well. An
example of such a display is one that portrays trends in a certain domain.
*
Aesthetically Pleasing
One of the most attractive characteristics of ambient displays is that they are aesthetically pleasing. If
they were not, there would be less of an argument to use them as an alternative to onscreen
interfaces. The pinwheels themselves are simple and elegant and the displays should not destroy this
effect. A bulky display is rather obtrusive, going against the definition of an ambient display.
*
Believable
Perhaps the biggest danger with ambient displays is that they are not believable. Users need to be
convinced that the pinwheels are truly and accurately portraying the data they claim to be mapped to.
4.2
Graphical User Interface
Ambient displays subconsciously give users a sense of events happening outside or away from them.
If users wish to act upon their new awareness, they should have the choice to turn to a graphical
interface that will give them more precise information about the events. Having a graphical interface
as part of an ambient display is beneficial in many ways. Besides giving more specific data, it can also
be used to make the displays more understandable because it can tell users what information is being
presented. It can also convince the user that the physical interface is correctly mapping the data being
monitored. The physical and graphical interfaces coupled should give users a sense of causality. This
is key in making the displays believable.
The graphical interfaces should be designed so that they are easily associated with the physical
interfaces they are complementing. Doing so will make the correlation between the graphical and the
physical interfaces more convincing and will also make it easier for people to understand the displays
and remember their contents.
31
4.3
Application I: Stock Market Activity
The stock market is very dynamic. Stock prices go up and down and large volumes of transactions
occur every second. Additionally, it is something that many people care about. They would all like to
know immediately when the market is crashing or going up tremendously. However, it is difficult for
most to be constantly watching the market because in order to do so, a significant amount of
foreground attention has to be paid to a stock ticker. An ambient display could "watch" the market
for them. For these reasons, stock market activity looked to be a good candidate for using an
ambient display.
Having a display that presents what is happening on the market can peripherally give people a sense
of the information they want about the market without them having to give any foreground attention
to it. If activities on the market were normal, then the pinwheels would probably spin normally,
sometimes on, sometimes off, sometimes fast, sometimes slow. The display could give people a sense
of activity on the market without causing them any disruption. If, however, there were remarkable
activity on the market (i.e. stock prices falling or rising rapidly) and the display reflected this activity
by either having all the pinwheels stop or spin rapidly, the user would very likely notice this and, if he
chooses to, turn his attention to the market. Ambient displays are well suited for this application
domain because displays can be ignored when activity is normal but will call for attention when it is
not. Additionally, if people do want a sense of what is happening on the market, they can find out
quickly by taking a glance at the display.
4.3.1
Possible Contents and Interfaces
4.3.1.1 Stock Price
One possible component of the stock market that could be tracked by the pinwheels is stock prices.
They can go through quite a bit of fluctuation just in the period of a day, as can be seen in the
following chart from the Yahoo! Finance pages.
32
Kana Commun icat ions Inc
28-Apr
1: 27pm
45.00
42.50
40.00
K
37.50
u2A
--------- --------------------__ _
---------
-
35.00
12pm
Prev CIs
10am
2pm
4pm
-----http: //f inance. yahoo. com/
Copyright 2000 Yahoo! Inc.
Figure 4.1: Stock Price Fluctuations in a day
One pinwheel could be used to track the changes in price of one stock. Forward spinning would
represent a positive change while backwards spinning would mean a fall in price. The rate of spin
would be proportional to the amount of change. With this mapping, it would be very easy for users
to take a quick glance at the pinwheel to see how the stock is doing. Constant rapid spinning in the
forward or backward direction would be noticeable and make users aware that the price is going
through a dramatic change. An ambient display could comprise a number of these pinwheels, one for
each stock in one's portfolio. This display would help a user keep track of how all of his investments
in the market are doing.
The arrangement of the pinwheels in this type of display needs to be one that facilitates the user
remembering which pinwheel corresponds to which stock. One possibility is to situate the pinwheels
in an array, with a complementary graphical interface that shows which stock is being tracked by each
pinwheel. The following figure shows such a setup.
Figure 4.2: Possible Stock Portfolio Ambient Display
33
Positioning the pinwheels in an array also calls more attention to the display when one pinwheel
starts to spin much faster than the others. The faster pinwheel stands out, making the change more
noticeable than when there is only one pinwheel in a display.
Another possibility is to give the user the freedom to situate the pinwheels. Since there is no strict
spatial relationship between the stocks, there is no need to keep the pinwheels that represent them in
a certain arrangement. Leaving the design to the users allows them to situate the pinwheels in a way
that would help them remember which pinwheels are mapped to which stocks. For example, they
could group stocks by company type, color code them, or order them by importance, placing the
more important ones closer by. This assumes that the display is primarily being used by only one
user, making it possible for him to personalize the display.
4.3.1.2 Volume of Activity
Another value of the stock market that can be tracked is the volume of activity for a stock, i.e. the
number of shares that are bought and sold for that stock. Again, this value is constantly changing
because both large and small transactions are happening all the time. The following figure shows the
variation in volume over a period of four hours.
Kana Communications Inc
28-Apr 1:27pm
60000
50000
40000
30000
20000 -
t--
-
--
-
--
L - _
------
hi-
10000
Copyright 2000 Yahoo! Inc.
Volume
http://finance. yahoo.com/
Figure 4.3: Volume Activity in a day
One pinwheel can again be used to map volume. However, volume is never negative, so the pinwheel
would only spin in one direction. The speed of the spin would depend on the volume of activity.
Slow or no spinning would indicate little or no activity; rapid spinning would mean the opposite.
Arrangements similar to the ones described in the previous section could also be used with this
content. However, given just one pinwheel, mapping volume is not as meaningful as mapping price
because people care more about the price than about the volume of activity for a stock. A more
interesting statistic that can be mapped with multiple pinwheels is how the volume of activity
fluctuates over time. If a number of pinwheels are each used to map the change in volume over
contiguous intervals of time, a timeline can be created that gives users information about activity over
an extended amount of time.
34
There are a number of ways the pinwheels can be situated to create this sort of timeline. One is to
arrange them in a way similar to that of a conventional graphical timeline, with events in
chronological order from left to right (Figure 4.4a). This means an array of pinwheels, with the
pinwheel tracking the most recent change in volume on the right. Another possibility is to also use
an array arrangement but instead of following the conventional timeline, have the information flow
through the array from left to right (Figure 4.4b). The pinwheel on the far left would then represent
the most current time point. Some time later, its information flows into the pinwheel next to it while
it gets reset to the most current change in volume. One other option is to simulate time by arranging
the pinwheels along the z-axis (Figure 4.4c). In other words, have the most current one in front, with
the others behind it. The further back the pinwheel, the further back in time it represents.
cuttent
past
past
(a)
c)
cument
(b)
Figure 4.4: Possible Timeline Arrangements. The arrow indicates information flow.
Having a timeline display also gives aggregate information about how much stock has been bought
and sold at multiple points of time, as opposed to just one. Given just one point, one cannot tell
from the display if a lot of activity has been happening or if one just happened to look at the display
at a moment of high activity. With multiple points, users can tell with one glance what the trend of
activity has been.
4.3.2
Implemented Display
The main display that was built for the stock market domain is a timeline display. It tracks the
volume of activity for a stock over contiguous intervals of time. We decided to track volume instead
of stock price because on most days, given different points of time, there is usually a much bigger
fluctuation in activity volume than in price. We were also interested in looking into the different
interfaces possible for tracking multiple points of time. Having time as a component of the display
makes the display aggregately informative as well.
35
The display built consists of two arrays of pinwheels, each made up of five pinwheels, with each
pinwheel mapping the amount of stock bought and sold in a 5 second interval for one stock. In one
array, information is traveling from left to right, so the leftmost pinwheel shows the amount of
activity in the past 5 seconds, the one to its right shows activity in the past 5-10 seconds, then 15-20,
then 20-25, and lastly, the rightmost pinwheel represents activity in the past 25-30 seconds. The
second array is the opposite, with information flowing from right to left. The pinwheels are updated
every five seconds.
Users choose which stocks they wish to track by inputting them on an initial GUI. They may also
track total activity on the NASDAQ or NYSE. The main accompanying GUI shows 1) the average
daily volume, 2) the time of the last reading, 3) the volume at the last reading, and 4) a bar graph
representation of the pinwheels. Each pinwheel corresponds to a bar whose height is the speed of
the pinwheel and each bar is labeled with the amount of change that it represents. From this GUI,
people can tell how much change occurred in one time interval and how the pinwheel speed matches
that change.
4.3.3
Implementation Details
4.3.3.1 Retrieving Data
All stock information is read off the Yahoo! Finance web pages located at http://finance.yahoo.com.
The Yahoo! pages have many views, e.g. DayWatch view which has the price the stock was last
traded at, the change in price from the previous day closing, the volume of stock bought and sold
today, the average daily volume, the opening price, and the day's range of prices. This view suited our
needs because we only needed the day's volume and the average daily volume (explained later in this
section). An example web page of this view can be seen at
http://finance.yahoo.com/q?s=kana&d=v2. In the program, such a web page is read in as a
java.net.URL object. The URL is parsed and the relevant data, mainly the current volume and the
average daily volume, is read in.
4.3.3.2 Mapping Volume Changes to Pinwheel Speeds
Seven basic pinwheels speeds were chosen for a pinwheel, 0 and 4-9. A SpeedMapping was created
that mapped speeds 0-6 to them. A wide range of speeds was not selected because it was important
that each speed be distinct from the others. Thus, speeds that were too similar were not picked.
36
Mapping the change in volume to a pinwheel speed was not as simple as it first seemed to be. The
problem is that there is a finite set of speeds for the pinwheels, while there are an infinite number of
possible changes in volume. In addition, the volume of activity that occurs for one stock can be very
different from the volume for another stock, so it would be difficult to come up with a strict
definition for what is a large change and what is a small change that would apply to all stocks. Thus, a
different approach was used that involves using the average daily volume. By determining how much
change usually occurs every five seconds for a stock, we have a base to which current changes can be
compared. The following snip of code determines the speed the pinwheel should spin at depending
on the change:
// determine how much activity there is on average during timeDifference
// timeDifference * (average volume / (6.5 hours * 60 minutes * 60 seconds))
double currentAverage = timeDifference * average / (6.5 * 60 * 60);
// normalize the amount of activity to the average amount to get a ratio between
/
and 6
double ratio = (currentVolume - lastVolume)/(2.0 * currentAverage) * 6.0;
newspeed = (int)ratio;
if (newspeed > 6) newspeed = 6;
timeDfference is the amount of time that has elapsed since the last reading. This is normally five
seconds, but because of network lag, it is sometimes more. The following chart shows how the
volume changes are mapped to the pinwheel speeds:
speed
6
5
0~
4
---
3
2
0
-
1/3
ave
2/3
ave
4/3
ave
532ve
change in
volune
ave
Figure 4.5: Mapping volume changes to pinwheel speeds
37
4.3.3.3
Creating the Timeline Effect
Once the new speed has been determined, the following code is used to update the pinwheels:
Pinwheel currentpw = pinwheelSet.getFirstPinwheeo;
while (currentpw != pinwheelSet.getLastPinwheelo) {
Pinwheel nextPinwheel = pinwheelSet.getNextPinwheel(currentpw);
currentpw.setSpeed(nextPinwheel.getSpeedo);
currentpw = nextPinwheel;
}
pinwheelSet.getLastPinwheeQ.setSpeed(newspeed);
The speed of each pinwheel is set to the speed of the next pinwheel in the set. The speed of the last
pinwheel then set to the newest speed. Thus, the last pinwheel in the set corresponds to the pinwheel
mapping the most current point in time and the first pinwheel corresponds to the point furthest back
in time. This achieves the timeline effect.
4.3.3.4 Pinwheels Control Program
The software implementation of the displays has three main classes: 1) StockMarket, 2) StockTracker,
and 3) TrackerGUL.
StockMarket
The StockMarket class starts the program by first initiating serial communication with the pinwheels.
This is accomplished using the javax.comm. serial communications package. The program opens a
connection between serial port COM1 on the computer and the pinwheels and sets the following
parameters to their necessary values:
baud rate = 2400
data bits = 8
stop bits = 1
parity = none
After serial initialization, the following GUI is brought up to let users choose which stocks they wish
to track.
Figure 4.6: Initial Stock Tracker GUI
38
As can be seen from the GUI, four stocks may be tracked at a time. Since only two arrays of
pinwheels were built, only two stocks were tracked at any given time in our display. As part of
initialization, the program creates a PinwheelSet of five Pinwheel objects for each array.
StockTracker
When the user chooses to start tracking the activity of a stock, a thread is created that is a
StockTracker object. The StockTracker class extends java's Thread class and its objects are created
with a stock name and a PinwheelSet. The StockTracker object reads, interprets, and maps the data
to the Pinwheels in the PinwheelSet. It uses a javax.swing.Timer object to repeat the process of
parsing the url and changing the speed of the pinwheels once every five seconds.
TrackerGUI
The last module in the implementation of this display is the TrackerGUI, the graphical interface that
accompanies the display and gives more information. The first GUI built only gave three pieces of
information: 1) the average daily volume, 2) the time of the last reading, and 3) the volume at the last
reading. This GUI was not convincing enough because it did not show what kind of data was driving
the pinwheels. From the GUI, people could guess that the information being mapped was volumerelated but there was nothing that suggested a strong correspondence between the data and the
physical display. Thus, a second GUI was built that also includes a bar graph representation that
shows how the pinwheels' speeds correspond to the change in volume during one time interval. This
GUI, shown below, is currently being used. One is created for each stock being tracked.
Figure 4.7: Individual Stock Tracker GUI
39
4.3.4
Discussion
An ambient display was created that successfully tracks the amount of activity on the stock market.
Figure 4.8: Close up of Single Array
Upon being told that it tracks stock market activity, most people's immediate reaction was to assume
that it was tracking stock price and that each pinwheel corresponds to one stock. This was because
the forward and backwards motion of the pinwheel map naturally to the rise and fall of stock price.
One of the most common questions about the display was about our choice of tracking volume
changes rather than stock price. This choice stemmed from our theory that active displays are more
interesting than mostly stationary displays. Since volume change varies much more than stock price
from one five second interval to the next, tracking it seemed more likely to produce an active display.
However, it seems that for the general user, a display that tracks stock price is more useful than one
that tracks volume changes. People care more about whether stock price is currently going up than
about how much stock is currently being bought and sold. Given this, we still chose to track volume
because we were interested in how displays could be aggregately informative. In this case, we wanted
to look at changes over time. It is possible to track stock price changes over time, but since stock
price does not change that much every five seconds, a display tracking stock price would be normally
quiet, with one pinwheel spinning occasionally. From our experience, such a display would also
probably be better at catching a user's attention because a quiet display that suddenly becomes active
is quite noticeable. However, we often need to demonstrate our projects, so we needed an active
display that would better serve as a proof of concept. For these reasons, we chose to track volume
changes instead of stock price. In the future, user evaluations should be performed on both displays
to determine which is more informative and which is more useful to a stock market conscious
individual.
40
Figure 4.9: Stock Market Ambient Display
As stated before, the two pinwheel arrays of the display had information flowing in different
directions; the one with it flowing from left to right was more natural than the other was. This may
have been because in our society, we are used to a left to right flow, since we read in that direction.
However, an interesting flow is created if the two arrays are combined. Because the right to left array
is situated behind the left to right array, information can be thought of as flowing through the closer
array from left to right, then curving to the back and flowing through the further array from right to
left.
A potential problem with this display is its lack of fiduciary marks. To a passerby, the physical
interface itself is simply arrays of pinwheels spinning randomly. There is nothing that indicates to
them what stock is being tracked and how it is being tracked. Even after they are told that the display
is tracking volume changes over time, they may be likely to forget the length of the time intervals and
in what direction information is flowing. Thus, it seems that in order for this display to be used in a
social space, the pinwheel arrays need to be put in context. However, doing so will most likely
require adding labels that can take away from the abstractness and cause bulkiness. The other option
is to make the display a more personal one, intended only for a single user or a number of users that
will interact with it and frequently have it in their periphery. The lack of some sort of legend or
labeling might then be acceptable because the small number of users can learn and get to know the
display.
41
-M
From my experience with the display, I have found that while it is normally ignored, it does
successfully grab attention when the pinwheels stop moving or are all spinning quickly. These
situations usually induce looking at the graphical interface to see what kinds of changes in volume are
occurring. Surprisingly, the noise caused by the spinning of the pinwheels is often more of an
indication of the state of the display than the visible motion of the pinwheels. Our preliminary
findings suggest that this type of display can be useful as a personal display.
42
4.4
Application II: Website Visitor Activity
People who have sites on the Internet are often interested in statistics about visits to their website,
e.g. who is visiting their website and how often it is visited. There are many free and paid services
that accumulate and keep these statistics but they are mostly web based and require the user to
actively look them up. This is information that people care about but cannot afford to spend much
time on, making it a seemingly ideal situation for using an ambient display.
An opportunity for such a display arose in collaboration with Dr. Muriel Waldvogel and Professor
Jeffrey Huang who are architecting and building the Swisshouse, a digital consulate for science and
technology located in Cambridge, MA. The Swisshouse combines physical facilities and digital
infrastructures to support a new diplomacy in advanced research and education [20]. Dr. Waldvogel
and Professor Huang hope to make it a place for Swiss scientists, researchers, and others to meet and
discuss their ideas, both physically and virtually. Their web site is an integral part of this because it
allows people to remotely visit the Swisshouse. By using an ambient display to track hits to their site,
the people who are physically at the Swisshouse can get a real sense of the activity on the website
through the physical motion of the pinwheels. It would be as if the actions of the remote visitors are
physically causing the pinwheels to move. On top of that, such a display could also give actual
visitors of the Swisshouse a sense of the number of virtual visitors who are there with them. The
Swisshouse website will most likely be one that is visited often, so an ambient display for hits to it
would be constantly active, making it more interesting than one for a site that is only occasionally
visited.
4.4.1
Possible Contents and Interfaces
A few possibilities for mapping visitor activity to the pinwheels were discussed. The possible
interfaces varied depending on the contents.
4.4.1.1
Geographical Location
Mapping visitors to pinwheels based on geographical location would give one a sense of where the
visitors were coming from. For people in the Swisshouse, a display with such a mapping could tell
them about the people they are interacting with. Since geography is often blurred by the Internet
these days, this would be more meaningful and would also provide a means for showing where there
is activity in the world. One could see if the hit patterns around the world matched the times when
different parts of the world are awake.
43
This sort of mapping is a little difficult because there needs to be a way of dividing the world up into
different parts. One possibility is to split it into four: north, south, east, and west. This is also tricky
because one cannot easily divide the globe up into these four segments. Since the display is meant to
give people a sense of activity in other parts of the world, it seems reasonable to set north, south,
east, and west portions of the globe relative to the location of the display. This would mean taking a
2D map of the world, centering it on Cambridge, MA and defining the regions accordingly. The
speed of the pinwheel for a certain region would then depend on the number of visitors to the site
from that region.
The pinwheel arrangement for this type of setup would need to clearly show which pinwheel is
mapped to which part of the world. This could be done by situating the pinwheels so that they are in
a north, south, east, and west configuration, as shown below. Each pinwheel would be in the position
that corresponds to the region they are representing. In order for this arrangement to work well
though, either the viewers of the display would have to be situated in a specific place or the
pinwheels would have to be spread throughout the space. The latter might weaken the effect of the
display since spreading the pinwheels apart would make it more difficult to make comparisons that
show that one area is more active than another.
Figure 4.10: Directionally Arranged Pinwheels
The world can also be divided by time zones. With this division, having one pinwheel per time zone
would clearly show if high activity corresponded to certain parts of the day for all parts of the world.
The pinwheels could be situated in a circle, following the positions of their respective time zones.
However, it would be difficult to tell which pinwheel represented which time zone, unless each was
labeled. Labeling the pinwheels adds text to the display, which tends to take away from its
abstractness, so it was generally avoided if possible. An alternative arrangement, shown below, is an
44
array of 24 pinwheels (for the 24 time zones), laid out on top of a map of the world, such that each
pinwheel falls in the time zone it represents. People are familiar with maps, making them an addition
that can be used without explanation. Such an arrangement is similar to the wall of clocks one often
sees at airports that shows the time in different cities of the world. It shows what area each pinwheel
is mapping and allows for comparisons of activity in different parts of the world.
Figure 4.11: Time Zone Arrangement
Instead of placing a pinwheel in a time zone on the map, they could also be placed in different
locations on the map to track activity from those different locations. This type of display would
work well because of its clarity in showing what each pinwheel is mapping. It is quite easy to make
the connection that a pinwheel situated on Geneva is tracking the number of visitors from Geneva.
There would be little doubt in a user's mind about the contents of each individual pinwheel. In the
case of the Swisshouse, it would make sense to place pinwheels on locations of people it wishes to
bring together with the Swisshouse, e.g. places in Switzerland, key research labs in the US, academic
institutions around the world, etc. An interactive display could even give users the ability to move the
pinwheels around to different locations, giving them full control of the information they see. This
could potentially make the display more believable. It would also make it easy to reconfigure the
display if, for example, a large research institution were to move locations.
4.4.1.2 Domain Name
The difficulty in mapping visitors by geographical location is in actually getting a visitor's
geographical location. A visitor's IP address is the most location relevant information that can be
gotten about a visitor. With an IP address, we can get the host name of the computer, which often
45
indicates what country the computer is located in. However, there are many .com or .edu or .org
hostnames that could be anywhere in the world. Frequent travelers also pose a problem because they
dial up and use the same hostname everywhere they go.
Since it is currently technically very difficult to pinpoint the exact location of a visitor to the website,
an alternative mapping is to map the visitors to the pinwheels based on domain names. For example,
with five pinwheels, all .com visitors could be mapped to one pinwheel, .edu to another, .org to
another, .gov to another, and all others to the last one. This display would give a sense of what kind
of people are visiting the web site, whether they are corporate employees, government officials,
members of academia, etc.
Since there is no spatial relationship between the contents of these pinwheels, a clean and simple
array arrangement would work best. But we would then again have the problem of users not being
able to easily identify which domain name each pinwheel is tracking. As an alternative to labeling, a
pictorial representation of the domain name could be used as a background for a pinwheel to indicate
which domain name it is tracking. Using a pictorial representation does not add text but is a potential
source of confusion because it is not always clear what a picture is representing.
4.4.1.3 Web Page
Yet another approach to mapping visitors to pinwheels would be to have the pinwheels present the
number of visitors to a certain page or group of pages. The Swisshouse website, like many other web
sites, is split into several sections. If one pinwheel were used to track one section of the website and
each pinwheel's rate of spin were directly proportional to the number of visitors to its section, the
physical display would show what sections of the site are most frequently visited. An accompanying
GUI could show the domain names of the visitors, making the overall display one that presents both
which sections of the site are popular and what kind of people are visiting the site. With these
contents, there is no particular spatial relationship between the sections of the website so again, a
clean and simple array would work best. The display would then give users a sense of how many
people are visiting a section of the site, allow for easy comparison of activity at different parts of the
site, and aggregately inform people whether or not there are virtual visitors to the Swisshouse. To put
the array in context, images that are used in the website as icons for certain sections can be used to
label the pinwheels. If these images are obvious icons of the sections, it would not be hard for users
to associate each pinwheel with its corresponding section of the website.
46
An alternative web page based content relies on the fact that web sites often have a tree like
structure. If one pinwheel mapped activity to one page, the pinwheels could be situated in the same
tree arrangement its pages are in. The display could then be used to track one visitor's path through
the web site and provide a means for studying how people normally navigate through the site. This
type of display, however, is no longer ambient in the sense that information cannot be gathered from
it peripherally. Peripherally, the most information one would get is that there is a visitor to the site.
But to get all the intended information, one would have to watch the pinwheels rather closely.
Figure 4.12: Tree Structure Arrangement
4.4.2
Implemented Display
The display built for this application domain tracks visitors by web page sections. The pinwheels are
situated in an array and control of which pinwheel is being mapped to which section is given to the
user. An array was chosen because it was clean and simple. The Swisshouse has a busy environment
and having a rather complicated arrangement amongst all the activity would most likely cause
confusion. The accompanying GUI allows users to choose a section to track and which pinwheel to
track it with. Users can change this section to pinwheel mapping at any time. The GUI also shows
the visitors to a certain section, their IP addresses, and the time they entered that section. The
pinwheels are updated whenever a visitor enters or exits a section of the website.
47
4.4.3
Implementation Details
4.4.3.1
Overall System Architecture
The most challenging part of the implementation was keeping track of the visitors to the different
areas of the site. This was accomplished with the use of PostgreSQL and AOLserver. PostgreSQL is
an object-relational DBMS that supports almost all SQL constructs. It is the database being used to
track visitor activity. With PostgreSQL, we can easily determine how many visitors are on a page,
where they are from, how often they have visited the page, and even where they were before.
AOLserver is a multithreaded, Tcl-enabled dynamic web server that is often used for large scale,
dynamic web sites. It supports AOLserver Dynamic Pages (ADPs) that allow embedding of Tcl
scripts in HTML pages. With the Tcl scripts, visitor activity can be inserted into and updated on the
database. Tcl has several functions, including getting the IP address of the visitor and the hostname
from an IP address, both of which are useful for this application. Both PostgreSQL and the
AOLserver are installed on grasp.media.mit.edu, a local lab machine.
We can successfully track visitors using PostgreSQL and AOLserver. The Java program that controls
the pinwheels accesses information from the database using JDBCTM. The following diagram shows
the architecture of the system.
grasp.me dia.mitedu
AOLserver:
Dynamlic
Pinwheels
C
COntrol
Progzamn
+-+
--
Pages
---
AOLsevei
PostgreSQL
_
(T a
HTM
pages
+
Figure 4.13: Website Tracker System Architecture
4.4.3.2 Tracking Visitors
It would be ideal if the Swisshouse web site were built using AOLserver because the Tcl scripts could
be embedded into the web pages, allowing them to directly communicate with the database and
thereby removing the middle layer in the diagram shown above. However, this should not be a
48
restriction, so the system that was built does not rely on the Swisshouse site using the AOLserver.
This also makes the system a black box that can easily be used to track other web sites. Since the
Swisshouse web site has not yet been completed, a prototype web site was built for the system. In
this prototype, the web site was broken down into six different sections, each consisting of one page.
It would be easy to increase the sections to ones of multiple pages.
Two tables are stored in the database, websitevisitors, and hostnames. The former tracks visitor
activity and the latter keeps a table of IP addresses and their corresponding hostnames.
The middle layer between the web pages and the database consists of two ADP pages, enter.adp and
exit.adp, that are called by a web page when it is loaded and exited, respectively. enter.adp takes one
argument, the page being entered entepage, and performs the following sql statement:
insert into websitevisitors values ('$ipaddress', '$enterpage', '$time', 't');
This creates an entry in the database indicating that visitor with IP address ipaddress entered page
enterpage at time time and is onsite (t for true). It also checks to see if Opaddressis in the hostnames table
and adds an entry for it if it is not. A call to enter.adpis embedded in the page to be tracked with the
following html code:
<body background=http://grasp /sren/enter.adp?enterpage=main bgcolor=white>.
The call is invisible to the user. exit.adp takes in two arguments, the page being exited, exipage, and the
page the visitor wishes to go to, topage. It then performs the following sql statement:
update websitevisitors set onsite='f'
where ipaddress ='$ipaddress' and page='$exitpage' and onsite='t';
This simply takes the entry created when this page was entered and sets the onsite variable to 'f',
indicating that the visitor has left the page. An alternative would have been to remove the entry
completely, but leaving it in gives us a history of the website activity. After updating the database, the
page redirects the browser to topage. While a normal link out of an html page looks like the following:
<a href="http://feng.media.mit.edu/background.html">Background</a>,
links are now coded like so to incorporate exit.adp:
<a href="http://grasp/sren/exit.adpexitpage=main&
topage=http://feng.media.mit.edu/background.html">Background</a>
Tracking another website would simply entail incorporating calls to the two ADP pages into the html
pages.
49
4.4.3.3 Mapping Number of Visitors to Pinwheel Speeds
With speeds, we again have the problem of mapping an infinite number of values (possible number
of visitors) to a finite number of speeds. In this case, it seems fair to say that if there are, for example,
100 visitors to a page, then it doesn't really matter if the number becomes 101. However, if there is
only one visitor to a page, then increasing to five visitors means a 5-fold increase, which is rather
significant. Thus, the following mapping was chosen:
Number of Visitors (x)
0
15 x <5
55 x <25
x <125
25
1255 x <625
6255 x <1000
1000
x
Speed
0
1
2
3
4
5
6
Figure 4.14: Mapping number of visitors to pinwheel speeds
There is a five-fold increase associated with each speed increment. These numbers were set assuming
that normal activity on the web site will be in the hundreds and that the number of visitors will
sometimes reach 1000. Once the Swisshouse web site is complete, the numbers should be
recalibrated to ones that make sense, given the usual amount of activity on the site.
4.4.3.4 Pinwheels Control Program
The Java program that controls the pinwheels uses JDBCTm to access the tables in the database and
retrieve information about the visitors who are currently on a page (or in the future, a section of the
site). It also has the GUI component of the display that shows where the visitors on the different
pages are from. The main classes of the program are 1) WebSiteTracking, 2) WebSiteInfoPanel, 3)
PageTracker, and 4) PinwheelInfoFrame.
WebSiteTracking
The WebSiteTracking class is the main class that performs program initialization. This entails first
creating a connection with the database. Then serial communications are set up, in the same way as
in the Stock Market Activity application. Lastly, the main GUI, consisting mainly of the
WebSiteInfoPanel, is created. From this GUI, one can choose to start tracking a new page with a
pinwheel, edit the tracker for a page, or stop tracking a page.
50
IIng-
Figure 4.15: Initial Website Tracker GUI
PinwheelInfoFrameandPageTracker
Choosing one of the first two options brings up the following PinwheelInfoFrame object.
Figure 4.16: GUI for Configuring Website Tracker
From it, one can set a page to track and which pinwheel to track it with. PageTracker objects are
thread objects that hold this information - each object checks activity levels for a certain page and
reflects it in the specified pinwheel and in the GUI. Upon closing of the PinwheelInfoFrame, a new
PageTracker object is created if a new page is being tracked (option 1) and values for an existing
PageTracker are reset if the tracker was edited (option 2).
Every two seconds, a PageTracker object checks the websitevisitors table in the database to
determine how many visitors are on its page using the following sql statement:
select * from websitevisitors where page= page name and onsite = 't'
The PageTracker object then determines the new pinwheel speed and updates the pinwheels
accordingly.
WebSiteInfoPanel
Each PageTracker object also has a GUI that lists the IP addresses of visitors to the page, their
corresponding hostnames (determined using the hostnames table of the database) and the time they
51
entered the site. This GUI is added to the WebSitelnfoPanel and is shown when the user selects the
page from the list of pages being tracked, as shown in the figure below.
Currenly Tracking
applications
backrond
2000-1615-47530
1,rfemedamtedu
18.
myediiamitiedu
___2_._
18.85-28.63
stomymedia.mitedu
185 28,63
smymedia mit edu
18.85-28.63
s
19.8.28.053
stormy media mt-9du
18.85 2863
stormy media mit edu
stomrmedia mitedu
18.8528.63
18,858.63
media mil.edu
stommedia
mlt edu
2000-03-1615:47:30.0
2000-03-1610:20:19.0
2000-03-1618-50:56,0
2000-03-1618-51:01.0
2000-03-16_18 51:14.0
2000-03-17 1534:19 0
2000-03-1715 34:37 0
2000-03-1715 58:09.0
Figure 4.17: GUI Displaying Visitor Information
This GUI gives people full control of which pinwheel is mapped to which section of the website.
Thus, users can choose what information they want to see and how they want to see it.
4.4.4
Location
An important characteristic of an ambient display that arose in discussions about the Swisshouse is
the location of the display. In the room the pinwheels are to be in, Dr. Waldvogel and Professor
Huang plan on having one wall that is a screen projecting things from the web. It was first thought
that the pinwheels could be placed in front of that screen, making it look as if the pinwheels were
being spun by a bit wind that is coming from the screen. However, this might cause confusion
between what the display is presenting and what the screen is projecting, since the two are not
related. Superimposing the display on a possibly active big screen might also lessen the effect of the
spinning pinwheels.
The point of the display is to give visitors of the Swisshouse a sense of the virtual visitors that are
there with them. Thus, its current tentative placement (since the House is not complete) is as a portal
to another area of the house where virtual communication will take place, e.g. video conferencing,
distant collaboration, etc. The pinwheels will line the entrance, giving users who are about to enter
that portion of the house a sense of how many people will also be in that area with them. A blueprint
of the Swiss House, with the pinwheels' locations marked can be found in appendix D.
52
4.4.5
Discussion
Unfortunately, the Swisshouse is not yet complete so the display for this application is not fully in
place and functioning. However, the prototype built is complete and will be easy to port to use with
the Swisshouse web site. The following is a rendering of the Swisshouse with the pinwheels.
Figure 4.18: Rendering of Ambient Display in Swisshouse
The only problem with implementation was that the tracking system relied on the ADPs on grasp
being called every time a visitor entered a page. However, this does not always happen because
browsers often cache pages. Thus, when a user goes back to a page, the database is not informed.
Besides this, the system successfully tracks visitor activity to a web site.
In the discussion of possible interfaces for the Swisshouse ambient display, we felt that an array
arrangement was good because of its simplicity. From our experience, arrays seem to be the best
arrangement for data that does not have obvious spatial relationships. With this type of data, it is
better to use a simple array instead of trying to convey some subtle spatial relationship with a
53
complicated layout. However, as discussed in the stock market application, the problem with arrays is
that it is not obvious what the contents of each individual pinwheel are. In order for the display to be
one that is informative to users that are not in frequent contact with it, as is the case with the
Swisshouse, it must be placed in context somehow, preferably by an addition to the physical interface
that is not bulky or distracting. Having the GUI also accomplishes the task but does so only if users
go to look at it.
54
4.5
Application III: Natural Wind Movement
The movements of natural phenomenons such as solar wind, hurricanes, and pressure systems are
hard for one to envision because of their intangibility and large span. Ambient displays can be used in
this type of application domain to present information that is normally hard to visualize. They will
not only inform people of natural events that are important to them but also give people a sense of
what is happening over their area in a global sense.
Wind movement is such a natural event. Many people, from the professional aviator to the casual
sailor, care about the speed and direction of the wind in their area. When they are outside, most
people can feel the wind direction at one certain point but do not know how it changes or flows over
a whole area. The information people normally get about the wind is in the form of numbers and
letters; nothing gives them a real sense of how the wind is blowing. An ambient display, on the other
hand, could possibly do so.
4.5.1
Possible Contents and Interfaces
4.5.1.1
Wind at a Single Location
An ambient display for tracking wind movement should firstly show what direction the wind is
blowing and secondly give a sense of the strength of the wind. One idea is to have the pinwheels spin
as if they were being blown by the wind. However, a pinwheel can only spin in two directions and
winds of different directions, for example a wind from the north and one from the northeast, will
likely cause it to spin in the same manner. Thus, using only the spin will not be enough to present
both pieces of information.
Since wind is directional, it can be thought of as flowing through a space. If many pinwheels
occupied this space, each would spin at a different time depending on when the wind flowed through
its particular area. Thus, another possible display would comprise a matrix of pinwheels, each
spinning at different times to portray wind flowing through the area. For example, a natural wind
from the northwest would create a "wind" that flows through a matrix of pinwheels and causes them
to spin in the pattern shown below.
55
1
Time Point 2
Time Point 3
Time Point 4
Time Point
Figure 4.19
Ambient display showing
wind direction at various
time points. The pattern is
repeated to convey wind
blowing in from the
northwest.
Time Point 5
A wave effect is created that shows the direction of the natural wind. The speed of the natural wind
can be portrayed either by the speed at which the "wind" flows through the matrix or by the speed at
which the pinwheels are spinning. The former would clearly show how fast the wind is blowing while
56
the latter would give more of a sense of the strength of the wind. A combination of both could be
used.
With this type of display, the pinwheels could be arranged in many ways over an area. One option is
an organized matrix; the resulting "wind" wave would then travel through the area very
systematically. Another option is to scatter clusters of pinwheels throughout the area. As opposed to
the former option, the timing of the spinning would not be as rigid. There would be pauses of
different lengths between the spinning clusters. The matrix would probably give a much clearer
indication of the direction of the wind while the scattered clusters would be more aesthetically
interesting and natural because of their higher entropy.
4.5.1.2 Multiple Wind Systems
Given a large area, many wind systems could be moving over it at the same time. An area of
pinwheels could again be used to show how all the wind systems are interacting. A wave effect can
also be used, but instead of moving through the area at a consistent angle, the wave could change
directions somewhere in the area, to portray that another wind system is more dominant in the area.
Another possible display for showing multiple wind forces does not use the wave effect. Instead, the
pinwheels spin at different speeds depending on how the different wind forces on them balance out.
Thus, in this collection of pinwheels, some would spin quickly, some slowly, and some not at all. This
display would not give a sense of wind direction but would give a sense of the strength of the wind at
different points of an area. Superimposing the physical display on top of a map of the area that is
being tracked would place the pinwheels in context and make it easy for users to tell what the
strength of the wind is at a certain location.
Tracking multiple wind forces means that the ambient display is tracking wind movement over a
large area since there is only one wind speed and direction at a given geographical point. Only from a
more global view can one see the different wind systems moving over a geographical area. Because of
this, tracking multiple wind forces would require a large collection of pinwheels. With only a few, it
would be difficult to see a change in wind direction or to compare wind strengths in different areas.
4.5.2
Implemented Display
For this application domain, wind at a single location was tracked. A 3x3 matrix was built and the
wave effect described previously was used to portray wind direction and speed. We decided to build
57
this type of display because it was more appropriate than one for multiple wind systems, given the
number of pinwheels we have. The program reads in wind speed and direction from the web every
hour and updates the pinwheels accordingly.
4.5.3
Implementation Details
4.5.3.1 Wave Effect Algorithm
The most difficult part of this implementation was creating the wave effect. This required
determining the time at which each pinwheel should start and stop spinning. To do so, the wind is
represented as a vector that enters the pinwheel area from the top left corner and travels through the
area at the angle of the natural wind it is representing. As it travels, it enters and exits pinwheels,
causing them to start and stop spinning respectively. The wave effect algorithm first calculates the
distance the vector must travel before it hits each pinwheel. This distance, enterLength, is defined as
the distance from point a to point b in the diagram below. The distance from point a to point c is the
distance, exit~ength, the vector must travel before it exits a pinwheel.
a windetection
(0,0)
d
)c+
theta
d
a-
2
ZC
if (d: wjndDjrectjon)
theta = windDirect ion - d
else
theta = d - windDirection
enterLength = z * cos(theta)
enterLength
b
+ pinwheel width,
y + pinwheel height)
e3itLength
C
Figure 4.20: Vector Calculations for Wave Effect Algorithm
The equations on the right side of the diagram show the calculations involved in determining
enterLength. To determine exitLength, x andy are substituted by x + pinwheel width andy + pinwheel height,
58
respectively, and the same calculations are made. With this set up, the pinwheels can be anywhere in
the area, not in a fixed order. Thus, both the matrix of pinwheels and the scattered clusters of
pinwheels can be implemented as an interface.
The speed at which the wave travels through the area is directly proportional to the speed of the
natural wind it is representing. Given the speed of the natural wind, natspeed, a timer is created that
goes off once every 2000/natpeedmsecs. A counter, count, is used that increments by one every time
the timer goes off. count is set to zero every time the wind vector first enters the pinwheel area and
keeps counting up until the vector has traveled through the whole area. When count = enterLength for a
pinwheel, the pinwheel starts spinning; when count = exitLength for a pinwheel, it stops spinning.
Thus, the time between counter increments is the time it takes for the wind vector to travel one unit
of distance through the pinwheel area. The following is a graph of this time vs. natural wind speed
for speeds between 0 and 50 mph.
2500 2000 1500 1000 500 00
10
30
20
40
50
60
Speed (mph)
Figure 4.21: Counter Increment Time vs. Natural Wind Speed
As can be seen from the graph, the faster the wind speed, the shorter the time increment. This
shortens the amount of time elapsed before the wind reaches a pinwheel, shortening the overall time
needed for the wind to travel through the area. Thus, the wave appears to be moving faster through
the area. After the wind has traveled through the area, count is reset to zero and the wind enters the
area again from the top left corner.
4.5.3.2 Retrieving Data
Currently, weather is being read in from The Weather Channel at http:/ /www.weather.com. It
provides relatively current reports of weather all over the world. For example,
59
http:/ /www.weather.com/weather/us /zips/02139.html, shows weather for Cambridge, MA.
Weather conditions are updated once an hour and wind conditions are reported in the format: speed
mph from the direction, e.g. 13 mph from the Southwest. The URL is read in and parsed in the same
way the Stock Market Activity application reads in its URLs. The speed is used as it is, while the
direction is converted into a numerical angle between 00 and 3600. The following figure shows the
conversion.
S
90
A
SW
,,45
SE
135
EIS8O
,OW
225/
NE
315
NW
270
N
Figure 4.22: Numerical Angles of Directions
4.5.3.3 Pinwheels Control Program
The Java program that controls the pinwheels consists of two main classes: 1) WindMovement and
2) WindPinwheel.
WindPinwheel
A WindPinwheel object contains a Pinwheel, (x,y) coordinates for the pinwheel's location in the area,
and the enterLength, and exitLength for the pinwheel. The (x,y) coordinates are used to calculate
enterLength and exitLength.
WindMovement
The WindMovement class is the initialization class. It first performs the serial communication
initialization that is required for each program to communicate with the pinwheels. It then reads in
60
the (x,y) coordinates for each pinwheel in the area from a file and creates a WindPinwheel for each
one.
The wave effect algorithm described above works only if the wind is coming from a direction
between North and West (between 2700 and 360'), for it assumes the wind vector enters the area
from the top left corner. Suppose, however, that the wind were blowing from a direction between
South and West (between 00 and 900). If we rotated the area by 900 to the right, we could then use
the same algorithm because the wind would now be coming from a direction between North and
West. Rotating the area involves changing the coordinates of each WindPinwheel. Thus, another task
of the WindMovement class is to rotate the area, change the coordinates for each WindPinwheel
accordingly, and change the direction of the wind. The figure below shows the rotation required for
the four different sets of directions and how the coordinates and direction should change given the
rotation.
90
windDirection < 180
0 5 windDirection < 90
windDirection = 180 - windDirection
x = are a_width - (x + pinwheelWidth)
y = are aheight - (y + pinwheelHeight)
windDirection = 90 - windDirection
x = area_width - (y + pinwheelHeight)
y=x
180 5 winddirection < 270
windDirection = 270 - windDirection
x=y
y = are a_width - (x + pinwheelWidth)
270 5 windDirection < 360
windDirection = 360 - wvmdDirection
x=x
y=y
Figure 4.23: Conversions for Area Rotations
After initialization and rotation has been completed, the WindMovement class calculates enterLength
and exitLength for each WindPinwheel. count is then set to zero, the timer is started, and the program
follows the rest of the wave effect algorithm, starting and stopping pinwheels at their appropriate
calculated times.
4.5.4
Display Orientation
A design issue that was overlooked in the initial design is the orientation of the display. We
experimented with two, a horizontal orientation where the pinwheels are facing up and a vertical
61
orientation where the pinwheels are facing forward. The horizontal orientation was more natural
because the simulated wind flowed through the display horizontally, the same way natural wind does.
When the natural wind was blowing from the northeast, the wave also traveled through the display
from the northeast.
Figure 4.24: Horizontally Oriented Matrix Display
We also experimented with the vertical orientation because displays are normally vertically upright.
This display was more unnatural because waves traveled vertically through it. However, from reading
maps, people are used to mapping the directions to vertical positions. Thus, it was not too difficult to
interpret a wave traveling from left to right as a wave traveling from west to east.
Figure 4.25: Vertically Oriented Matrix Display
62
Another possible orientation is to have the pinwheels laid out horizontally and face down from the
ceiling. With the display over them, users can get a real sense of movement when the wave travels
through the display. This might be most effective in giving users the sense that wind is blowing
through the area.
4.5.5
Discussion
Since the positions of the pinwheels are read in from a file, it was easy to create displays of many
different pinwheel arrangements. Thus, both a matrix arrangement and a scattered clusters
arrangement were implemented and evaluated.
The matrix arrangement worked well in portraying the direction and speed of the wind. Upon being
told that the display tracked wind movement, most people were able to see the direction in which the
wave was moving and inferred that it was the direction in which wind was moving.
The scattered arrangement also worked well in portraying direction, mainly because there were only
eight possible directions.
Figure 4.26: Scattered Clusters Display
To explore the ability of the displays to present directions other than those eight, angle directions
were manually keyed into the program. What was found was that it was easier to distinguish between
directions that varied by 100 with the matrix arrangement. This was due to its systematic timing; since
there was never a point where no pinwheel was spinning, it was easier to track the path of the wave
through the matrix. Exact direction could not be determined, but most people got a sense that the
wind, for example, "was not coming directly from the north, but from a little to the left".
63
Another display that was tested was one where all the pinwheels were spinning constantly, but sped
up when the wave went through its area. On our 3x3 matrix, this only made the display confusing,
but it would probably work much better with a larger matrix. This type of display could show that
there is wind over the area, with the constantly spinning pinwheels, while also showing the direction
and speed of the wind.
A display that visualizes characteristics of natural wind was successfully built for this application
domain. However, this display may not qualify completely as being ambient for it does not
peripherally give people a sense of the direction of the wind. Without directly focusing on it, it seems
that users can become aware that there is wind and even get a sense of how fast the wind is, but their
foreground attention may be needed to tell wind direction.
Though using the wave effect to portray wind direction seems to work, one may argue that it is
unintuitive because it portrays wind as a block of flowing air that moves through a space, affecting
different areas of the space at different times. Thus, it might be more natural to use a weathervane to
portray wind direction, or an anemometer that gives both wind speed and direction. While this is
true, the advantage of using this type of ambient display is that it can gives users a real sense of
movement. A weathervane can indicate that there is wind from a certain direction, but it does not
show that movement in any way. Since one of the key characteristics of ambient displays is their
kinetic motion, it made sense to us to use them to portray wind movement through an area.
64
4.6
Application IV: Network Activity
Of the three previous applications, the first tracked a rather global event that many people care
about, the second tracked a more personal piece the we hope will give viewers a sense of presence of
others, and the third tracked a natural event. For a fourth application, we wanted something that
could potentially cause people to take action in one way or another. For this to happen, the
application domain needed to be something we had control over and therefore something closer to
home. Network activity in the Media Lab turned out to be a fitting choice.
The MIT Media Lab is one of constant activity. There are people working at almost all hours of the
day and many are often engaging in network intensive work. Thousands of bytes per second are sent
through the lab's routers throughout the day. What makes it an interesting domain for ambient
displays is that network activity varies from floor to floor and at different points in the day. An
ambient display could show where the most activity is occurring in the building. This may be because
that particular area has a large demo or paper submission deadline the next day. An ambient display
could also show if there is a pattern of activity in the building, i.e. if one area of the building usually
arrived and was active earlier than another. It could also help explain to its viewers network
problems. If the network is slow, a glance at the display could tell them if there was heavy activity on
their particular router that was causing the lag. If a user's network is down, the display could tell him
if it was only his machine that was experiencing the problem, or if the network in his area (or the
whole building) was down.
We also wanted to see if such a display could induce its viewers to take some action. For example, if
the display indicated that there was no activity in a certain area, during a point of the day when it was
highly unlikely that no activity was occurring, it might induce people to check to see if there was
something wrong with the router in that area. If several routers were dependent on a main router that
suddenly went down, such a display could also indicate the source of the problem and lead to
someone fixing the main router.
For these reasons, an ambient display for network activity seemed to be a potentially interesting
display.
65
4.6.1
Possible Contents and Interfaces
The contents of for the display were not difficult to decide on. By network activity, one usually
means the amount of traffic on the routers, both incoming and outgoing. The following graph shows
the activity on one of the routers in the media lab, plexus-6-1.media.mit.edu.
'Daily' Graph (5 Minute Average)
12560.0 k
2560.0
k
-
-
-
--
--:-
---
-
1i280.0 k
(i 640.0 k
19I2.
OX
/ (2.%
k
.Bs(..9%)
CrntI:6
%
A....g. I:2..kB.(.
k
0
22
20 :18 16 14 12 10
6
6
4
2
0
22
20
18 ±16__
Max Jn.2529.S kB/s (20.2%) Average In 127.3 kB/s (1.0%70) Current In.-116.6 kB/s Q).9%)
Max Out: 518.8 kB/s (4.2%) Average Out: 47.0 kB/s (0.4%) Current Out: 51.6 kB/s (0.4%)
Figure 4.27: 'Daily' Router Activity
As can be seen from the graph, there can be quite a big of variance in activity from one five-minute
interval to the next. The difference between the amount of incoming and outgoing data also varies
with the time intervals. Thus, an ambient display tracking both incoming and outgoing data will be
quite active and will allow for comparison of the two.
Tracking network activity is similar to tracking stock market activity so many of the interfaces
discussed for that application could apply to this domain as well. However, while stocks had no
spatial relationship to each other, the different routers in the building do. This characteristic
introduces some other potential interfaces. One possibility is to use two pinwheels to track each
router (one for incoming bytes and one for outgoing bytes), and to lay them out in the same spatial
arrangement as they are physically situated in the building. This means creating different levels that
correspond to floors and laying the pinwheels horizontally on each level. Given some simple
landmarks to use as points of references, it would then be simple for a viewer who is familiar with
the Media Lab to see how much activity is currently occurring in any section of the lab.
Another possible interface that takes advantage of the spatial location of the routers is one that is not
as specific about where activity is occurring. Instead of tracking activity by individual routers, it
categorizes it into activity on different floors of the lab and uses two pinwheels to track activity on
66
each floor. The pinwheels are laid out in a matrix of two columns, with each row representing one
floor. This arrangement is simple and allows users to easily sense and compare the amount of activity
occurring on the different floors. The arrangement of the pinwheels in a matrix also makes it simple
to determine which pinwheels are mapping activity on which floor; the top row maps activity on the
top floor and the bottom row maps activity in the basement. In this way, we can quite successfully
show the contents of each individual pinwheel.
4.6.2
Implemented Display
The ambient display that was built for this application domain tracks the amount of activity floor by
floor and has the pinwheels
laid out as a 2x4 matrix. The floors represented in the matrix are, in
order, the fourth, third, second, and basement. No lab activity occurs on the first floor. Each row
represents one floor and consists of two pinwheels. The left pinwheel represents the average number
of bytes per second of incoming data that went through the routers in the past five minutes on that
floor and the right pinwheel represents the average number of bytes per second of outgoing data.
The pinwheels are updated every five minutes, so they are always showing the average amount of
activity in the past five minutes. The left pinwheel spins in a clockwise motion while the right spins
counterclockwise. This makes it look as if there is an incoming "bit wind" that is spinning the left
pinwheel while the right pinwheel is spinning out an outgoing "bit wind".
This interface was chosen because of its simplicity. The other interface discussed, where each router
had its own set of pinwheels, could potentially get very large and bulky. In addition, users would
probably not be able to get much information peripherally if there are numerous pinwheels each
spinning with their own information. This interface, though very informative, might be rather
confusing. Thus, we decided to implement the 2x4 matrix, hoping it would peripherally give users a
general sense of the activity in the lab.
However, we did not want to restrict users to tracking activity by floors. They should be able to track
activity on specific routers if they wish. The display's accompanying GUI provides this option. For
each row of pinwheels, users can choose which router on the floor they wish to track, or they may
track all activity on the floor. Thus, if there is a specific area of the lab they want to pay attention to,
they may still do so.
67
4.6.3
Implementation Details
4.6.3.1 Retrieving Data
Current activity for all the routers is available from the Media Lab's NeCSys (network and computing
systems) router monitor page, at http://necsys.media.mit.edu/mrtg. Information for specific routers
is at http://necsys.media.mit.edu/mrtg/<name of router>.html. Each of these pages has graphs of
activity on their specific router, like the graph shown previously. The four pieces of information that
are used are the daily average number of bytes of incoming and outgoing data per second, and the
current (5 minute) average number of bytes of incoming and outgoing data per second. The URL is
read in as a java.net.URL object and parsed for those pieces of information. As stated before, users
could choose to track total activity on the floor. This is accomplished by reading in data for all the
routers on a specific floor and adding them all up.
4.6.3.2 Mapping amount of activity to pinwheel speeds
The amount of activity is mapped to pinwheel speeds the same way changes in volume were mapped
to the speeds in the stock market application. Thus, the speed is set according to how the current
average compares to the daily average. The seven basic pinwheel speeds chosen for this display were
0, and 5-10. These speeds worked best for the pinwheels used. Speeds below five could not make the
pinwheels spin, most likely because of the weight of the paper pinwheel.
4.6.3.3
Pinwheels control program
The Java program for this display contains two main classes: 1) NetworkActivity and 2) Tracker.
The NetworkActivity class is the initialization class. It performs serial initialization and creates the
accompanying GUI shown below.
68
Router
CurTentBytes In
Current Bytes Out
Figure 4.28: Initial Network Activity GUI
From the GUI, users can input the specific router they wish to track or they may input "All" to track
all routers on a floor.
Pressing the start buttons creates Tracker objects that are responsible for tracking activity for one or
multiple routers. Each Tracker is created with a router name and a PinwheelSet of two pinwheels.
Every five minutes, the object reads in the appropriate URL(s) and parses them for the relevant data.
After normalizing current activity to average activity, it updates its pair of pinwheels to the
appropriate new speeds. Additionally, it updates the GUI to reflect the current statistics.
69
plexus.meaia.mit eau
Router
Curren~ytes in
Current ytes Out
-~ -~
t x .-- 4 2. meJi
r a tri i .
7 CA 0 1
129j95
Figure 4.29: Tracking Router Activity
The GUI was designed so the numbers are physically laid out and colored the same way the
pinwheels are. This allows for easy correlation between the specific numbers and the pinwheels they
are spinning.
4.6.4
Discussion
An ambient display for presenting network activity information was successfully created. With it,
users can get a sense of how much activity is occurring in different areas and floors of the lab. It also
allows for comparison of activity between the floors.
70
Figure 4.30: Ambient Display for Network Activity
unconnected to data (left), and a typical state of
the display during the day (right)
The display did present to us a pattern of activity. The second floor pinwheels are most consistent
from day to day. This follows because most people on the second floor follow normal working hours
and are the most consistent in their day-to-day work. For the other floors, we often see a sudden fast
spinning of the pinwheels, indicating that someone is making heavy use of the network, or spinning
late into the night, indicating that someone is working late.
The simple layout of the pinwheels does make the display easier to read. Upon being told that the
display tracked incoming and outgoing activity on the different floors, all of the small number of
people that we introduced to the display were able to conclude that the different rows mapped to the
different floors, in top to bottom order. A simple addition to the acrylic on which the pinwheels are
mounted could take away the need to tell users that the display is floor-based. For example, we could
add a picture collage of the projects that are on the fourth floor between the two pinwheels of the
71
first row, to indicate that that row is tracking fourth floor activity. One potential problem is that
people may forget which pinwheel is tracking incoming data and which is tracking outgoing data.
While again it could be solved with a "IN" and "OUT" label, a more aesthetically pleasing solution
might be to use icons that people are familiar with and naturally associate with incoming and
outgoing events.
The display does a good job of peripherally giving one a sense of network activity in the Media Lab.
A next step would be to use this display for an extended amount of time to see if it is a good
indication of network problems and if it induces users to take action. A scenario for doing so is
discussed in section 6.
72
5
Discussion
Throughout the design, implementation, and evaluation of the ambient displays built for this thesis,
various issues were recurring themes that came up with each application. This next section will
discuss these issues and make some conclusions about ambient displays based on the research that
was done for this thesis.
5.1
Appropriate Display Contents
One of the main goals of this thesis was to determine characteristics that make for appropriate
contents for the displays. Through our experiences with the displays, the following are some such
characteristics we have discovered:
0
Constantly varying
Application domains that have constantly varying data are appropriate for ambient displays because
the displays easily reflect these changes in data. With an ambient display, users will most likely not
have to devote part of their foreground attention to the data sources for this domain. The ambient
display can, in real-time, give them a sense of what is happening. Such an ambient display seems to
be more effective because it is more active and therefore less likely to fall completely into a user's
background of attention and be forgotten.
*
Imprecise
Ambient displays seem best at giving people a sense of activity. There is almost no way for people to
get precise data from just the physical displays. Thus, any domain that depends on precision should
probably not use ambient displays to convey information. In our applications, multiple changes in
information were often mapped to the same speed. It did not matter to us if 1000 shares of a stock
were just sold or if 1005 shares were just sold. What did matter was that a larger than average amount
of stock was just sold; this is the information the displays convey.
0
Intangible
As discussed in the wind movement section, ambient displays can be used to visualize information
that is intangible and hard to imagine. For example, they can be applied to many natural events, such
as wind movement, solar winds, and earth movements. These are hard to envision because of their
global scale but can be materialized with ambient displays. While materialization of these natural
73
events may make them unnatural, e.g. interpreting wind as a block of flowing air, the advantage of an
ambient display is its ability to portray movement through real motion of its own.
0
Alerting
Ambient displays can also be useful for telling users when an event has occurred, for example, when
someone one needs to speak to has arrived at work or when an important email has arrived.
Assuming the display can successfully move out of the periphery of the user's attention and into the
foreground, it can save users the need to constantly check if the person or the email has arrived.
*
Important to intended users
Ambient displays should portray information that is important to their intended users. Having a
display that tracks stock market activity in a kindergarten classroom would not make as much sense
as having one that tracks wind movement. Thus, in picking contents for an ambient display, one
should start with all domains that are interesting to the people who will be exposed to the display.
5.2
Purpose of Ambient Displays
One prevalent research issue about ambient displays is what purpose they should serve. Should they
be solely informative displays or should they be action inducing? In other words, to what level should
they induce the user to react to the information they are presenting? Ambient displays should mostly
remain in the periphery of one's attention and should have the capability of transitioning to the
foreground of one's attention should anything extreme occur. This transition should bring the
situation to the user's attention but should not force the user to take any action. The purpose of
ambient displays, like the natural ambient events they are derived from, is to inform users of the state
of events; whether users choose to act upon this knowledge is up to them.
To fulfill this purpose, displays need to be able to catch users' attention. Obtrusive displays can do
this, but ambient displays should be unobtrusive. Thus, an alternative method is needed. From the
displays built, it seems the best method is to have some sudden change in the motion in the display,
i.e. have all the pinwheels stop suddenly, or have them all spin very quickly for an extended amount
of time. These sudden changes seem to work best for catching a user's attention. The displays built
all use one of these effects to portray extreme states in their application domain.
74
5.3
Intended Users
The intended audience of a display is a crucial factor in designing displays. Not only is it important in
determining what domain is appropriate for a display; it also influences the design of the physical
interface. From the different situations we have thought of, it seems the space of ambient display
environments can be split, based on intended users, into the following 2x2 matrix:
Business
Home
Personal
These users are in constant contact with the ambient display. They
"own" the display and have the chance to get accustomed to it and
learn how its rhythms and motions correspond to different data.
Social
These users share the display
Many people are exposed to
with a few others and come in
displays in this location,
frequent but not constant
potentially only for a short
contact with the display.
while and infrequently.
Figure 5.1: Users in Ambient Display Environments
The home/personal and business/personal spaces were combined because their only potential
difference is in the type of content that is mapped. The displays in that combined space can be very
personalized and their single intended user should be given as much control as possible over their
configuration. Giving the user control makes the display much more meaningful to the user because
he has chosen exactly what he wants to see. These displays can get by with almost no fiduciary marks
that put the pinwheels in context because it seems that the user is less likely to forget something he
configured himself. The persistency of the display in his space will allow him to get used to the
display and it is expected that he will eventually learn to "read" the display fully without paying too
much attention to it. As an example, our stock market display could work well for this type of user
because he knows exactly which stock(s) is being tracked and how it is being tracked.
By "Social", we mean displays that are accessed by many during a day. In a home environment, the
users of the display stay relatively constant, so it seems that there is some chance for them to
personalize the display (based on some group consensus) and to grow accustomed to it. However,
these users are not likely to have the display constantly in their space, so it is more likely for them to
forget the current mapping of the display. Thus, these displays need to be either somewhat intuitive
or they need to be put in context, at least enough so that the users can be quickly reminded of the
contents of the display, if they should forget.
75
The social/business block has displays that have many users, each of whom are in contact with the
display for varying amounts of time. The advantage of these displays is that they can reach a larger
number of people and therefore can be used for domains like network activity, a local (as opposed to
global) application that is important to all users in the building. In order to be meaningful to all their
possible users, these displays must be clear in what they are representing. It seems that people who
are not in frequent contact with a display will not likely remember what domain it is mapping and are
even less likely to remember how it is mapping events in that domain to the pinwheels. Therefore,
putting the displays in context is especially important in a social/business space.
It is hard for displays to be ambient in a social/business space because people are not around it
enough to get information peripherally. Thus, for this thesis, I focused a lot more on a personal and
social/home environment, where people have the chance to get accustomed to the display. For me,
using an ambient display in those settings makes more sense because people get the chance to
personalize the display and understand how the information is being mapped. Creating an ambient
display for social/business settings can be useful because the display is in the physical environment
and accessible to a multitude of users. However, in my opinion, these displays are no longer ambient
media and are only physical interfaces that visualize information. Thus, the displays need to be
almost as explicit as graphical representations. Marian Petre, in her paper Why Looking Isn't Always
Seeing: Readership Skills and Graphical Programming[21], states:
"Much of what contributes to the comprehensibility of a graphical representation
isn't part of the formal programming notation but a 'secondary notation' of layout,
typographic cues, and graphical enhancements that is subject to individual skill."
It is difficult for ambient displays to use these secondary notations because that would entail the
addition of labels and other enhancements that could very likely take away from the display. Thus, it
is important to consider carefully whether an ambient display for a certain domain is appropriate for
social/business settings. Sometimes it is probably better to use a real physical implementation of a
graphical interface instead of an abstract ambient display that might very likely be ignored by many
because of its lack of clarity.
5.4
Creating Convincing and Believable Displays
The accompanying GUI enhances the believability of the display. Showing users exactly what data is
being presented by the displays makes the display more convincing than a standalone physical
interface. With the latter, there would be nothing to keep users from suspecting that the pinwheels
76
were being randomly spun at different speeds. Given the GUI, users can see that the data really
corresponds to the motion of the pinwheels. For this reason, our displays have accompanying GUIs.
Allowing people to interact with the displays seems to be an even stronger way to convince people of
the validity of the display. It seems that users believe in a display more when they have a sense of
causality. They want to see a display behave differently if they change its contents. If it does, they are
more likely to believe that the display is showing actual data. If it does not, the display loses all
credibility. Currently, this interaction is being done completely through graphical interfaces. An
alternative would be to make the display a sort of tangible user interface. Tokens could be created
that hold different contents. Putting different tokens on the display would then cause it to be
mapped to different contents. This sort of interaction might be a good way to give users a sense of
causality because their physical act of placing a token on the display causes the display to act a certain
way. However, making a display this sort of TUI requires adding to the physical interface and should
therefore be designed carefully so as not to make the display colossal or obtrusive.
The mapping of a display can also attribute to its credibility. If people have a sense of what the data
is doing and therefore an expectation of how the display should act, their expectations should be
met, or they may decide that the display is inaccurate. This is also why it is crucial to use a mapping
that makes sense to users.
5.5
Putting the Displays in Context
Given just the physical display of pinwheels, it is almost impossible for users to determine what
information the display is presenting without someone telling them or the presence of a visual cue
there to clue them in. For this reason and reasons discussed in the previous section, it is essential that
each display be put in context in some way, especially if it is intended for a social setting. This can be
done by adding on to the physical interface or with the accompanying GUI. The former is better
because it allows people to get information fully from the display, without first glancing at the GUI.
The challenge is to do this without adding bulk to the display and taking away from the elegance and
simplicity of the pinwheels.
5.5.1
Indicating Display Contents
Several ways of putting the displays in context by adding to the physical interface were discussed in
the design of the displays. One is to label the display. We tried very hard to stay away from textually
labeling the displays for two reasons. The first was that it made it obvious that the display was an
77
informational display and would prevent it from blending and fitting in with the architectural space.
The second was that it took away from the abstractness of the display. In my opinion, ambient
displays will never be as explicit and quantitatively informative as a graphical display. If an ambient
display can only work if it is as explicitly labeled as a graphical display is, then one should probably
consider whether or not a graphical representation might be more appropriate for the given situation.
Another option for indicating display contents is to use a graphical background that literally puts the
pinwheels in context with an image. This way people can tell what information is being mapped to
each pinwheel and with the right background, the display remains artistic and aesthetically pleasing.
These images, however, need to be obvious in what they are representing. A good method for
choosing images seems to be to choose ones that people are familiar with and would very naturally
associate with the intended content.
5.5.2
Indicating Individual Pinwheel Contents
Once the displays are in context, the next challenge is to show what information each individual
pinwheel is presenting. This is important because it is how people can get information out of the
display. Simply knowing what domain a display is mapping is almost useless if the user does not also
know how and what events in the domain are being mapped to the display.
In addition to indicating display contents, the GUI can also be used to show the contents of each
pinwheel. The Stock Market application uses this approach. Having the GUI be similar to the actual
physical interface makes the information each pinwheel is mapping clearer. It allows users to directly
correlate each pinwheel to its complementary part in the GUI.
Another option is to give control to the users, allowing them to determine which pinwheel is
presenting each stream of information. As previously discussed, this is suitable for displays that are in
a personal setting. Giving users the ability to personalize the display allows them to create a display
that works best for them.
Graphical backgrounds can also be informative of what each pinwheel is mapping. As with the
images discussed in the previous section, it is also essential for these backgrounds to be intuitive. The
use of a map seems to be a particularly good choice because people are familiar with them and know
how to use and read them. Giving people something they are used to as a reference can aid greatly in
putting the individual pinwheels in context.
78
The arrangement of the pinwheels in the display can also show the contents of each pinwheel. If
done well, the display will be more intuitive to its viewers, making it easier to understand. In the
Stock Market application, the interface used is a left to right array of pinwheels. This interface does
not make people think that the pinwheels are creating a time line. An interface that would probably
have been more intuitive for this particular content is an array in the xz plane, where the pinwheels
are placed one behind another. With it, people would be more likely to think in terms of time flow,
since pinwheels that represent points further back in time are physically further away from the
viewer. In the Network Activity application, the layout of the pinwheels correlates easily to the
building, with each level of the matrix representing a floor of the building. This is a good example of
how layout can be used to imply contents since one can see that the higher the pinwheel, the higher
up in the building the activity it is tracking.
5.6
Using Displays vs. Using Fixtures
Ambient displays consist of ambient fixtures, standalone ambient media objects. The purpose of
ambient fixtures was also to peripherally convey events through changes in motion, sound, light, etc.
The ambient displays are much more powerful because they comprise a collection of ambient
fixtures. Through the design of these displays, we have found that having multiple fixtures creates a
much stronger effect not only because of its ability to portray aggregate information but also because
it allows users to compare pinwheels to each other. It seems to be easier to see that a pinwheel is
spinning quickly when it is next to a slower spinning pinwheel than when it is by itself. Giving them
this basis for comparison helps the user discern the differences in speed and makes the displays more
meaningful to them.
5.7
Using Audio vs. Visual Effects
The pinwheels convey changes in information through changes in their spinning. Their intended use
was for people to visually notice a change in their motion and thus sense a change in information.
Surprisingly, their accompanying audio was more effective. Faster spinning pinwheels made more
noise, thus lots of noise corresponded to high activity, and little noise corresponded to low activity. A
sudden stop in noise worked very well in catching people's attention. For me, it is mainly how the
displays transitioned from the background to the foreground of attention.
This finding seems to imply that ambient displays that convey changes through audio effects might
be more effective than those that use visual effects. Consider an ambient display that uses flowing
79
water to convey change. It has both a visual effect and an audio effect. The sound of flowing water is
unobtrusive and not disturbing for most people and can be processed in the background of one's
attention. While people who are not paying much attention to it might not visually notice if the water
stops flowing, a stop in the sound of water flowing will likely catch their attention. The visual effect
is necessary though to present the information. Thus, a good model for an ambient display might be
one that has both visual and audio effects; the visual to convey information and the audio to perform
the background to foreground transition.
5.8
Learning Effect
As found in studies with the ambientROOM, there is a learning effect associated with ambient
environments. A user's ability to get information from such an environment is dependent on how
long they have been associating with it. From the displays built, we found that this is also true with
ambient displays. However, one problem with ambient displays is that people may get so used to
them that they fall into the background of a user's attention completely. At that point, it is very
difficult for the display to perform the transition between the foreground and background of a user's
attention. It seems that this problem may be overcome by having the displays have audio effects. Just
like visual effects, users get used to the audio effects and learn to interpret the amount of noise as the
amount of activity. As discussed earlier, audio effects are also more effective in grabbing a user's
attention.
5.9
Location
Location of the displays is an important factor in how effective the displays are. In order for them to
be truly ambient, the displays need to be in a location that is peripherally accessible to their intended
users. It is almost impossible for an ambient display to be visually informative if it is not at least
peripherally viewable by its users. Ambient displays that are not are then only revealing when users
glance at them. Although this requires foreground attention, they are still useful because they are
quickly accessible, definitely much more so than a graphical display from the web. Thus, it is best if
visually dependent displays are in the periphery of a user's vision, but if this is not possible, they
should at least be in a visually easily accessible area.
Audio effects are again relevant here because they can be used to convey information when a user
cannot easily view the actual display. Most of the time, the Stock Market display is not in my line of
80
vision, but from the sounds of its spinning, I can still get a sense of the level of activity on the
market.
Lighting is another effective technique for portraying information to users when the display is not in
their line of sight. Casting lighting on the pinwheels in a certain way creates shadows that change
depending on the motion of the pinwheels. This technique is used in the Web Site application. In
the Swisshouse, using this technique allows users to get information from the display even if they
have their backs to the display and are looking at another exhibit. The shadows are cast onto visitors,
giving them a sense of the activity of the display and therefore the number of virtual visitors that are
in the Swisshouse with them. This technique allows an ambient display to reach many more users.
5.10
Pinwheel wind vs. Real wind
Another issue that often came up in discussion is that of whether the pinwheels should be spinning
realistically, that is if the "bit wind" that spins them should follow the rules of nature. Imagine strips
of paper being blown by the wind. One would defmitely find it odd if two adjacent strips were being
blown in opposite directions, because it would look unnatural. Deciding whether this should be an
issue with ambient displays was crucial in the design stage because it would have a lot of affect on the
resulting displays. What we found is that imposing the restriction that a display also has to be natural
would take away almost all the changeable dimensions of the display since in nature, it would be
almost impossible to have a collection of pinwheels not all spin in the same direction and speed. In
addition, the bit winds that flow through the displays are supposed to be metaphors for actual data,
so users understand that the displays will not follow the laws of physics. In fact, almost no one who
has been introduced to the displays has commented on their unnatural behavior. Thus, we decided
not to take the laws of nature into account in designing the displays.
5.11
Mapping multiple values to finite number speeds
In almost all the applications, we were faced with having to map an infinite number of possible
values to a finite number of speeds. Only a small number of speeds were chosen because it is
necessary that users are easily able to distinguish between the various speeds. Having multiple
pinwheels instead of a single one facilitated this, but it was mainly made possible by the choice in
speeds. The speeds chosen are 1) all distinguishable from each other, 2) capable of making the
pinwheels spin, and 3) not so noisy as to make the display obtrusive and annoying. To map the
81
multiple values to the finite set of speeds, we normalized all data to its average and set the speed
depending on how the data compared to the average.
82
6
Future Work
Ambient displays were born out of ideas and research done on peripheral awareness and tangible
user interfaces (TUIs). We have many ideas and hypotheses about how ambient displays can be used.
With our toolkit, we now have a technically sound infrastructure from which we can quickly develop
numerous ambient displays. We have also built a number of displays, proving the concept of ambient
displays, and have looked into the multitude of possible contents and mappings for the pinwheels.
The next step for research in ambient displays should be to bring the displays into different
environments to test our ideas and hypotheses. These user evaluations, if successful, will complete
the evolution of ambient displays and solidify their place in the world of ambient and tangible media.
6.1
Difficulty of User Evaluations
User evaluations for ambient displays cannot be done in a short amount of time. It would not work
to have subjects come and "use" these displays for just an hour. During that time, the subjects,
knowing the point of the experiment, would probably not be able to "ignore" the displays. They
would most likely give the displays too much foreground attention and get more information than
they would if the displays were really in the periphery of their attention. A short period is also not
enough for the learning effect to take effect. It would probably take a significant amount of time just
for the subjects to get used to the display.
Thus, to truly test the effectiveness of ambient displays, there needs to be long evaluation periods
where users can get accustomed to the displays and interact with them on a regular basis. The results
from these evaluations will more accurately give us a sense of whether the displays work the way they
are intended to. Given the timeframe and other goals of this thesis, we were unable to perform these
long term evaluations, but in this section, we propose some possible scenarios for user evaluations.
6.2
User Evaluation Goals
The user evaluations should serve to answer the following questions about ambient displays:
*
Do displays give users the intended information?
We claim that creating a timeline for stock market activity can give users a sense of the trend of
activity. User evaluations should determine if users can peripherally get this sort of intended
information from the displays.
83
*
Are they intuitive to understand?
Many different interfaces and pinwheel arrangements were discussed in the design of the
displays. They were all designed to be simple and easy to understand. While this may theoretically
be true, user evaluations should test the validity of the ideas behind the interface designs.
*
Are they believable?
This is a simple question to ask users but it is a crucial one. In order for ambient displays to be
an alternative to graphical information interfaces, they need to be believable.
*
Do they perform the background to foreground transition well?
As discussed before, a key characteristic of ambient displays is their ability to catch a user's
attention if an extreme state occurs. While we hypothesize that this is done best with a sudden
stop or start of the pinwheels, user evaluations should test our hypothesis and help pinpoint the
exact threshold at which an object moves to a user's center of attention.
User evaluations that answer these questions will not only give us feedback about the effectiveness of
our displays but also provide some insight into how the displays may potentially be improved.
It would be best if the user evaluations had subjects that are not already familiar with ambient media.
The evaluations would then give feedback about how the general public would react and use the
displays, not just about how those of us who have already thought about and interacted quite a bit
with ambient media would.
6.3
Possible User Evaluation Scenarios
6.3.1
Stock Market Conscious Individuals
The stock market ambient display is definitely a good candidate for user evaluations. It would require
having one subject who is invested in the stock market and cares about its daily ups and downs. Since
most people who have seen the displays have commented that a display showing the stock price
activity of the stocks in one's portfolio would be more useful, such a display should be used instead
of the one tracking volume that we currently have built.
The user evaluation could involve having the subject use this display for an extended amount of time,
perhaps several months. At the end of each day, the user could be asked to answer the following
questions:
84
1.
According to the display, how did your stocks do today? Do you believe it?
2. Was the display
3.
a.
noticeable most of the time
b.
sometimes noticed, sometimes not
c.
almost unnoticed
Were there ever periods where the display caught your attention? If so, describe the
circumstances and what your course of action afterwards.
By watching these answers over a long period of time, we would be able to tell if the subject was able
to get information out of the display and if this information was received peripherally or because the
subject paid attention to the display. We would also be able to watch the learning effect take place;
we would be able to see when the novelty of the display wears out and whether the user grows more
accustomed to the display with time.
The evaluation period should be at least a month. At the end, asking the subject whether or not he
enjoyed having the display and his suggestions for it would help us determine the feasibility of using
displays as an alternative to, in this case, a stock ticker.
6.3.2
Physical Presence and Activity
One of our main ideas about ambient displays is that they should give people a sense of activity.
Thus, a fitting place for a display is one where there is a lot of human activity and where a person's
physical presence is pertinent to the rest of the people in the area. One such place is an office or a
research lab. Another possible evaluation, then, is to place a display in a professor's office that
comprises of pinwheels that represent presence and activity of all of his students. The spinning of a
pinwheel would correlate to the presence and activity level of a student. If a student were not in lab,
the pinwheel would be inactive. If a student were in his office, but not working much, the pinwheel
would spin slowly. High activity would cause the pinwheel to spin quickly.
This user evaluation would be particularly good for evaluating how effective displays are at
performing the transition between the background to foreground of a user's attention by spinning or
stopping suddenly. The subject (professor) would be asked to note when each student arrives at and
leaves the lab area. He will be able to do so if the display successfully catches his attention. He can
also be asked question two from the previous section to determine how and when he becomes
accustomed to the display. At the end of the evaluation period, the subject should be asked if he has
85
come to rely largely on the display to determine if a student is in the lab and if the student is busy or
in the middle of something. If this is true, then the display is successful in at least that respect.
6.3.3
Network Activity
Like the stock market, network activity is also a good candidate for user evaluations. A good test of
the display would be to leave it in an area of the Media Lab that is at least peripherally accessible to a
number of students. After a brief introduction, the display should be left alone and not given special
attention. By doing so, it is more likely to fade into the periphery of attention and its true
effectiveness can be explored.
After a year of use, the subjects (people in the area) should be asked the following questions:
1.
Do you think the display was successful in peripherally giving you information about the
current network activity?
2.
Did you use the display? for what reasons?
3.
Did the display help in indicating sources of problems with the network?
4.
Did the display ever catch your attention? If so, in what circumstances? What was your
following course of action?
Leaving the display in an environment for an extended amount of time and asking for subject
evaluations afterwards will give us very good insight into how people react to the display and how
they learn to use it. This evaluation would also explore the issue of how action inducing the displays
should and can be.
6.3.4
Interface Interpretation Questionnaires
A less time consuming evaluation would be on the different interfaces for the displays. The displays
need to be representative, intuitive, and believable. The different interfaces for each application
domain can be tested for each of these qualities through questionnaires that probe into how people
intuitively interpret the displays.
One type of questionnaire could present an application domain, show an arrangement of the
pinwheels and ask the subjects how they would interpret the display. With the stock market
application, most people assumed that the display was tracking stock price, with one pinwheel per
stock. This seemed to indicate that people might tend to map single bits of information to each
86
pinwheel; a single bit of information in the stock market is the price of one stock. This type of
questionnaire would give more of this type of indications.
Another questionnaire could simply present the interfaces and ask users if each of the contents and
mappings made sense and if each display was believable. Given out to enough people, this type of
questionnaire could give us a good sense of people's general reactions to the different interfaces.
6.4
Summary
The scenarios described here are just a few of the many possible user evaluations that can be done
for the ambient displays. We are now at a point where the most important next step is to move the
displays out into different environments. Doing so will show us how realistic our concepts and intent
for ambient displays are.
87
7
Conclusion
The goal set forth for this thesis was three-fold. The first was to create a sound technical
infrastructure for the pinwheels. The second was to study appropriate contents for ambient displays
and the third was to study the different mappings possible for these displays.
To accomplish these goals, a toolkit for pinwheel development was first built that allows for easy
assembly and configuration of robust ambient displays. This toolkit consists of hardware and
software components and abstracts away many of the low level implementation details of the
pinwheels. It will allow for easy development of ambient displays by many users in the future. From
the applications developed thus far, the toolkit has shown to be reliable and easy to use.
In an effort to reach some conclusions about ambient displays, four displays were built. The first
tracked stock market activity by following the volume of stock that exchanged hands over contiguous
intervals of time. The second was a website application catered towards the Swisshouse that gives
actual visitors to the Swisshouse a sense of the virtual visitors that are there with them. The third
tracked a natural event, wind movement, showing that ambient displays can be used to visualize
occurrences that are normally hard to envision. The last tracked network activity in the Media Lab, a
domain that is important and in our control, in hopes that it would show activity patterns in the lab
and help users pinpoint causes of network problems.
Through the development of these displays, we were able to make some conclusions about the
contents and design of ambient displays. Ambient displays are mainly informative interfaces that give
users a sense of activity in a domain that is important to them. Peripherally, they present a general
sense of the current happenings of that domain, but up front, they can present aggregate information
on top of individual pieces of data. The layout and background of the displays can be used to place
the pinwheels in context and to show what information is spinning each pinwheel, two elements that
are crucial in making a display meaningful to users. An ambient display is also not complete without a
GUI interface from which people can get specifics about the information if they wish; the GUI is
also instrumental in convincing users of the validity of the display.
The next big step in the research of ambient displays is to perform user evaluations to see if they can
really be used as an informative layer between users and graphical interfaces that demand constant
foreground attention. From the research done for this thesis, the possibility for this is promising.
88
Appendix A:
H'
[H'
UIND
HHI"_Wz
(*4 I~b *
00
--4
0
Circuit Schematics
11.
14
This appendix contains the circuit schematics.
14
ON
~
-0
'0
(*1
III
Figure A.1: Pinwheel Board
89
14
-4
M
~
j
-tin
-A I
(*1 t~ *
sn
0
0
0
(*1
(*1
III
1,041
(.4
I
o
+5V
+5V
Ji
1
-
2T10UT
3
44
T15
Ul
1
15
10
6
0
2
1
RIN
R10UT
3
20
1
R22
R20UT
13
2
3
4
5
12
6
i
19
6
RJ12
TN
T2IN
T20UT
18
2+8
2+
C2+
C2C2-
Cl+
Ci-
N
V
GND
VCC
V-
V-
______
14
32
RJ12
__
MAJ033CSD
0
p
Title
Pinwheel Serial htjafface Board
Size
B
Date:
File:
Number
(c) 2000 TMG
LIT Media Lab, Tangible Media (roup
Sheet of
12-May-2000
C:\WIDOWS\.eria.sch
Drawn By:
Revisin.
Sandia Ren
Appendix B:
Pinwheel Structure Components
This appendix contains pictures of the pinwheel components and how they fit together.
Figure B.1 Pinwheel Structure Components
91
B.1
Pinwheel Circuitry
Figure B.2
Finished pinwheel
circuit board.
To minimize size
of board, RJ12
connectors were
mounted on one
side with all other
components on
the other.
B.2
Pinwheel Container
Figure B.3
Front view of
Pinwheel Container.
Figure B.4
Side view of
Pinwheel Container.
92
Figure B.5
Back view of
Pinwheel Container.
Figure B.6
Front view of
container with motor.
The motor shaft
protudes out of the
large hole in front.
Screws inserted in
the two small holes
hold the motor in place.
Figure B.7
Back view of
container with motor.
Wires of the motor
lay along the ridge
and connect to
pinwheel board at
the top.
93
Figure B.8
Fully stuffed
pinwheel container.
The motor slides
into the bottom
half while the
pinwheel circuit
board fits into
space on top.
B.3
Pinwheel
Figure B.9
Front view of
pinwheel.
The paper is
slid onto motor
shaft and its
corners are
folded in.
A shaft collar
on top of the
paper corners
keeps them
in place.
94
Figure B.10
Side view of
pinwheel.
95
Appendix C:
Pinwheel API
This appendix contains the Pinwheel API, the software component of the toolkit. It can be used by
any Java program. The overall package summary is followed by the API for each main class. This
documentation was created using javadoc.
C.1
Pinwheels Package Summary
Package pinwheels
Clas Summar
Pinwheel
PinwheelSet
SpeedMapping
Exception Summary
_
InvalidSetException
InvalidSpeedException
LastPinwheelException
NoSuchSpeedException
NotInSetException
96
C.2
Pinwheel Class
pinwheels
Class Pinwheel
java. lang.Object
+--pinwheels . Pinwheel
public class Pinwheel
extends java.lang.Object
Field Summary
static int BACKWARDS
static int FORWARDS
Constructor Summary
Pinwheel(char c,
java.io.OutputStream outstream)
Pinwheel(char c, java.io.OutputStream outstream,
SpeedMapping smapping)
Pinwheel (char c, java. io .OutputStream outstream,
java. lang. String pwname)
Pinwheel(char c, java.io.OutputStream outstream,
java. lang. String pwname, SpeedMapping smapping)
Method Su mmary
d,
4)
void decSpeed()
decreases pinwheel speed by one
int getDirection()
get current pinwheel spin direction
char getID()
get pinwheel id
java. lang. String getName ()
get pinwheel name
int cretSDeed()
97
get current pinwheel speed
SpeedMapping getSpeedMapping ()
get pinwheel speed mapping
void incSpeed()
increases pinwheel speed by one
void setDirection(int dir)
sets direction of spin for this pinwheel
void
setName (java. lang. String strName)
sets the name of this pinwheel
void setSpeed(int newspeed)
sets pinwheel to a new speed in the current direction
void setSpeed(int changespeed, int direction)
sets the pinwheel to a new speed in a certain direction
void setSpeedMapping (SpeedMapping speeds)
sets speed mapping for this pinwheel
java.lang.String toString()
prints out pinwheel characteristics: name, id, and speed
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait,
wait, wait
Field-Deti
FORWARDS
public static final int FORWARDS
BACKWARDS
public static final
mnt
BACKWARDS
Constructor Detail
Pinwheel
public Pinwheel(char c,
java.io.OutputStream outstream)
Parameters:
c - unqiue pinwheel id
outs tream - OutputStream to allow writing to serial port
Pinwheel
public Pinwheel(char c,
java. io . OutputStream outs tream,
SpeedMapping smapping)
Parameters:
c - unique pinwheel id
98
out sream - OutputStream to allow writing to serial port
smapping - this pinwheel's SpeedMapping
Pinwheel
public Pinwheel(char c,
java.io.OutputStream outstream,
java. lang. String pwname)
Parameters:
c - unique pinwheel id
outsream - OutputStream to allow writing to serial port
pwname - pinwheel name
Pinwheel
public Pinwheel(char c,
java.io.OutputStream outstream,
java.lang.String pwname,
SpeedMapping smapping)
Parameters:
c - unique pinwheel id
outsream - OutputStream to allow writing to serial port
pwname - pinwheel name
smapping - this pinwheel's SpeedMapping
Method Detail
getID
public char getID()
get pinwheel id
Returns:
char id
getSpeed
public int getSpeed()
get current pinwheel speed
Returns:
int current speed
getSpeedMapping
public SpeedMapping getSpeedMapping ()
get pinwheel speed mapping
Returns:
SpeedMapping speed mapping
getDirection
public int getDirection()
get current pinwheel spin direction
Returns:
int current direction
getName
public java.lang.String getName()
get pinwheel name
Returns:
String pinwheel name
99
setName
public void setName (java. lang.
sets the name of this pinwheel
Parameters:
String strName)
String - name of pinwheel
setDirection
public void setDirection(int dir)
sets direction of spin for this pinwheel
Parameters:
int - direction for pinwheel to spin
setSpeedMapping
public void setSpeedMapping(SpeedMapping speeds)
sets speed mapping for this pinwheel
Parameters:
SpeedMapping - speed mapping
setSpeed
public void setSpeed(int changespeed,
int direction)
throws InvalidSpeedException,
java.io.IOException
sets the pinwheel to a new speed in a certain direction
Parameters:
int - new speed
int - direction of spin
Throws:
InvalidSpeedException - new speed is invalid (unmapped) speed
java.io.IOException - problem writing to the serial port
setSpeed
public void setSpeed(int newspeed)
throws InvalidSpeedException,
java.io.IOException
sets pinwheel to a new speed in the current direction
Parameters:
int - new speed to set pinwheel
Throws:
InvalidSpeedException - new speed is an invalid/unmapped speed
java.io.IOException - problem writing to the serial port
incSpeed
public void incfpeed()
throws InvalidSpeedException,
java.io.IOException
increases pinwheel speed by one
Throws:
InvalidSpeedException - increasing speed results in invalid speed
java.io.IOException - problem writing to serial port
100
decSpeed
public void decSpeed()
throws InvalidSpeedException,
java.io.IOException
decreases pinwheel speed by one
Throws:
InvalidSpeedException - decreasing speed results in invalid speed
java.io.IOException - problem writing to serial port
toString
public java.lang.String toString()
prints out pinwheel characteristics: name, id, and speed
Overrides:
toString in class java.lang.Object
Returns:
String pinwheel characteristics
101
C.3
PinwheelSet Class
pinwheels
Class PinwheelSet
java.lang.Object
+--pinwheels .PinwheelSet
public class PinwheelSet
extends java.lang.Object
Constructor Summary
PinwheelSet ()
constructor creates an empty set of pinwheels with no name
PinwheelSet (char initpw, char endpw, java.lang.String name,
java.io.OutputStream outstream)
constructor creates a set of pinwheels with ids from a start id to an ending id.
PinwheelSet (java.util.Vector pinwheels)
constructor creates a set using the pinwheels passed in.
PinwheelSet (j ava. util.Vector pinwheels, java. lang. String name)
constructor creates a set using the pinwheels passed in and the name
Method Summary
void addPinwheel (Pinwheel pinwheel)
adds a new pinwheel to the set
Pinwheel getFirstPinwheel ()
returns first pinwheel of the set
Pinwheel getLastPinwheel ()
get
last pinwheel
of the set
java. lang. String getName ()
returns name of the set
Pinwheel getNextPinwheel (Pinwheel pinwheel)
gets the next pinwheel from a set
int getNumPinwheels ()
gets number of pinwheels
Pinwheel getPinwheel (char pinwheelID)
gets a pinwheel from the set with a certain id
Pinwheel getPinwheel (java.lang. String pinwheelName)
gets a pinwheel from the set with a certain name
java.util .Vector getPinwheels ()
returns pinwheels in the set as a vector
102
void removePinwheel (char pinwheel)
removes pinwheel with a certain char id
void removePinwheel (Pinwheel pinwheel)
removes pinwheel
void removePinwheel (java. lang.String pinwheel)
removes pinwheel with a certain name
void replacePinwheel (Pinwheel oldPinwheel,
Pinwheel newPinwheel)
replaces a pinwheel in the set with another one
void setSpeedMapping (SpeedMapping speeds)
sets a SpeedMapping for all pinwheels
void start(int speed)
starts all pinwheels spinning at a certain speed
void start(int speed, int direction)
starts all pinwheels spinning at certain speed in certain direction
void stop()
stops all pinwheels
java. lang.
String toString()
returns string with pinwheel set infoormation
Methods inherited from class java.lang.Object
clone, equals,
wait, wait
finalize,
getClass,
hashCode,
notify,
notifyAll,
Co structor Detail
PinwheelSet
public PinwheelSet ()
constructor creates an empty set of pinwheels with no name
PinwheelSet
public PinwheelSet (java.util.Vector pinwheels)
constructor creates a set using the pinwheels passed in.
Parameters:
Vector - pinwheels to create set out of
PinwheelSet
public PinwheelSet (java.util.Vector pinwheels,
java.lang.String name)
constructor creates a set using the pinwheels passed in and the name
Parameters:
Vector - pinwheels to create set out of
String - name of this set
103
wait,
PinwheelSet
public PinwheelSet(char initpw,
char endpw,
java.lang.String name,
java.io.OutputStream outstream)
throws InvalidSetException
constructor creates a set of pinwheels with ids from a start id to an ending id.
Parameters:
char - id of first pinwheel
char - id of last pinwheel
String - name of this set
OutputStream - handle on serial port to create pinwheels with
Throws:
InvalidSetException - thrown if end id is before start id
Method Detail
getPinwheel
public Pinwheel getPinwheel(char pinwheelID)
throws NotInSetException
gets a pinwheel from the set with a certain id
Parameters:
char - id of pinwheel to retrieve
Returns:
Pinwheel pinwheel in the set with that id
Throws:
NotInSetException - thrown if no pinwheel in set has that id
getPinwheel
public Pinwheel getPinwheel (java. lang. String pinwheelName)
throws NotInSetException
gets a pinwheel from the set with a certain name
Parameters:
String - name of pinwheel to retrieve
Returns:
Pinwheel pinwheel in set with that name
Throws:
NotInSetException - thrown if no pinwheel in set has that name
getNumPinwheels
public int getNumPinwheels()
gets number of pinwheels
Returns:
int number of pinwheels in set
getPinwheels
public java.util .Vector getPinwheels ()
returns pinwheels in the set as a vector
Returns:
Vector vector of all the pinwheels in the set
getName
public java.lang.String getName()
104
returns name of the set
Returns:
String name of pinwheel set
getFirstPinwheel
public Pinwheel getFirstPinwheel ()
throws pinwheels .exceptions .EmptySetException
returns first pinwheel of the set
Returns:
Pinwheel first pinwheel in the set
Throws:
pinwheels.exceptions.EmptySetException - thrown if set is empty
getNextPinwheel
public Pinwheel getNextPinwheel(Pinwheel pinwheel)
throws NotInSetException,
LastPinwheelException
gets the next pinwheel from a set
Parameters:
Pinwheel - pinwheel to get next pinwheel after
Returns:
Pinwheel pinwheel after the one passed in as a parameter
Throws:
NotInSetException - thrown if passed in pinwheel is not in the set
LastPinwheelException - thrown if passed in pinwheel is the last one in the set
getLastPinwheel
public Pinwheel getLastPinwheel()
throws pinwheels .exceptions .EmptySetException
get last pinwheel of the set
Returns:
Pinwheel last pinwheel of the set
Throws:
pinwheels.exceptions.EmptySetException - thrown if the pinwheel set is empty
addPinwheel
public void addPinwheel (Pinwheel pinwheel)
throws pinwheels .exceptions .DuplicatePinwheelException
adds a new pinwheel to the set
Parameters:
Pinwheel - pinwheel to add to the set
Throws:
pinwheels.exceptions.DuplicatePinwheelException - thrown if a pinwheel with the same
char id already exists in the set
replacePinwheel
public void replacePinwheel(Pinwheel oldPinwheel,
Pinwheel newPinwheel)
throws NotInSetException,
pinwheels .exceptions .DuplicatePinwheelException
replaces a pinwheel in the set with another one
Parameters:
105
Pinwheel - pinwheel to replace
Pinwheel - pinwheel to replace with
Throws:
NotInSetException - thrown if pinwheel to be replaced doesn't exist in the set
pinwheels.exceptions.DuplicatePinwheelException - thrown if another pinwheel with the
same char id as the new pinwheel already exists in the set
removePinwheel
public void removePinwheel (java. lang. String pinwheel)
throws NotInSetException
removes pinwheel with a certain name
Parameters:
String - name of pinwheel to remove
Throws:
NotInSetException - thrown if no pinwheel by that name exists in set
removePinwheel
public void removePinwheel (char pinwheel)
throws NotInSetException
removes pinwheel with a certain char id
Parameters:
char - id of pinwheel to remove
Throws:
NotInSetException - thrown if no pinwheel with that id exists in set
removePinwheel
public void removePinwheel(Pinwheel pinwheel)
throws NotInSetException
removes pinwheel
Parameters:
Pinwheel - pinwheel to remove
Throws:
NotInSetException - thrown if pinwheel isn't in set
start
public void start(int speed)
throws InvalidSpeedException,
java.io.IOException
starts all pinwheels spinning at a certain speed
Parameters:
int - speed to start pinwheels spinning at
start
public void start(int speed,
int direction)
throws InvalidSpeedException,
java.io.IOException
starts all pinwheels spinning at certain speed in certain direction
Parameters:
int - speed to start pinwheels spinning at
int - direction to spin pinwheels in
106
stop
public void stop()
stops all pinwheels
setSpeedMapping
public void setSpeedMapping(SpeedMapping speeds)
sets a SpeedMapping for all pinwheels
Parameters:
SpeedMapping - speed mapping for all pinwheels
toString
public java.lang.String toString()
returns string with pinwheel set infoormation
Overrides:
toString in class java.lang.Object
Returns:
String pinwheel set name and information about all pinwheels in set
107
C.4
SpeedMapping
pinwheels
Class SpeedMapping
java. lang.Object
+--pinwheels .SpeedMapping
public class SpeedMapping
extends java.lang.Object
Constructor Summary
.Vector speeds)
SpeedMapping (j ava .util
constructor - sorts the speeds that are passed in and maps speeds 1, 2, 3, etc.
IMethod'Summr y
void add(int speed)
adds a new speed to the mapping - it wil be inserted in the
appropriate place to keep the order of the mappings
java . util .Enumeration getAllMappedSpeeds ()
returns all mapped speeds in order
int getMappedSpeed(int speed)
returns mapped speed
int getNextSpeed(int speed)
returns speed mapped to the next speed after the argument
speed.
int getNumSpeeds()
gets number of speeds in the mapping
int getPrevSpeed(int speed)
returns speed mapped to the one before the argument speed
for example, getPrevSpeed(3) returns f(2), where f is the mapping
function
void remove(int speed)
removes a speed from the mapping
java. lang.
String toString ()
toStringo procedure
108
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait,
wait, wait
Constructor Detail
SpeedMapping
public SpeedMapping (java. util .Vector speeds)
constructor - sorts the speeds that are passed in and maps speeds 1,2,3, etc. to them
Parameters:
vector - mapped speeds
Method Detail
add
public void add(int speed)
adds a new speed to the mapping - it will be inserted in the appropriate place to keep the
order of the mappings
Parameters:
int - new speed to add
remove
public void remove(int speed)
throws NoSuchSpeedException
removes a speed from the mapping
Parameters:
int - speed to remove
Throws:
NoSuchSpeedException - speed does not exist in mapping
getNumSpeeds
public int getNumSpeeds()
gets number of speeds in the mapping
Returns:
int number of speeds in the mapping
getMappedSpeed
public int getMappedSpeed(int speed)
throws InvalidSpeedException
returns mapped speed
Parameters:
int - unmapped speed to get mapped speed of
Returns:
int speed mapped to argument
getNextSpeed
public int getNextSpeed(int speed)
throws InvalidSpeedException
returns speed mapped to the next speed after the argument speed. for example,
getNextSpeed(3) returns f(4), where f is the mapping function
109
~-~------"
.-.
~
-
-
-~
~
z.~
U
I
Parameters:
int - unmapped speed to get next mapped speed of
Throws:
InvalidSpeedException - thrown if argument is not mapped in this mapping or if no next
speed exists
getPrevSpeed
public int getPrevSpeed(int speed)
throws InvalidSpeedException
returns speed mapped to the one before the argument speed for example, getPrevSpeed(3)
returns f(2), where f is the mapping function
Parameters:
int - unmapped speed to get previous mapped speed of
Throws:
InvalidSpeedException - thrown if argument is not mapped in this mapping or if no
previous speed exists
getAllMappedSpeeds
public java.util . Enumeration getAllMappedSpeeds ()
returns all mapped speeds in order
Returns:
Enumeration all the mapped speeds
toString
public java.lang.String toString()
toStringo procedure
Overrides:
toString in class java.lang.Object
Returns:
String string representation of mapping
110
Appendix D:
Swisshouse Floorplan
This appendix contains an image of the Swisshouse floorplan, as designed by Dr. Muriel Waldvogel
and Professor Jeffrey Huang. The proposed location of the website ambient display is shown. The
black squares represent the individual pinwheels.
L
I
I
I
b~~
I
_.IK
0
U.
C)
Figure D.1: Swisshouse floorplan showing tentative location of ambient display
111
References
1.
Wisneski, C., Ishii, H., Dahley, A., Gorbet, M., Brave, S., Ullmer, B., Yarin, P. (1998). Ambient
Displays: Turning Architectural Space into an Interface between People and Digital Information.
in Proceedings of International Workshop on Cooperative Buildings (CoBuild '98), (Darmstadt,
Germany, February 1998). Lecture Notes in Computer Science. Springer Publishing. 22-32.
2.
Dahley, A., Wisneski, C., and Ishii, H. (1998) Water Lamp and Pinwheels: Ambient Projection of
Digital Information into Architectural Space. In Summary of CHI '98, ACM Press, 269-270.
3.
Ishii, H., Wisneski, C., Brave, S., Dahley, A., Gorbet, M., Ullmer, B., and Yarin, P. (1998).
ambientROOM: Integrating Ambient Media with Architectural Space. In Summary of CHI '98,
ACM Press, 173-174.
4.
Ishii, H., Ullmer, B. (1997). Tangible Bits: Towards Seamless Interfaces between People, Bits and
Atoms. Proceedingsof CHI '97, ACM Press, 234-241.
5.
Streitz, N. (1998). Integrated design of real architectural spaces and virtual information spaces. In
Summary of CHI '98,ACM Press, 263-264.
6.
Marcus, A. (1998). Metaphor design for user interfaces. In Summary of CHI '98, ACM Press, 129130.
7.
Mynatt, E., Back, E., Want, R., Baer, M., and Ellis, J. (1998) Designing Audio Aura. Proceedingsof
CHI '98, ACM Press, 566-573.
8.
Card, S., Mackinlay, J., and Shneiderman, B. (1999) Readings In Information Visualization.
Morgan Kaufmann Publishers, Inc. San Francisco, CA.
9.
Kamada, T., Kawai, S. (1991) A general framework for visualizing abstract objects and relations.
ACM Transactionson Graphics,ACM Press, Vol. 10, No.1, January 1991, 1-39.
10. Sebrechts, M., Cugini, J., Laskowski, S., Vasilakis, J., and Miller, M. (1999) Visualization of search
results: a comparative evaluation of text, 2D, and 3D interfaces. Proceedings of SIGIR '99, ACM
Press, 3-10.
11. Terveen, L., Hill, W., and Amento, B. (1999) Constructing, organizing, and visualizing collections
of topically related Web resources. ACM Transactionson Computer-Human Interaction,ACM Press,
Vol. 6, No. 1, 67-94.
12. Meyer, J., Staples, L., Minneman, S., Naimark, M., and Glassner, A. (1998) Artists and
technologists working together (panel). Proceedingsof UIST '98,ACM Press, 67-69.
13. Pederson, E., Sokoler, T. (1997) AROMA: abstract representation of presence supporting mutual
awareness, Proceedingsof CHI '97, ACM Press, 51-58.
14. Cherry, E.C. (1953). Some experiments on the recognition of speech with one and two ears.
Journalof the Acoustical Sodety ofAmerica, 25: 975-979.
112
15. Gibson J.J. (1979). The Ecological Approach to Visual Perception. Lawrence Erlbaum
Associates.
16. Neisser, U. (1976). Cognition and Reality. W H Freeman & Co.
17. Rensink RA, O'Regan JK, and Clark JJ (1997). To See or Not to See: The Need for Attention to
Perceive Changes in Scenes. PsychologicalScience,8: 368-373.
18. Treisman, A. (1960). Verbal cues, language, and meaning in selective attention. QuarterlyJournal
ExperimentalPsycholog, 12: 242-248.
19. Pederson, E. (1998). People presence or Room Activity Supporting Peripheral Awareness over
Distance,Proceedings of CHI '98, 283.
20. Waldvogel, M., Huang, J. (2000). Swisshouse, A Digital Consulate for Science and Technology in
Cambridge.
21. Petre, M. (1995). Why Looking Isn't Always Seeing: Readership Skills and Graphical
Programming.Communicationsof the ACM, v38, June 1995, pp. 33-44.
22. TINI board homepage. http://www.ibutton.com/TINI/index.html
23. W3C Reference Library. http://www.cs.vu.nl/-eliens/WWW5/papers/W3C.html
24. Requirements for Writing Java API Specifications.
http://java.sun.com/products/jdk/javadoc/writingapispecs.html
113