User
Interfaces
for 3D
K. Gatland and D. Jefferis
The World of the Future: Robots.
1979, Usborne Hayes.
Outline
•
•
•
•
•
•
•
•
•
Why bother with 3D ?
Some Basic Tasks
Input devices
Output devices
Properties and pitfalls of 3D
Scientific Visualization
Information Visualization in 3D
Interaction techniques and interface schemes
Demos
Why bother with 3D ?
• Displaying objects or environments that are
naturally 3D (architectural plans, industrial
designs, …). Examples:
– using VR to shop for a new kitchen
– designing an automobile
– visualizing CT scans or MRI data
http://www.tjhsst.edu/TechLabs/CAD/cad98/rciszek/ahome.htm
Why bother with 3D ?
• Scientific Visualization
– Where the data has a natural 3D spatial structure
Weyl scalar fields from
orbiting binary neutron stars.
http://jeanluc.ncsa.uiuc.edu/Movies/
Two colliding black holes.
http://jeanluc.ncsa.uiuc.edu/Exhibits/exhibits.html
sphere eversion
•Sometimes, 3D isn’t enough !
Why bother with 3D ?
• To harness natural human abilities ?
– Pre-conscious processing by the human visual system
– Spatial memory
Example:
The cow jumped over the moon.
The moon is the largest natural
satellite of the earth, and is composed
of 30 % cheddar, 40 % mozzarella,
25 % star dust, and 5 % Elmer’s glue.
Yesterday, at 12:15 pm, the cow
owned by Mrs. Farmwell jumped
over the moon.
http://www.angelfire.com/pa2/klb01/spheregallery2.html
•To not use 3D seems like a waste of bandwidth !
•But how do we represent abstract data in 3D ?
Why bother with 3D ?
• We can pack more information, and more
complex relationships, into 3D
– Information visualization
• Where abstract data is embedded into 3D
• Example below: Anemone (Benjamin Fry)
Using the process of organic information design to visualize the changing structure of a web site,
juxtaposed with usage information.
http://acg.media.mit.edu/people/fry/anemone/
Why not bother with 3D ?
• Added complexity
• Many more degrees of freedom to handle
• “Standard” input/output devices not
designed for 3D
• Interfaces may be difficult to learn
The benefits should
outweigh the costs !
Some Basic Tasks
•
•
•
•
•
•
•
Specify a point (3 DOF)
Specify an orientation (3 DOF)
Specify a path
Selecting an object or region in space
Create an object or surface
Navigation (6 DOF)
Finding out where something is
Navigation: Camera Control
Zoom vs Dolly (Translation)
From slides by Chris North
Metaphors for Camera Navigation
•
•
•
•
World-in-hand
Eyeball-in-hand (a.k.a. egocentric)
Walking
Flying
Some Input Devices
• Plain old 2D mouse
– Use picking ray to select objects
– Translate a 3D cursor, 1 or 2 dimensions at a time
• 2+1 D pointers
– E.g. mouse + thumbwheel; lever that can be
pushed/pulled
• 3D pointers
– E.g. trackers, floating mice, …
• Higher DOF devices
– Data glove, shape tape, …
Spaceball; Logitech Magellan
http://www.alsos.com/Products/Devices/SpaceBall.html
•6 DOF, but …
•Rate control rather than position control
Rockin’ Mouse
R. Balakrishnan, T. Baudel, G. Kurtenbach, G. Fitzmaurice (1997).
The Rockin’ Mouse: Integral 3D manipulation on a plane. CHI’97.
Shape Tape
“Doll’s Head”: Props-based interface for 3D Cutting Plane
• Ken Hinckley
From slides by Chris North
Output devices
• Flat screens
• Stereoscopic displays
– Red/green glasses, LCD shutters, head-mounted
displays
– Autostereoscopic displays
• Re-imaging displays
• Parallax displays (e.g. holographic displays)
• Volumetric displays
For more info on autostereoscopic displays:
http://web.media.mit.edu/~halazar/autostereo/autostereo.html
High Fidelity 3D Output
• Stereoscopic
• Convergence
• Accommodation
Boom Chameleon
(G. Fitzmaurice et al.)
•Navigation is easy to learn
•3D view can be shared
•Not stereoscopic
G. Fitzmaurice and fakespacesystems.com
Virtual Reality
Head-mounted display
High DOF input device
•“Immersive”, but also cumbersome ?
•Stereoscopic, but no ocular accommodation
Volumetric Display
The Perspecta display. www.actualitysystems.com
•Stereoscopic, and ocular accommodation !
•No occlusion, and no view independent shading 
•Interesting property: no perspective projections possible
Volumetric Display
Elizabeth Downing, www.3dtl.com
Properties and Pitfalls of 3D
What is this ?
•What’s behind it ? What’s on the other side ?
•What’s behind me ?
•3D is inherently subjective
Where am I ?
Where am I ?
Where am I ?
Cues (visual, and depth)
• Occlusion
– Gives ordinal information
• Transparency
• Perspective
– Relative size, foreshortening, converging lines
•
•
•
•
•
•
Stereopsis
Motion parallax
Contour, shading, specular highlights, reflections
Shadows (e.g. drop shadows)
Ground plane grid, coloured sky
Landmarks, compass arrows
Example use of cues
Vida Dujmovi\'{c}, Pat Morin, David R. Wood
Path-Width and Three-Dimensional Straight-Line Grid Drawings of Graphs
GD 2002
Example use of cues: card readers
Example use of cues:
Shading with surface normals
(images by Michael McGuffin)
Example use of cues
Product Logo
http://www.cri-mw.co.jp/products/product_adx_e.htm
Example use of cues
Plumb Design’s
Visual Thesaurus
http://www.visualthesaurus.com/
Scientific Visualization
From an ad for GRAFTOOL software, made by 3-D Visions,
appearing in the January 1992 issue of Scientific American.
From an ad for GRAFTOOL software, made by 3-D Visions,
appearing in the January 1992 issue of Scientific American.
From an ad for GRAFTOOL software, made by 3-D Visions,
appearing in the January 1992 issue of Scientific American.
Cutting Planes & Isosurfaces
http://www.slicerdicer.com/2astrolg.html
Geological Data
http://www.slicerdicer.com/6geolg.html
Volumetric Data
Janet Haswell
“Visualizing
Electromagnetic Data”
in
G. Grinstein and H.
Levkowitz (Eds.)
“Perceptual Issues in
Visualization”
pp. 109--125
1995
Springer
Textures for enhacing cues
L. M. de la Cruz, I. Garcia, V. Godoy, E.
Ramos, “Case study: parallel lagrangian
visualization applied to natural convective
flows”, ACM PVG 2001
Sphere Eversion
http://www.geom.umn.edu/~munzner/ieee94/ieee/node25.html
Sphere Eversion
http://www.geom.umn.edu/graphics/pix/Video_Productions/Outside_In/blue-red-alpha.html
Hyperbolic Space
http://www.geom.umn.edu/~munzner/ieee94/ieee/node25.html
Hierarchical Flow Diagram
PhD thesis of H. Loeffelmann
http://www.vrvis.at/vis/
Medical Vis
C. Balazs et al.
http://www.vrvis.at/vis/research/npvr/
Medical Vis
C. Balazs et al.
http://www.vrvis.at/vis/research/npvr/
Information Visualization in 3D
If Keanu Reeves does it, it must be cool, no ?
3D message board
•Looks cool, but …
•User spends most of their time navigating 
•How to fix ?
3D message board
A static diagram of the
Prothos Application Framework
What problems would
arise if this diagram were
dynamic/interactive ?
http://www.cs.ualberta.ca/~hoover/cmput660/readings/SoftwareArch/section/prothos.htm
http://www.cs.ualberta.ca/~hoover/cmput660/readings/SoftwareArch/image/framework.gif
Collapsible Cylindrical Trees
(Dachselt and Ebert, IEEE InfoVis 2001)
ThreeDFM
http://www.icewalkers.com/Linux/Software/516470/ThreeDFM.html
File System Navigator (FSN) by SGI
From slides by Chris North
http://www.sgi.com/fun/freeware/3d_navigator.html
File System Visualizer (FSV)
http://fsv.sourceforge.net/
Information Pyramids
Andrews and Wolte and Pichler
IEEE Visualization 1997, Late Breaking Hot Topics
Cone Trees (G. Robertson et al.)
G. Robertson, J. Mackinlay, and S. Card. Cone trees: Animated 3d
visualizations of hierarchical information. CHI '91.
Image courtesy of G. Fitzmaurice.
Kodisein’s file browser
http://www.mulle-kybernetik.com/software/kodisein/manual/filebrowser.html
Innolab 3D File Manager
(Ferris wheel-like arrangement)
Adam Miezianko, Kristopher Rambish, Karen Fung, Zavnura Pingkan
http://www.sciencemag.org/cgi/content/full/301/5639/1476
http://www.sciencemag.org/feature/data/vis2003/illus_first.html
http://www.sciencemag.org/feature/data/vis2003/images/fileman_large.jpg
“Hierarchical Net”
Hierarchy based 3D Visualization of Large Software Structures
Michael Balzer and Oliver Deussen
poster at Vis 2004
PolyPlane (Hong and Murtagh,
IEEE InfoVis 2003 poster)
Information Cube (Rekimoto & Green)
Adapted from slides by Chris North
Cube: a 3D visual programming
language (Marc Najork)
http://www.research.compaq.com/SRC/personal/najork/cube.html
SAM: An Animated 3D Programming
Language (Christian Geiger et al.)
“3D-PP” Visual Programming
System (Oshiba and Tanaka)
Force-directed layout of graphs in 3D
•Pseudo-physical
simulation of forces
leads to automatic
layout
•Nodes are mutually
repelled by an
electrical force
•Edges are springs
http://www.dgp.toronto.edu/~mjmcguff/research/graph3D/
Valence (Ben Fry)
•Nodes are words in a text
•Edges connect words that
appear consecutively in the
text (thus, the text is a path
through the graph)
•Words that appear many
times are pushed outward
•Pairs of words that appear
many times have shorter
edges connecting them
•Graph is constructed
dynamically as text is read in
Image from Ben Fry’s master’s thesis
Valence (Benjamin Fry)
http://acg.media.mit.edu/people/fry/valence/
Gradus (Matt Grenby)
•Scatter plot of
dictionary words
•x, y, z axes
correspond to time,
familiarity, alphabetic
ordering
•Overall form reveals
something about
underlying data
http://acg.media.mit.edu/people/grenby/gradus/
Examples of
Interaction Techniques,
Interface Schemes
(Very) Indirect Manipulation
Through Traditional Widgets
• Creates a divided attention problem
•User must switch between mouse and
keyboard
Thomas Strothotte. “Computational Visualization: Graphics, Abstraction,
and Interactivity”, p. 318
Indirect Manipulation
Through 3D Widgets
http://www.refractions.net/terrainserver/screenshots/snapshot_vrml.jpg
http://www.mmu.ac.uk/art-des/arc/people/sforestiero/diss/FIG05.JPG
http://hackberry.chem.trinity.edu/IJC/Text/adam.gif
“Cosmo Player” VRML browser, by SGI
(Almost) Direct Manipulation
Through 3D Widgets
“Manipulators”
in Maya for
translating,
rotating, and
scaling a cube
•All operations
performed with a
regular mouse
Traditional Widgets
Embedded in the 3D scene
http://www.ssec.wisc.edu/~billh/sc95.gif
Smarter Cameras
•
•
•
•
Camera always stays upright
Camera detects surrounding geometry
Glances (J. Pierce et al.)
StyleCam (N. Burtnyk et al.)
J. S. Pierce, M. Conway, M. Van Dantzich, G. Robertson.
Toolspaces and Glances.
1999 Symposium on Interactive 3D Graphics (I3D '99).
Voodoo Dolls (Jeff Pierce)
•“Crushing head”
selection
•Other possible schemes:
casting fishing line
J. S. Pierce, B. Stearns, R. Pausch.
Two Handed Manipulation of Voodoo Dolls in Virtual Environments.
1999 Symposium on Interactive 3D Graphics (I3D '99).
Haptic feedback: The Phantom
http://www.sensable.com
R. Jagnow and J. Dorsey.
Virtual sculpting with haptic displacement maps.
Proceedings of Graphics Interface, 2002.
3D desktop (3dna.net)
•Improved harnessing of spatial memory ?
Win3D, by
clockwise3d.com
3D OS X
http://www.acm.uiuc.edu/macwarriors/projects/3dosx/screenshots.html
Project Looking Glass
(Sun Microsystems)
http://wwws.sun.com/software/looking_glass/
A 3D Window Manager
(www.3dwm.org)
www.3dwm.org
www.3dwm.org
www.3dwm.org
Ferris Wheel layout
http://www.lego.com/eng/create/designer/default.asp?x=x&id=4100
Metisse (Olivier Chapuis and
Nicolas Roussel)
http://insitu.lri.fr/~chapuis/metisse/screenshots/
Data Mountain (G. Robertson et al.)
"Our pre-attentive ability to
recognize spatial relationships
[...] makes it possible to place
pages at a distance
(thereby using less screen
space) and understand their
spatial relationships without
thinking about it."
G. Robertson et al.
Data Mountain: Using spatial memory for document management. UIST ’98.
Task Gallery (G. Robertson et al.)
G. Robertson et al.
The Task Gallery: A 3D Window Manager. CHI 2000.
2D versus 3D (Cockburn et al.)
A. Cockburn and B. McKenzie.
Evaluating the effectivenessof spatial memory in 2D and 3D
physical and virtual environments. CHI 2002.
Proprioception and VR
Reference for above pictures: Mine et al., "Moving objects in space: exploiting proprioception in
virtual-environment interaction", SIGGRAPH '97. For related work, see also Pierce, Conway, van
Dantzich, Robertson (1999), Toolspaces and Glances, I3D’99
CSCW in 3D
• Goal: awareness of location & view of other
users in a common 3D space
• Avatars not enough, because
– When avatar is distant or facing away, can’t see
what user is looking at
– When avatar is out of view, can’t tell anything
about the user
Groupspace
• 7 techniques to increase awareness of
location, perspective, and proximity of
others:
–
–
–
–
–
–
–
Nose ray
View cone
Head light
Awareness slider
Rotating participant
WYSIWIS participant
Grand tour
Groupspace
J. Dyck and C. Gutwin. Groupspace: a 3D workspace supporting
user awareness. CHI 2002.
Lessons ?
• For things that are “naturally” 3D (e.g. cars, scientific
visualization)
– 3D is necessary, but still not simple
– Good design will make it better
• For information visualization,
– 3D can be worse than 2D
– Good design is essential
• Good design includes
– Use of constraints where possible, to simplify navigation
and manipulation
– Use of cues, to ease interpretation of visual information
Demos
Extra Material
http://dform1shiftfunc.net
http://dform1shiftfunc.net
Doom as a tool for system administration
(Dennis Chao)
http://www.cs.unm.edu/~dlchao/flake/doom/
Doom as a tool for system administration
(Dennis Chao)
http://www.cs.unm.edu/~dlchao/flake/doom/
Doom as a tool for system administration
Advantages:
• “The machine load is immediately apparent to the player,
who can see how crowded a room is.”
• “A new sysadmin can be given less power by providing
her with a smaller weapon.”
• “Drastic action takes work. In a command line interface,
all actions take approximately the same amount of effort.
One can ls just as easily as rm -rf * […] In a cyberspace
environment […] performing large actions takes time
and effort.”
• “Important processes can be instantiated as more
powerful monsters. They can then defend themselves
against inexperienced sysadmins.”
Doom as a tool for system administration
Disadvantages:
• “Mapping processes to appropriate monsters is
difficult. Should large processes be mapped to
large monsters? Should the monster type reflect
the CPU as well as memory usage? Should
processes and their children look alike?”
• “It is difficult to tell if your employees are doing
real work or just goofing off when tools and
games have the same GUI.”