@ somewhere…
Visual Design for 3D User Interfaces
Daniel C. Robbins dcr@microsoft.com
The Ideal?
• Why is it so hard to use computers?
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Structure of This Talk
•
•
•
•
•
•
•
Who am I?
Problems with current UIs
What to do about it?
Examples from our research
Lessons we learned
Metaphors
Future directions
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Who Am I?
• Fine-arts major
• Furniture, sculpture, jewelry, etc.
• UI Designer for 13 years
– 7 years at MS, 6 in MSR
– Combining academic and product focus
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Contact Info
• dcr@microsoft.com
• http://research.microsoft.com/~dcr
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
What Is a User-interface?
• Means by which a person uses a
computer
• The sensory connection between a user
and the functionality of a computerbased application
– Human-computer interaction
– Human-computer-human interaction
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
UI Puzzlers
• Has the Web killed innovative UI?
• Video: what is truth?
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Problems w/ Current UIs
•
•
•
•
Too hard to use
Too many options
Unreliable
Inconsistent
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Specific Problems
• Multiple representations for an object
– Objects look completely different in
different places
• Low-level interaction
– Manipulation centric vs. task-centric
– Should be:
“Just do what I wanted you to do!”
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Specific Problems (2)
• Discretely shifting spatial configurations
– The rug is pulled out from under the user
• Multiple simultaneous views
– Divided attention
• Homogeneous representations
– Everything looks the same
• No context
– How do changes to one thing affect another?
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
So What? Why Fix It?
• Increase customer base
– Different ages
– Different abilities
• New ways of using computers
– On-the-go
– One-function-at-a-time
– Step-up use
– Over the Net
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
How to Fix It?
Mantra: Cause and Effect
Past
Future
What can the user do?
Present
Present
What is the user doing?
Future
Past
What has the user done?
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Specifics
• Take advantage of human perception
– 3D, animation, constancy, richer cues
• Take advantage of human experience
– Appropriate use of metaphors
• In-situ assisted learning / “hard-fun”
– Video games, apprenticeship, sports
– Not the office-assistant!
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
What is a 3D UI?
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Examples From Our Research
Data Mountain, TaskGallery, PhotoMountain
The TaskGallery
“Let’s change everything”
Task Gallery: Functionality
• UI exploration for new desktop UI
1) Windows management
– Comparing multiple windows is hard
2) Task management
– Switching between user tasks is hard
– Use rooms concept, but use 3D to engage
spatial memory
• Push the technology
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
The Task Gallery: Video
• Runs on commodity hardware
• Uses unmodified applications
• Runs on Windows 2000 with
“Application Redirection”
• Results
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Good UI Practices
•
•
•
•
•
•
•
Show cause-and-effect
Combine related user actions, params.
Distinguish UI from data
Combine UI and data (Direct Manip.)
Use appropriate “channels”
Appropriate metaphor
Support current functionality
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Spatial Memory: the Bet!
• Evolution: perceptual abilities
– Remember (generally) where stuff is (?)
• Engage navigational memory
• Reinforce with landmarks: proximity
• Requires spatial
stability/constancy
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Spatial Cues #2
• Kevin Lynch: The Image of the City
– Urban Design
• Boundaries, neighborhoods, topography
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
The DataMountain
Back to basics
The Data Mountain
• What is the role of landmarks?
• Is the arrangement of the data itself
enough of a cue?
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Cues: Neighborhood
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Boundaries
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
The PhotoMountain
Adapting the DataMountain to Photographs
PhotoMountain Functionality
• Import / Add photos
• Categorize photos
– Auto, manual
• Find photos by
– Time
– People
– Place
• Create slide shows
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Design Refinements
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Lessons We Learned
How do we actually design this stuff?
Principles
•
•
•
•
Get the “Big Picture”
Channels
Geometry
Metaphor
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Focus + Context
• Scarcest resource is human attention!
• Hierarchical regions for organization
• Broaden or narrow attention
– Vs switching attention
– Parallel attention
– 3D is a natural way to do this
“Get interactions cheaply”
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
High-level Interaction
• Make one click do lots of stuff
• Free-form Navigation vs. High-level
– Doom vs. “chauffer”
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Channels: Bag of Tricks
•
•
•
•
•
Animation
Rendering style , Transparency
Abstraction
Audio
Harder in 3D
– Channels already taken (size, color, etc.)
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Methodology
How to free up design
Basic Geometries
•
•
•
•
•
•
Carousel
P-Wall
Bowl
Landscape
3D Star (chandelier)
Layering
– Embedding multiple dimensions
• Endocentric vs. Exocentric
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Carousel
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Perspective Wall
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Bowl
• Natural fisheye
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Landscape
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
3D Star
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Data Layering (1)
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Data Layering (2)
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Other?
Axis 1
# of tics
# of spines
axis of
bend
amount
of bend
Axis 2
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
All at Once
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Using These Principles
The next version of the PhotoMountain
Axes / Dimensions
• Identify Axes / Dimensions
• Prioritize dimensions (rankings)
– Which is primary?
• Bounded vs. Unbounded
– If bounded, how big?
•Extra Credit ->
•Does there have to be a Primary Axis?
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Axes / Dimensions for the
PhotoMountain
Bounded
Time
People
Place
Composite:
No
(within
cluster)
?
Yes
“Places I have
Lived”
Hierarchy Year, Month,
Depth
Day, etc…
Friends -> USA -> RI College ->
>
Close ->
Providence
Girlfriends -> George
St.
USA -> MI…,
RI…, ME…,
WA…
Hierarchy
Breadth
Friends,
Family, Coworkers,
etc…
George St.,
Brook St.,
Hope St., etc…
 Back
Probs.
Good UI
Alaska,
Alabama,
Focus+
Channels
Metaphor
Draw Some Geometries
•Blimp
•Time
showsslice
•clusters
through
Map
•Dual selection
•Slice through
Perspective
•Tents
for
•Indexed
•floats Geography
and gives
Walls Slide-show
collections
Cylinder
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Cleanup an Image…
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Try a First Pass Mapping
Cluster based on?
Current Time
Width = # of items
Time
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Add Some Real Data
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Metaphors
Is there anything new?
Why Use a Metaphor
• Capitalize on previous knowledge
• Make a UI seem friendlier (more
approachable
• Provide a set of rules that explain and
predict the behavior of objects
– Under computer control
– Under user control
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Inspiration for Metaphors?
• Nature
• The machine
• The spiritual
• Is everything in our world designed?
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Example Metaphors
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Problems With Metaphors
•
•
•
•
•
Mapping can be arbitrary
Scalability
False impressions
High expectations
Question:
– Can we use “real-world” objects as
landmarks in an abstract world?
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Future Directions
Other Geom #1
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Future Directions
3D UI and the connected world
Wall Displays
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Working Together
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Working from Home
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Dynamic Tracking
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Sharing and Revising
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Shared Environments
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
The Way-out
Lets get wacky
Public Kiosks #1
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Public Kiosk #2
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Reading List
• Principles of Three-Dimensional Design,
Luecking
• Principles of Three-Dimensional Design,
Wong
• The Image of the City, Lynch
• Design and Form (Bauhaus), Itten
• Metaphors We Live By, Lakoff et al.
• Tufte, Shneiderman, Laurel, Wurman, etc…
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
END
Window Comparison
• Rearrange
• “Peel-back”
• Transparency
 Back
Probs.

Good UI
Focus+
Flash
Mockup
Channels
Metaphor
User Studies
– Four iterative
usability tests
completed
– Numerous usability
problems identified
and fixed
– Very high
satisfaction ratings
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
What Did Users Like?
•
•
•
•
Transparency (to deal with occlusion)
Glancing (spatial awareness)
Context menus
Overview of tasks
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Problems
•
•
•
•
Occlusion
Discoverability
Can it scale?
Target audience?
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Future Work
•
•
•
•
Other metaphors
Other geometric arrangements
Peripheral notification
Semantic zooming for thumbnails
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Multiple Representations
 Back
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Low-level interaction
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Discrete Changes
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Multiple Views
5R x 3C
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Homogeneous Reps.
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Context and Relationships
 Back
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Animation
•
•
•
•
No animation #1, #2?
“If it’s in the way, make it go away”
Shift load from cognitive to perceptual
Don’t force user to figure out what
changed
– Animation styles
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Discrete Switching
But with
animation!
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Discrete Broaden
But with
animation!
Animation
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Rendering Styles #1
• 2D Wireframe
for navigation
UI elements
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Rendering Styles #2
• Icons for Window
Manipulation
controls
• Different styles
Widgets
mockup
Select
X-ray
Vision
Send
Back
Add to
Selection
 Back
To
Podium
Probs.
Good UI
Move
To
Space
Focus+
Channels
Close
Metaphor
Abstraction
• Icons vs. Thumbnails
• Depends on:
– Degree of attention
– Perceptibility
– Confusability
– Content (high freq. Vs. low freq.)
• Need semantic zooming!
• Tied to metaphor? 
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
2D vs. 3D Channels: Size
4
3
2
2
1
1
 Back
Probs.
4
3
Good UI
Focus+
Channels
Metaphor
2D vs. 3D Channels: Color
4
3
2
2
1
1
 Back
Probs.
4
3
Good UI
Focus+
Channels
Metaphor
Criteria: Fixing the Problems
•
•
•
•
•
•
Spatial memory
Context: human attention
Visual recognition
Stability
Scalability
Good UI practices
 Back
Probs.
Good UI
Focus+
Perception
vs.
Cognition
Channels
Metaphor
Output Redirection
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Input Redirection
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
The Guts
• Commodity hardware
• Unmodified applications
• Windows 2000 with “Application
Redirection”
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Audio Comparison
• No audio
 Back
Probs.
• With audio
Good UI
Focus+
Channels
Metaphor
Scalability
Many tasks
A few tasks
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Spatial Cues: Landmarks
Shadows aid in
depth perception
Landmarks aid in
spatial memory
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Visual Recognition (1)
• Objects
represented
by
thumbnails
• User
selectable
wallpaper
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Visual Recognition (2)
• Live
applications
– (Dynamic
updating)
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Task Gallery: Evaluation
•
•
•
•
Different interactions
User Studies
Problems
Future work
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Body Centric
•
•
•
•
Left, Right Hands
Feet
Look Up
Glancing
– Lightweight, ephemeral interaction
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
size
Sort on
by Name
Spatial Stability
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Transparency Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Animation Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
New Task Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Drag Task Video (Direct
manip)
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Rename Task Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Switch Tasks Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Show All Tasks Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Transparency Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Wallpaper Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Freeform Nav Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Hands Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Glancing Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
TG Full Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
DM Short Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Broaden Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor
Narrow Video
 Back
Probs.
Good UI
Focus+
Channels
Metaphor