Novel Visualization and
Interaction for Small to
Jumbo Displays
Mary Czerwinski
Microsoft Research
Who Contributed: VIBE Team
Large Display Surfaces Are Here
Workstation in the real world
Overview
Initial large display research
Prototypes around usability issues observed
Visualization and interaction
New user experiences have to scale the wide
continuum of displays
Future directions
Harris Poll Responses (7/02, N=1197)
Mutiple PCs and Displays
Percent Respondants
80%
70%
60%
50%
40%
All
30%
20%
10%
0%
None
Multiple monitors
attached to
multiple
computers.
Laptop and
Dualmon or higher
desktop monitor
connected
together.
Config
Multimon Usage Trends
use multimon
32%
Peddie
Research,
2001 (N>6000)
no multimon
30%
plan to use
multimon
38%
Why A Larger Display Surface?
Prices
dropping
Footprints
getting smaller
Projected LCD Pricing 2002-2005
$1,200
$1,000
$1,089
$905
$800
$US
Productivity
benefits 1030% (despite
sw usability
issues)
Users prefer
more display
surface
$600
$400
$597
$437
15" -13.5%
$752
$699
$378
$510
$327
$200
$625
17" -14.6%
$436
18" -16.9%
$283
$0
2002
2003
2004
2005
1st Prototype--dSharp Display
Triple projection
matrox
parhelia card
3028 x764
resolution
42 in. across
Slightly curved
120 degree FOV
Task Times – Significant
Effects of Display Size on Task Times
Average Task Time (Seconds)
160
140
120
100
Small
80
Large
60
40
20
0
DISPLAY
User Satisfaction - Significant
Average Rating (1=Disagree,
5=Agree)
the tasks were easy to perform
5
4
3
2
1
0
Small
Large
Display Size
Windows Layout - Significant
Average Rating
(1=Disagree,
5=Agree)
I was satisfied with the ease of windows
layout
5
4
3
2
1
0
Small
Large
Display Size
Tenets--Large Display UX
User studies show large display surfaces
fundamentally change user interaction
Designed tools to better understand/
complement how work practice changes
Large display surfaces provide non-linear
productivity increases
Additional space has different utility
e.g. focal/peripheral displays provide
different cues
But…Usability Issues
Why click to bring a
clearly visible
window into focus?
caused many errors
Where is my
cursor?
Where is my
start button?
Where is my
taskbar?
Where are my
dialogs?
The software
doesn’t know where
the bezel is…
Vibelogger
1st activity repository
for studying windows
usage in aggregate
can profile users
based on display size
can be extended to
visualize workflow
and capture context
single user: capture
task contexts to
surface pertinent ui or
provide reminders
Multitask Visualization
colored block for each time point and app
amount of shading indicates percentage
of visibility of the window
tasks
subtasks
Task Switching Visualization
switching tasks (red to blue)
how are email windows arranged and used?
compare to...
Windows and Task
Management Issues Emerge
aggregation model not
task-based
users can’t operate on
groups of related
windows
Relationship between # of Monitors and # of
Windows Left Open
18.00
16.00
Avg. # of Windows Left Open
Larger displays =
more open windows
Multimon users
arrange windows
spatially
Taskbar does not
scale:
14.00
12.00
Single Monitor
10.00
DualMon
8.00
TripleMon
6.00
4.00
2.00
0.00
No. of Monitors
100
Changes in Window Access
Patterns
90
Percentage of Access Technique
80
70
60
Win
Taskbar
50
40
30
20
10
0
1
2
Number of Monitors
3
Input: Drag-and-Pop
problem
large displays create long distance
mouse movement
touch & pen input has problems
moving between screen units
solution
drag-and-pop brings proxies of targets
to the user from across display
surfaces
the user can complete drag
interactions locally—no need to deal
with distances or to cross display
borders
Problem:
User wants to
access
content
physically far
away
Solution:
Pan the
desktop to
user
Compress
content to the
right of focus
Grab content
you need and
snap back
Table Cloth
Multitasking Support
Projectbar, layoutbar,
groupbar
Scalable fabric
Task flasher
Secret passwords
Table cloth
Wincuts
Task Management: Groupbar
Taskbar for lightweight grouping of
windows
Allows for multiple bars, spatial
placement of bars
Desktop snapshotting; task
snapshots
Task management: Scalable
Fabric
Configurable central focus + periphery
Easy task switch from periphery to focus
Leverages human spatial memory
Task Flasher
A more visual alt + tab
Uses 3d scaling and selection animation
Windows stay on the monitor on which
they are positioned
Secret Passwords
Problem: many touchscreen systems (eg.
SmartBoard, TabletPC) have no keyboard
Use software virtual keyboard
Hard to hide password on a virtual keyboard
Previous research showed users think anything on
large display is considered public…they watch (Tan et
al, CHI ‘03)
Solution: Secret Passwords
Meeting Support: Wincuts
What about co-located collaborative work?
People bring different expertise and
information on personal devices to meetings
and need to share
today’s model is broken—only one person gets to
display entire desktop at a time (for better or
worse)
or, must share applications and files with others
what about private information?
What about UI that should be scaled for
the task at hand?
Visualization Research
DateLens with Ben
Bederson
Facetmap
Team Tracks (SW Vis)
Datelens with Ben Bederson
Fisheye
representation of
dates
Compact
overviews
User control
Integrated search
Integrated with
outlook
Pen-enabled
FacetMap
FacetMap
Scalable Visualization for “all of your stuff”
Queries MyLifeBits SQL database interactively
Domain includes web pages, digital images,
documents, email, SenseCam physical activity data,
VibeLog PC activity data
Heterogeneous data, metadata, annotations, and
relationships organized into “facets”
Browsing and searching accomplished by iterative
selection from among available item attributes
(and/or full text search)
Uses Piccolo.Net (on top of GDI+) for graphics
& animation
Scalability
Scales in three ways:
Space: Recursive, space-filling algorithm generates
useful views for any size screen
Items: Large numbers of items
aggregated/abstracted into groups and counts at
multiple levels
Facets: New item attributes can be added to the
visualization dynamically
Fixed minimum node size to guarantee
readability
Consistent information density
Greater levels of detail are collapsed until they are
feasible to present, given display constraints
Medium Size (No Filters)
Medium Size (Filters Active)
Small Size
Large Size (Wall Display)
FacetMap for Mobile
Phones
Quick searching through structured data
Scalable UI
Half visual and half text list
Prototyping stage (desktop)
Multiple visualizations (pick the best for a
given results set)
SW Visualization: Team Tracks
Goal: help developers new to a code base
familiarize themselves quickly
Based on logs of teams using the code base
Assumption: most frequently visited areas of
the code are more important
We empirically verified this
Show developers most related areas of code based
on where they currently are
Also give them previews of that code
Screenshot of Team Tracks
Evaluation Results
Better task completion rates
9 / 9 completed tasks 1 and 2 (same)
3 / 9 completed task 3 (versus 1 / 7)
dominated by algorithmic detail
7 / 9 completed task 4 (versus 1 / 7)
dominated by finding relevant code fragments
Better quiz scores (t(16)=-2.04, p<.03,
one-tailed)
Same importance ratings (r=0.45, p=.02)
Future VIBE Directions
Novel interaction and visualization
techniques that scale from small to very
large displays
Continued evaluation and iteration of
designs from a user-centered perspective
Automatic task identification
For more information:
http://research.microsoft.com/research/vibe
Thank You!
http://research.microsoft.com
/research/vibe