novel visualization and
interaction for large displays
mary czerwinski
microsoft research
• Patrick
Baudisch
• Mary
Czerwinski
• Brian
Meyers
• George
Robertson
• Greg Smith
• Daniel
Robbins
• Desney Tan
• Nuria
Oliver
vibe team
large display surfaces are here
Workstation in the real world
home setup
Generic snapshot in real person’s bedroom
overview
• initial large display research
–prototypes around usability issues
observed
• visualization and interaction
–new user experiences have to scale
the wide continuum of displays
• future directions
harris poll responses (7/02,
N=1197)
Mutiple PCs and Displays
Percent Respondants
80%
70%
60%
50%
40%
All
30%
20%
10%
0%
None
Multiple monitors
attached to
multiple
computers.
Laptop and
Dualmon or higher
desktop monitor
connected
together.
Config
why a larger display surface?
Projected LCD Pricing 2002-2005
• prices dropping fast
• footprints getting
smaller
$1,200
$1,000
$1,089
$905
$800
$US
• productivity benefits
10-30% (despite sw
usability issues)
• users prefer more
display surface
$600
$400
$597
$437
15" -13.5%
$752
$699
$378
$510
$327
$200
$625
17" -14.6%
$436
18" -16.9%
$283
$0
2002
2003
2004
2005
1st prototype--dSharp display
• triple
projection
• matrox
parhelia card
• 3028 x764
resolution
• 42 in. across
• slightly curved
• 120 degree
FOV
task times – significant
Effects of Display Size on Task Times
Average Task Time (Seconds)
160
140
120
100
Small
80
Large
60
40
20
0
DISPLAY
user satisfaction - significant
Average Rating (1=Disagree,
5=Agree)
the tasks were easy to perform
5
4
3
2
1
0
Small
Large
Display Size
windows layout - significant
Average Rating
(1=Disagree,
5=Agree)
I was satisfied with the ease of windows
layout
5
4
3
2
1
0
Small
Large
Display Size
but…usability issues
• why click to bring a
clearly visible window
into focus? caused
many errors
• where is my cursor?
• where is my start
button?
• where is my taskbar?
• where are my dialogs?
• the software doesn’t
know where the
bezel is…
input: high density cursor
use
mo ion
t
mo
regular mouse cursor
f
ursors p ill-in c
c
n
i
l
l
fi
e revio urso
t fram
us f rs
curren
ram
e
high-density cursor
use
mo ion
t
mo
problem
• at high mouse speeds, the mouse cursor
seems to jump from one position to the
next
solution
• high-density cursor inserts additional
cursor images between actual cursor
positions
• the mouse cursor appear more continuous
the windows mouse trail…
• makes mouse trail last longer
• drawback: cursor images lag behind
Windows mouse trail
...is not high-density cursor
• hd cursor makes mouse trail denser
• lag-free: mouse stops => cursor stops
high-density cursor
input: drag-and-pop
problem
• large displays create long distance
mouse movement
• touch & pen input has problems moving
between screen units
solution
• drag-and-pop brings proxies of targets
to the user from across display surfaces
• the user can complete drag interactions
locally—no need to deal with distances
or to cross display borders
vibelog: 2 research paths
• 1st activity repository for
studying windows usage in
aggregate
– can profile users based on
display size
– can be extended to visualize
workflow and capture context
• single user: capture task
contexts to surface pertinent
ui or provide reminders
windows and task management
issues emerge
– aggregation model not
task-based
– users can’t operate on
groups of related windows
Relationship between # of Monitors and # of
Windows Left Open
18.00
16.00
Avg. # of Windows Left Open
• larger displays =
more open windows
• multimon users
arrange windows
spatially
• taskbar does not
scale:
14.00
12.00
Single Monitor
10.00
DualMon
8.00
TripleMon
6.00
4.00
2.00
0.00
No. of Monitors
changes in window access patterns
100
90
Percentage of Access Technique
80
70
60
Win
Taskbar
50
40
30
20
10
0
1
2
Number of Monitors
3
multitasking support
•vibelog
•projectbar, layoutbar,
groupbar
•scalable fabric
task management: groupbar
• taskbar for lightweight grouping of
windows
• allows for multiple bars, spatial placement
of bars
• ~400 internal downloads
• desktop snapshotting; task snapshots
• licensing to external companies
task management: scalable fabric
•
•
•
•
configurable central focus + peripheral context
easy task switch from periphery to focus area
leverages human spatial memory
over 600 downloads, licensing externally
table cloth
• problem:
– user wants to
access content
physically far
away
• solution:
– pan the desktop
to user
– compress
content to the
right of focus
– grab content
you need and
snap back
meeting support: wincuts
• microsoft supports:
–
–
–
–
individual user productivity (eg. word, excel, …)
disseminating information (eg. powerpoint, sharepoint, …)
communication (eg. outlook, MSN messenger, …)
distance collaboration (eg. Netmeeting, Livemeeting, …)
• but what about co-located collaborative work?
– people bring different expertise and information on
personal devices to meetings and need to share
• today’s model is broken—only one person gets to display entire
desktop at a time (for better or worse)
• or, must share applications and files with others
– what about private information?
• what about ui that should be scaled for the task at
hand?
scalable ui research
• datelens with Ben Bederson
• facetmap—visualizing your digital
memories
datelens with Ben Bederson
• fisheye
representation of
dates
• compact overviews
• user control over
the view
• integrated search
(keyword)
• enables overviews,
fluid navigation to
discover patterns
and outliers
• integrated with
outlook
facetmap
• a way to visualize all of your digital
memories
– soon, everything will be recorded
• heart rate, blood oxygen levels, conversations,
where you go, who you talk to, what you touch
electronically, etc.
– need ways to browse all these streams of
media
• most studying text lists of search results
• can we do something purely visually that scales?
future vibe directions
• novel interaction and visualization techniques
that scale from small to very large displays
• continued evaluation and iteration of designs
from a user-centered perspective
• more focus on collaboration and group
awareness with large displays
• more information:
http://research.microsoft.com/research/vibe
thank you!