Interaction
Jing Li
CPSC 533C
March 3, 2003
Overview
Toolglass and Magic Lenses: The SeeThrough Interface (1993)
by Eric A. Bier, Maureen C. Stone, Ken
Pier, William Buxton, Tony D. DeRose
SDM: Selective Dynamic Manipulation of
Visualizations (1995)
by Mei C. Chuah, Steven F. Roth, Joe
Mattis, John Kolojejchick
Toolglass and Magic Lenses
The see-through interface that can appear
as though on a transparent sheet of glass
between the application and the traditional
cursor.
These tools create spatial modes that can
replace temporal modes in user interface
systems.
Introduction
Toolglass widgets: semi-transparent interactive tools that
are used in an application work area, and appear on a
virtual sheet of transparent glass called a Toolglass
sheet.
The user can line up a widget, a cursor and an
application object in a single two-handed gesture.
A widget that has been stretched across the entire work
area creates a command mode.
User can move sheet over application object or move the
object to a widget (trackball can control both sheet and
scrolling).
Figure 1. Click-through buttons. (a) Six wedge objects. (b) Clicking through a green
fill-color button. (c) Clicking through a cyan outline-color button
Introduction (Cont.)
Many widgets can be placed on a single sheet
Widgets and lenses can be composed to create
a number of specialized tools from a basic set
(e.g. an outline color palette over a magnifying
lens, to point to individual edges)
Figure 2. A sheet of widgets. Clockwise from
upper left: color palette, shape palette,
clipboard, grid, delete button, and buttons
that navigate to additional widgets
Figure 3. An outline color palette
over a magnifying lens
Related Work
Multi-Handed Interfaces
Movable Tools
– Menus that pop up at the cursor position
– Toolglass sheets can be moved without tying up the
cursor
Transparent Tools
– Make menus over the work area transparent
Viewing Filters
– Magic lenses are visual filters that may be used to
change type of views or reveal hidden information.
Examples
Shape and Property Palettes
Figure 4. Shape palette.
(a) Choosing a shape.
(b) Placing the shape.
Figure 5. Font face palette. The word "directly"
is being selected and changed to bold face.
Examples (Cont.)
Clipboards
Figure 6. Symmetry clipboard.
(a) Picking up an object.
(b) Rotated copies appear.
(c) The copies are moved and pasted.
Figure 7. Fill-color rubbings.
(a) Lifting a color.
(b) Moving the clipboard.
(c) Applying the color.
Examples (Cont.)
Previewing Lenses
Figure 8. An achromatic lens over a drop shadow lens over a
knotwork. (Knotwork by Andrew Glassner)
Examples (Cont.)
Selection Tools
Figure 9. Vertex selection widget.
(a) Shapes.
(b) The widget is placed.
(c) A selected vertex.
Figure 10. The local scaling lens that
shrinks each object around its own
centroid (Tiling by Doug Wyatt)
Examples (Cont.)
Grids
Figure 11. Three grid tools
Examples (Cont.)
Visualization
Figure 12. Gaussian curvature pseudo-color lens with overlaid
tool to read the numeric value of the curvature. (Original
images courtesy of Steve Mann)
Implementation of
Toolglass Sheets
Multi-Device Input and Screen Refresh
– Handles simultaneous input from two pointing devices
and updates the screen after multiple simultaneous
changes
Filtering Input Through Lenses and Widgets
– Modifies pointing events as they pass through
widgets
Filtering Output Through Lenses and Widgets
– Modifies graphical output as it passes up through
each widget
Implementation of
Magic Lens Filters
Recursive Ambush
– catch the low level drawing calls
typically a wrapper around the graphics context
– and modify them, and then call the original
graphics primitive
e.g. modify “DrawLine” to set the color to “red”
– works transparently across all applications
– but complicated to make a new lens and
debug
Implementation of
Magic Lens Filters (Cont.)
Model-In Model-Out
– create new objects and transform them
can be saved and used by any lenses over it
– very powerful, but
must allocate storage
cross application is an issue
Implementation of
Magic Lens Filters (Cont.)
Reparameterize & clip
– Modify global parameters to drawing
– Redraw the model, but clipped to the
boundary shape of the lens
– Best example: scaling
Implementation of
Magic Lens Filters (Cont.)
Multiple model types need to be supported
for cross-application functionality.
Achieved by:
– Type conversion: convert the model to be of
the type required
– Type tolerance: unchanged if does not
understand a model
Toolglass & Magic Lens
Conclusions
See-through interface based on spatial modes
Structured well for two-handed interaction
May be integrated into any screen-based
applications and tools
SDM: Selective Dynamic
Manipulation of Visualizations
Mei C. Chuah, Steven F. Roth,
Joe Mattis, John Kolojejchick
(Carnegie Mellon University)
Motivation
• Need new interactive techniques for 2D and 3D
visualizations. We want:
– Selective: a high degree of user control
– Dynamic: interactions all occur in real time,
and interactive animation
– Manipulation: Users can directly move and
transform objects to perform different tasks
• This set of techniques is called SDM
Current Static Visualization
• Users cannot focus on specific object sets in
detail while keeping them in the larger context
• Clutter and occlusion (hidden data) in dense
information space
• Difficult to give a sense of scale: some objects
may be dwarfed in the scale used for the entire
data set (green objects in [Figure 1])
• No tools provided to classify data sets and save
those classifications
• Difficult to compare quantities when objects are
not spatially contiguous (e.g. when objects are at
different distances from the user)
SDM Sample
• SDM deals with these issues
• Selection and control methods rely on objects
rather than spaces.
Sample: Supply Distribution Network
•
•
•
•
Cylinders (red) : supply centres
Rectangular bars (purple) : shelters
Lines (black) on the floor plane : major routes
Heights of cylinders and bars : quantities available at supply centres
and needed by shelters
• Selected objects : green
SDM Components
• Object Centered Selection
– Select objects by clicking or using sliders to
place data constraints
– Object sets may be named, saved and used
later from a scrollable menu
– Object sets need not be made up of
homogeneous types (e.g. a set may contain
supply centers, shelters, and routes)
SDM Components (Cont.)
• Dynamic and
Flexible Operations
– Directly manipulate
through object handles
– Attach a handle to an
object, and push or pull
on it: causes the object
to expand, shrink or
move continuously
– Objects change with
motion of handles
(A) controls the radius of
cylinder object
(B) controls object height
(C) controls bar width
– Arrow handles: shift
SDM Components (Cont.)
• Object Constraints
Context persistence
– SDM maintains some relationship between the
focus objects set and its environment.
Set-wide operations
– If you can move or scale one object in a focus
set, you can move or scale any other object in
this set.
SDM Feedback Mechanisms
• Clearly Identify the
Selected Set
– The selected set is painted
differently (green)
– A white grid may be drawn
beneath all selected objects
to indicate positions and
spread
• Maintain Scene Context
– Used to maintain context
when objects are displaced
– Object ‘body’ (green) and
object ‘shell’ (white)
Feedback Mechanisms (Cont.)
• Maintain Temporal Continuity
– Use animation to help users to perceive
changes to the scene
• Maintain Relationships Between the
Selected Set and the Environment
– Keep a scale of the differences on screen, for
example, the ratio axes technique (next slide)
• Allow Objects to be Easily Returned to
Their Home Positions
Ratio Axes Technique
Scale of the environment (black) & the selected set (red)
The left is 1:1; the right is 1:8
SDM Solutions
• Focusing on a Select Set of Objects while Keeping Scene
Context
– Objects can be painted distinctly and the width can be increased
– Users may shift all focus objects to the front of the scene
SDM Solutions (Cont.)
• Viewing and Analyzing Occluded Objects
– Selected object set is elevated to solve occlusion
problem
SDM Solutions (Cont.)
– Reduce the heights of unconcerned objects to zero
– Make all unconcerned objects very thin
SDM Solutions (Cont.)
• Viewing Different Sets of Elements Based on Different
Scales
– Ratio axes technique (mentioned above)
• Interactively Augmenting the Visualization with New
Classification Infomation
– Add width and color, save as a set and apply them later (i.e.
identify important shelters)
SDM Solutions (Cont.)
• Comparing the Patterns, Widths, and Heights of Objects
– Draw a line of reference in the scene plane and move any set of
objects to the reference line
– Compare data trends among multiple sets
SDM System Architecture
SDM Conclusions
• Good interactive techniques deals with the
issues from current static visualizations
• Visualization is clear, precise and easy to
understand
• The goal of SDM is to provide users with
enough tools and flexibility (multiple
alternative solutions) that they can solve a
wide spectrum of data analysis tasks.