CHAPTER 5:
Direct Manipulation and Virtual Environments
Designing the User Interface:
Strategies for Effective Human-Computer Interaction
Fifth Edition
Ben Shneiderman & Catherine Plaisant
in collaboration with
Maxine S. Cohen and Steven M. Jacobs
Addison Wesley
is an imprint of
© 2010 Pearson Addison-Wesley. All rights reserved.
Introduction
• Good interfaces => Satisfied user
–
–
–
–
Competence in performing tasks
Ease in learning the system
Confidence in retaining mastery over time
Enjoyment in using the system
• Direct-manipulation interfaces => Good
interfaces
– Visible objects
– Rapid, reversible, incremental actions
– Pointing actions
1-2
© 2010 Pearson Addison-Wesley. All rights reserved.
6-2
Introduction
• Newer concepts extend direct manipulation
• Include
–
–
–
–
–
Collaborative interfaces
Touchable interfaces
Virtual reality
Augmented reality
Teleoperation
1-3
© 2010 Pearson Addison-Wesley. All rights reserved.
6-3
Overview
•
•
•
•
•
Reviews important historical examples
Explores principles of direct manipulation
Provides psychological jsutifications
Raises concerns
Newer concerns
– 3D
– Teleoperation
– Virtual and augmented reality
1-4
© 2010 Pearson Addison-Wesley. All rights reserved.
6-4
Examples of Direct-Manipulation
Systems
•
•
•
•
•
Applications
Office Automation
Spatial Data Management System
Video Games
Computer Aided Design
1-5
© 2010 Pearson Addison-Wesley. All rights reserved.
6-5
Direct-Manipulation Systems
Command line vs. display editors
• Line editors (vi) are generally more flexible and powerful
• Training times with display editors are much less
• WYSIWYG display editors (WORD PROCESSORS):
– Display a full page of text
– Display document in the form it will appear when printed
– Show and control cursor motion through physically obvious and
intuitively natural means
– Use of labeled icon for actions
– Display results of an action immediately
– Provide rapid response and display
– Offer easily reversible actions
1-6
© 2010 Pearson Addison-Wesley. All rights reserved.
6-6
Direct-Manipulation Systems:
WYSIWYG word processing
1-7
© 2010 Pearson Addison-Wesley. All rights reserved.
6-7
Direct-Manipulation Systems
Technologies influenced by concepts learned about the
UI in the word processor:
• Integrated graphics, animations, etc.
• Spreadsheets
• Desktop publication software
• Presentation software
• Hypermedia environments (eg WWW)
• Improved macro facilities
• Spell checker and thesaurus
• Grammar checkers
1-8
© 2010 Pearson Addison-Wesley. All rights reserved.
6-8
Direct-Manipulation Systems
VisiCalc spreadsheet (1979) and its descendants
• VisiCalc - propagates changes across the worksheet
• Successful spatial data-management systems depend on
choosing appropriate:
– Icons
– Graphical representations
– Natural and comprehensible data layouts
• Competitors
– Lotus 1-2-3 (1980s)
– Quattro Pro
– Excel (current leader)
© 2010 Pearson Addison-Wesley. All rights reserved.
1-9
6-9
Direct-Manipulation Systems :
spreadsheet
1-10
© 2010 Pearson Addison-Wesley. All rights reserved.
6-10
Direct-Manipulation Systems :
Office Automation
• Xerox Star (1982)
– Text formatting, graphics, fonts, cursor-based UI
• Apple Lisa
– Not commercial success; laid groundwork for ..
• Apple MacIntosh
– Simplified decisions
– Pull-down menus, window manipulation, editing
graphics and text, dragging of icons
• Microsoft Window now dominates
– Improved window management, adaptable UI
© 2010 Pearson Addison-Wesley. All rights reserved.
1-11
6-11
Direct-Manipulation Systems :
spatial data management
• Xeroc PARC’s
Information Visualizer
• ArcGIS
• Google Maps
• Google Earth
1-12
© 2010 Pearson Addison-Wesley. All rights reserved.
6-12
Direct-Manipulation Systems :
Video Games
• Most exciting, well-engineered, and commercially successful
example of direct-manipulation systems Video Games
• Pong first video game
• Nintendo Wii, Sony PlayStation, and Microsoft Xbox continue to
evolve
–
–
–
–
3D, stereo sound, multiplayer, wireless controllers
Commands are physical actions whose results are immediately shown on screen
No syntax to remember no syntax error messages
Games continuously display a score feedback encourages mastery
• Game Examples:
–
–
–
–
–
–
SimCity – good example of direct manipulation / urban planning
Second Life – social virtual world
Spore – evolving creatures
Myst – puzzle; well received
DOOM and Quake controversial
World of Warcraft – massive multiplayer game
© 2010 Pearson Addison-Wesley. All rights reserved.
1-13
6-13
Direct-Manipulation Systems
Guitar Hero video game
1-14
© 2010 Pearson Addison-Wesley. All rights reserved.
6-14
Direct-Manipulation Systems :
Computer-aided design
• Computer-aided design (CAD) use direct manipulation
– Engineering, Architecture, Automobile Parts Designers
– Examples: AutoCAD, Autodesk Inventor
• Computer-aided manufacturing (CAM)
– Honeywells’ Experion Process Knowledge System
– plant operations/process control
•
•
•
•
Manipulate the object of interest
Generate alternatives easily
Explain the impact
Problem solving by analogy to the real-world
1-15
© 2010 Pearson Addison-Wesley. All rights reserved.
6-15
Continuing evolution of DirectManipulation Systems
•
•
•
•
•
•
•
•
•
Personal finance / checkbook maintenance
Travel
Home automation
Dashboards – manage information
Ubiquitous computing
Immersive Environments
Augment Reality
Ambient Devices
Touchable Devices
1-16
© 2010 Pearson Addison-Wesley. All rights reserved.
6-16
Continuing evolution of DirectManipulation Systems
Direct-Manipulation interfaces are being used in a wide range of
applications, e.g. management dashboard for a retail store
1-17
© 2010 Pearson Addison-Wesley. All rights reserved.
6-17
Continuing evolution of DirectManipulation Systems
1-18
© 2010 Pearson Addison-Wesley. All rights reserved.
6-18
Direct Manipulation Problems
• Spatial or visual representations can be too spread out
• High-level flowcharts and database-schema can become
confusing
• Designs may force valuable information off of the screen
• Users must learn the graphical representations
• The visual representation may be misleading
• Typing commands with the keyboard may be faster
1-19
© 2010 Pearson Addison-Wesley. All rights reserved.
6-19
Principles of Direct Manipulation
1. Continuous representations of objects and
actions of interest with meaningful visual
metaphors.
–
Novices learn, masters quick, intermittent users
retain, can see actions further goals
2. Physical actions or presses of labeled buttons,
instead of complex syntax.
–
Easy to learn, fewer errors, users feel in control
3. Rapid, incremental, reversible actions whose
effects on the objects are visible immediately.
1-20
© 2010 Pearson Addison-Wesley. All rights reserved.
6-20
Visual Thinking and Icons
•
Attracts right-brained thinkers and challenges leftbrained thinkers
I in WIMP is icon - an image, picture, or symbol
representing a concept
•
•
Icon-specific guidelines
–
–
–
–
–
–
–
–
Represent the object or action in a familiar manner
Limit the number of different icons
Make icons stand out from the background
Consider three-dimensional icons
Ensure a selected icon is more visible
Design the movement animation, e.g. dragging
Add detailed information (color for age of file)
Explore combinations of icons to create new objects or
actions
© 2010 Pearson Addison-Wesley. All rights reserved.
1-21
6-21
3D Interfaces
• “Pure” 3D interfaces have strong utility in some context,
– e.g., medical, product design, scientific visualization
• Other situations, more constrained 2D interfaces may be
preferable to simplify interactions
– e.g. 3D bar charts, air-traffic control, digital libraries
• “Enhanced” interfaces, better than reality, can help
reduce the limitations of the real-world,
– e.g., providing simultaneous views, flying, x-ray vision
• Games are good example
– Avatars in multiplayer 3-D worlds
– First person games
© 2010 Pearson Addison-Wesley. All rights reserved.
1-22
6-22
3D Interfaces
1-23
© 2010 Pearson Addison-Wesley. All rights reserved.
6-23
3D Interfaces
Features for effective 3D:
–
–
–
–
–
–
–
–
–
Use occlusion, shadows, perspective, etc. carefully.
Minimize number of navigation steps for users to accomplish
their tasks.
Keep text readable.
Avoid unnecessary visual clutter, distraction, contrast shifts, and
reflections.
Simplify user movement.
Prevent errors.
Simplify object movement
Organize groups of items in aligned structures to allow rapid
visual search.
Enable users to construct visual groups to support spatial recall.
1-24
© 2010 Pearson Addison-Wesley. All rights reserved.
6-24
3D Interfaces
Guidelines for inclusion of enhanced 3D features:
– Provide overviews so users can see the big picture
– Allow teleoperation (more later)
– Offer X-ray vision so users can see into or beyond
objects.
– Provide history keeping (for undo, redo)
– Permit rich user actions on objects
– Enable remote collaboration
– Give users control over explanatory text and let users
select for details on demand.
– Offer tools to select, mark, and measure.
1-25
© 2010 Pearson Addison-Wesley. All rights reserved.
6-25
3D Interfaces
Guidelines for inclusion of enhanced 3D features :
– Implement dynamic queries to rapidly filter out
unneeded items.
– Support semantic zooming and movement
– Enable landmarks to show themselves even at a
distance
– Allow multiple coordinated views
– Develop novel 3D icons to represent concepts that are
more recognizable and memorable.
1-26
© 2010 Pearson Addison-Wesley. All rights reserved.
6-26
Teleoperation
• Derives from two concepts:
– direct manipulation in personal computers
– process control in complex environments
• Physical operation is remote controlled
– Surgery, power plants, chemical plants, military ops, computersupported collaborative work
• Complicating factors in the architecture of remote
environments:
– Time delays
• transmission delays
• operation delays
– Incomplete feedback
– Feedback from multiple sources
– Unanticipated interferences
1-27
© 2010 Pearson Addison-Wesley. All rights reserved.
6-27
Virtual and Augmented Reality
• Virtual reality can break physical laws and allow users to
act as though they were somewhere else
• Situational awareness shows information about the real
world that surrounds you by tracking your movements in
a computer model
• Augmented reality is an important variant
– Enables users to see the real world with an overlay of
additional interaction.
1-28
© 2010 Pearson Addison-Wesley. All rights reserved.
6-28
Virtual and
Augmented Reality
• Successful virtual environments depend on the
smooth integration of:
–
–
–
–
–
–
–
Visual Display
Head position sensing
Hand-position sensing
Force feedback
Sound input and output
Other sensations
Cooperative and competitive virtual reality
1-29
© 2010 Pearson Addison-Wesley. All rights reserved.
6-29
Impact of this technology in our
everyday lives
1-30
© 2010 Pearson Addison-Wesley. All rights reserved.
6-30
Summary
• Most appealing systems
– Enjoyable UI
– Customizable
– Natural representations of objects and actions
– Easy to learn, use, and retain
– Actions are rapid, reversible and incremental
– Performed with movements and not syntax
1-31
© 2010 Pearson Addison-Wesley. All rights reserved.
6-31
Summary
• Having direct-manipulation interface does
not ensure success
– Bad design
– Slow implementation
– Inadequate functionality
– May not be appropriate
• Use iterative design process (ch 3) and
test advanced direct-manipulation systems
1-32
© 2010 Pearson Addison-Wesley. All rights reserved.
6-32
