Chapter 6 – Direct Manipulation
and Virtual Environments
6.1 Introduction
• Good interfaces produce positive feelings
• Desirable:
– Visibility of objects
– Visibility of actions
– Rapid, reversible, incremental actions
– Direct manipulation of objects of interest
6.2 Examples of Direct
Manipulation Interfaces
•
•
•
•
•
•
•
•
Windows environment (Xerox, Apple, Microsoft)
Air Traffic Control
Automobile?
WYSIWYG Word Processors
Mapping and GIS
Modern computer games
CAD
Programming of industrial robots by moving robot by
hand (actions recorded)
5.3 Explaining Direct
Manipulation
• Beneficial attributes:
–
–
–
–
–
–
–
Novices learn quickly
Experts work rapidly
Intermittent users can retain concepts
Error messages are rarely needed
Users see if their actions are furthering their goals
Users experience less anxiety
Users gain confidence and mastery - encourages
exploration
Explaining Direct
Manipulation
• Problems with direct manipulation
– Spatial or visual representations can be too spread out
– Users must learn the graphical representations
– The visual representation may be misleading
– Typing commands with the keyboard may actually be faster
– Choosing the right objects, actions, metaphors is not easy
– May need greater system resources
– History and other tracing may be difficult to maintain
– Visual impaired users may have difficulty
Icons
• An icon is an image, picture, or symbol representing
a concept
• Icon-specific guidelines
• Represent the object or action in a familiar manner
• Limit the number of different icons
• Make icons stand out from the background
• Consider three-dimensional icons
• Ensure a selected icon is visible from unselected icons
• Ensure harmony and distinctiveness
• Design the movement animation
• Add detailed information
• Explore combinations of icons to create new objects or actions
Direct Manipulation
Programming
• Robots Programmed by workers leading them
through task once (e.g. painting motion)
• Excel macros programmed via doing the tasks
by hand
• MS Access Query by Example
Programming in the User Interface
• Five challenges of programming in the user
interface:
– Sufficient computational generality
– Access to the appropriate data structures and
operators
– Ease in programming and editing programs
– Simplicity in invocation and assignment of
arguments
– Low risk
• Possible alternative to Agents
6.7 Remote Direct Manipulation
• Examples:
– Telemedicine
– Robotic Space exploration
– Home automation
• Complicating factors in the architecture of
remote environments:
– Time delays
– Incomplete feedback
– Feedback from multiple sources
– Unanticipated interferences
6.8 Virtual Environments
• Virtual reality
• Augmented reality
• Situational awareness
Virtual Reality
• Examples
– Architecture – actually walk into building
and look around (instead of more traditional
direct manipulation looking at it from
various directions and perspectives)
– Possible - medicine
– On the boundary – real flight simulator
Example: Firefighter Training
• This system will simulate the progress of a fire in a single family
dwelling
•
will respond to actions made by the user to rescue occupants and
put the fire out.
• The user of the VE will be a Fire Company Officer being trained
or evaluated for his/her skills at commanding a fire crew.
• In the VE, the user will speak commands that are translated by an
operator into a predetermined animation sequence in the virtual
environment.
• As the fire company officer issues commands, the virtual fire crew
will go through animations reflecting these commands,
• fire burns in response to virtual crew actions.
Example: Meditation Chamber
• The goal of this research is design and build an
immersive virtual environment that uses visual, audio,
and tactile cues to create, guide, and maintain a
patient's guided relaxation and meditation
experience.
• The use of meditation and guided imagery is well
established as helpful in the treatment and prevention
of a number of diseases
• The possibility of increasing the effectiveness and
repeatability of this type of therapy
• This project is aimed at creating a working prototype
of this system
Example: Virtual Geographic
Information System
• VGIS (Virtual Geographic Information System) is a
large, multifaceted project to allow navigation of and
interaction with very large and high resolution,
dynamically changing databases while retaining realtime display and interaction.
• The system allows users to navigate accurate
geographies with sustained frame rates of 15-20
frames per second.
• The user can not only see these terrains from any
viewing angle but also buildings, roads, high
resolution imagery draped on the terrain, and other
features
Example: Virtual Reality
Phobia Therapy
• Virtual Reality Exposure involves
exposing the patient to a virtual
environment containing the feared
stimulus in place of taking the patient
into a real environment or having the
patient imagine the stimulus, which is
what traditional exposure therapy
usually involves.
Example: Helping Burn
Patients Cope with Pain
• using immersive VR for pain control (in addition to pain
medicine).
• Their first virtual world used was SpiderWorld. Spiderworld
was originally designed to treat spider phobics, but has also
proved quite distracting for burn patients.
• now developing several new virtual environments
specifically designed for treating pain (e.g., especially
attention-grabbing virtual environments).
• SnowWorld has been developed with support from the Paul
Allen Foundation for Medical Research.
• Patients fly through an icy canyon with a river and frigid
waterfall. Patients shoot snowballs at snowmen and igloos
(with animated impacts).
• Since patients often report re-living their original burn
experience during wound care, SnowWorld was designed to
help put out the fire.
Example:Virtual Gorilla Exhibit
• being developed to explore techniques
for using Virtual Reality to present
information to users experientially that
would otherwise be difficult for them to
learn.
• Based upon actual data from the Zoo
Atlanta gorilla exhibit,
• modeling an environment where the user
can explore areas that are normally off
limits to the casual visitor.
Maybe not quite virtual reality
• "Macys.com gets rights to 3-D model software"
by Reuters, CNET News.com, September 2, 1999
• Software developer Broderbund, sell its new
Cosmopolitan Fashion Makeover software
exclusively through Macys.com.
• With the Cosmopolitan software women can create
their own three-dimensional model based on their
own body measurements and digitally ``try on''
brand name clothing.
• Users can also link via the makeover software
directly to the Macys.com online shopping site,
where they can buy the clothing online.
Virtual Reality Headgear
(1999)
Alternative to Headgear
Exploring using the CAVE
Virtual Environments
• Successful virtual environments depend on the
smooth integration of:
–
–
–
–
–
–
–
Visual Display
Head position sensing
Hand-position sensing
Force feedback
Sound input and output
Other sensations
Cooperative and competitive virtual reality
U Washington Virtual Reality Projects
1. Virtual Retinal Display (VRD)
2. Shared Space
3. Learning in Virtual Environments
4. PAIN MAN
5. Virtual Motion Controller
6. Interactive VRD
7. Virtual Pilot
8. Greenspace
9. Virtual Chess
10. Starship
11. New Media
11. Expert Surgical Assistant
13. Tactile Augmentation
13. Geoscientific Visualization
13. Motion Sickness
16. FLIGHT
17. Blocksmith
18. SS Working Group
19. Parkinson's Project
20. SS Working Group
21. Driving Simulator
22. Two-Handed User Interface
22. Motion Sickness
24. Wearable Interfaces
25. Virtual Classroom
26. Situation Awareness
27. CEDeS Lab
27. Endoscopic Surgery Simulator
27. Virtual Playground
27. Virtual Reality Toolkit
31. Engineering Study of an Endoscope Design
31. Virtual Mirrors
31. Starship
31. Human-Computer Symbiote
31. Virtual Chess
36. 4d mouse
36. collaborative mixed reality
36. VR Interaction Techniques
39. Collaboration through Wearables
39. Phobia Desensitization
41. VRD
41. architecture and virtual reality
41. Multimodal Interfaces
41. situation awareness
41. Medical Robotic Interfaces
46. VRD Emulator
46. laparoscopic surgical simulator
48. Flicker Sensitivity
48. knowledge base project
48. /
48. self-motion perception
52. Design for a Low Vision Aid
52. PRISM
54. LIMIT
54. Interface Sickness
54. design for a low vision aid using a scanned
laser display
57. visual-inertial nulling: cross-over asymmetry
57. design for a low vision aid
57. Measures for Presence
60. functional effects of refractive surgery on
driving performance
Progress on Visual Display
• The Virtual Retinal Display (VRD) team has been focused
on developing improvements to the current prototype
systems and on creating the parts needed for future
prototypes. The VRD, based on the concept of scanning
an image directly on the retina of the viewer's
eye, was invented at the HIT Lab in 1991. The
development program began in November 1993 with the
goal of producing a full color, wide field-ofview, high resolution, high brightness, low
cost virtual display.
• http://www.hitl.washington.edu/research/vrd/project.html
Progress on Movement Sensing
• For some applications, a hands-free, body-operated
walking interface is ideal;
• the UW HIT Lab has been developing prototypes of
"sufficient-motion" interfaces, which allow the user to
interact by using a subset of the real-world kinesthetic
inputs.
• Though the ranges of motion are less than full, these inputs
are sufficient to convince the user that he or she is moving
in the virtual world.
• Development of these interface devices is called the Virtual
Motion Controller (VMC) Project.
Virtual Motion Controller
• The HIT Lab's VMC working
prototype measures body position
over the working surface with an
arrangement of four weight sensors
• The curved working surface provides
important feedback to the user about
his or her physical location, and
therefore body locomotion input to
the device.
Progress on Cooperative
Augmented Reality
• The Shared Space interface demonstrates how augmented
reality, the overlaying of virtual objects on the real world, can
radically enhance face-to-face and remote collaboration.
• For remote collaboration, system allows life-sized live virtual
video images of remote user to be overlaid on the local real
environment, supporting spatial cues and removing the need to
be physically present at a desktop machine to conference.
• computer vision techniques are used to precisely register virtual
images with physical objects, extending the currently popular
"Tangible Interface" metaphor.
• work in the context of a collaborative card-game application that
allows face-to-face and remote users to collaboratively interact
with each other and virtual animations.
• http://www.hitl.washington.edu/research/shared_space/
Science Fiction?
• Most of this stuff won’t be in an office
near you next year …
• But the future comes quickly in computer
science !
End Chapter 6