VR Interfaces – Navigation,
Selection and UI Elements
By David Johnson
VR Interfaces
• How do we tell the computer to do things?
• How do we select things?
• How do we navigate around?
VR Interface Challenges
• Intuitive
– Make interaction work like the real world
– Minimize cognitive overhead
• Augmentation
– Give users new capabilities
Quick UI review
• Norman’s Principles of Design
– Make things visible
– Provide a good conceptual model
•
•
•
•
Affordance
Mapping
Constraints
Feedback
Visibility
Good Conceptual Model
• A good conceptual model allows us to
predict the effects of our actions
• Without a good model we operate blindly
– Simply follow rules without understanding a
reason
– No understanding of cause or effect
– No recourse when something breaks
• Fridge/freezer controls
• Thermostat
Affordances
Mapping
Constraints
• Prevent you from doing what you shouldn’t
do
– Grey out selections that don’t apply at the
current time
Feedback Examples
• Clicker on your turn signal
• Animated icon while waiting for a web
page to load
Why is usability important?
• Poor usability results in
– anger and frustration
– decreased productivity in the workplace
– higher error rates
– physical and emotional injury
– equipment damage
– loss of customer loyalty
2D Interfaces
• Dominant computer
interface uses a
mouse and graphical
elements
Xerox Star (1981)
2D Interfaces
• Why is it a WIMP
interface?
–
–
–
–
Windows
Icons
Menus
Pointer
Xerox Star (1981)
3D Interfaces
• Need to map 2D
interfaces to 3D
• Hopefully, create
whole new expressive
interfaces
3D equivalent of a Mouse?
• Mouse
– 2D positioning
– Buttons to hold or click
6DOF mouse
• Flying mouse
• Fledarmaus
• The Bat
• How do you
clutch/ratchet?
– In 2D, picking up
disables tracking
Menus in Virtual Space
• Cannot easily overlay
menus
• “Float” menus in
space
– Select by raycasting
– Keep near user’s head
Jacoby, Ellis 1992
Menus in Virtual Space
• Ring Menus
– JDCAD 1993
• Liang
– ISAAC 1995,
• Mark Mine
– Rotate hand to move
selection point
Menus in Virtual Space
• Darken, 1994
• Overlaid menus
• Speech selection
Menus in Virtual Space
• Pen and Tablet
– Track a tablet and pen
– Put 2D menus on
tablet
– “Haptic Hand”
Menus in Virtual Space
• Bowman
• Pinch Gloves
– Select with thumb to
finger
– High-level menu on
ND hand
– Secondary menu on D
hand
– First tries
• Scrolling menu using
pinches or
• More items on pinkie
TULIP Menus
• Three-Up, Labels in
Palm
• Virtually raise hands
• Rotate menus
• Put ‘next’ groups on
palms
• Users preferred over
floating and tablet
menus
– Perhaps slower
Menus in Virtual Spaces
• ‘Virtual tricorder’
• Wloka 1995
Gestures
• Symbolic
– Cultural meaning (O.K. sign)
• Deictic
– Pointing, direct viewer’s attention
• Iconic
– Showing an example path with hand
• Pantomimic
– Act out the activity
Gestures
• GIVEN (1992)
– Neural net recognition
– 20 gestures
• Fly, grab, etc.
• Mine
– “Physical mnemonics”
• Pull-down menus from near head
• Delete by throwing over shoulder
Numerical Input
• Mark Mine
• A digit at a time
• Sliders too
imprecise
Text Input
• Bowman
• Pinch glove
• Thumb to home row
finger
• Hands in/out to go
down/up row
• Rotate to hit extra
keys
Basic Navigation Tasks
• Exploration
– Untargeted movement
– Build internal map
• Positioning
– Move to known location
• Maneuvering
– Precise positioning of viewpoint
– Typically short motions
Natural Interfaces
•
•
•
•
Walking
Biking
Snowboard
Swimming
• Issues?
Walking workaround
• Redirected walking
– movie
Flying Interfaces
• Flying
• Magic carpet
• Guided navigation
– River analogy
• Issues?
Steering Interfaces
• Pointing
– Expressive
– Hand shake
• Torso
• Gaze-directed
– simple
• Physical device
Hand-based Interfaces
• Colin Ware (1990s)
– World-in-hand
– Eye-in-hand
Point-to-point Travel
• Select a point in a scene
– Computer picks path
• Teleport
– Bowman et al. found significant spatial
disorientation from teleport
World in Miniature
• User holds dynamic
map in one hand
• Navigation is reduced
to object positioning
WIM Setup
• Physical props – clipboard and interface
ball
Two-handed Flying
• Mark Mine
Fundamental Operations in a UI
• Select an object
• Manipulate an object
– Translate
– Rotate
• What are some techniques in 2D
interfaces?
From the Beginning
• Sutherland and
Vickers
– Sorcerer’s Apprentice
(1972)
• Track stylus
• Selection of vertices
– Intersection of cube at
tip of stylus
Pointing: Put That There
•
•
•
•
1979
Ray from tracked hand
Speech interface
movie
Pointing: JD-CAD (1993)
• “Laser gun” from
hand
– Tracker noise
– Harder to select far
away
• Spotlight
– Add a cone to the ray
– Select objects based
on
• Distance from cone axis
• Distance from hand
Silk Cursor
• Replace wireframe
selection box with
translucent box
– Visual cues to
containment
Pointing: Aperture
• Spotlight from eye
• Cone angle based on
distance from hand to
eye
• Selection modified by
hand orientation
Pointing: Flexible Pointer
• Two-handed
• Hand direction bends
pointer
• Can select occluded
objects
• movie
Hand: GoGo Interaction (1996)
• Go-Go uses Non-Linear
mapping between
virtual and real hand
• Control-display ratio
• Stretch go-go variation
• Pros:
– Extended reach when needed
– Direct manipulation
• Cons:
– Reach still limited by arm
length
– Precision suffers when reach is
extended (low level of control)
Movie
Image Plane Techniques
• Point or gesture at an objects
projection onto the viewing
plane
– “head-crusher”
• Kids in the hall
– “Sticky finger”
• Similar to ray casting
• Pros:
– Very intuitive
– Allows user to reach objects at
an arbitrary distance
• Cons:
– Limited by the need for line of
sight
– Can be fatiguing
– Virtual hand may obscure small
objects
Two Handed and Body-Centered
Interaction
• What can you do with two hands?
• What if you use your body as a
reference point?
•
Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin
(1997). Moving Objects in Space: Exploiting Proprioception
in Virtual-Environment Interaction. Proceedings of
SIGGRAPH 97, Los Angeles, CA.
• “Scaled-space grab”
HOMER technique
Hand-Centered Object
Manipulation Extending
Ray-Casting
• Select: ray-casting
• Manipulate: hand
• Translation
proportional to initial
object distance
Time
World in Miniature
• User holds dynamic
map in one hand
• Objects can be
moved in map
• What about fine
positioning?
• What about selection
of small objects?
Voodoo Dolls
• User creates map
with image plane
selection
Summary
• Ergonomics an issue
• Usability still low
• Standard GUI elements translate poorly