Butterfly Effect:
An Augmented
Reality
Puzzle Game
Marleigh Norton
Blair MacIntyre
Steven Dow
Maribeth Gandy
Game Overview
• Spatial puzzle where physical world is not
modeled
• Virtual butterflies in physical environment
• Catch them!
– Butterflies are stationary
– Challenge comes from reaching the butterflies
• Bring butterflies into reach by rotating the
virtual world
– Virtual world rotates in 90 degree chunks about a
player defined axis.
Equipment
Head-Mounted Display
Tornado Stick
Rotations
• Tornado Stick controls Virtual Axis
– First Button – Place / Remove Axis
– Second Button – Pause / Unpause Axis
– Third Button – Rotate Butterflies
Tornado Stick
• People had trouble
understanding rotations
– Not everyone is used to
3D geometry
– Though 3D puzzles have worked before
• Change the metaphor, not the
interaction
– “Tornado Stick”
Butterfly Effect
Butterfly Effect Video
QuickTime™ and a
H.263 decompressor
are needed to see this picture.
Design Context
• Start with medium, then design game
– Mapping physical and virtual worlds is hard
– AR works best when user is moving slowly
• Tracking more accurate
• Safer for player – Cognitive Tunneling
Design Context
• Home environment
– Run on next generation game consoles
– Must be easy to set up
– Minimize use of special equipment
1st Prototype: 2D Paper
2nd Prototype: 2D Interactive
3rd Prototype: Video
Tuning
• Two Major Challenges
– 90 degree rotation constraint
– Understanding virtual space and physical space
• Possible Changes
– Loosening rotation constraint
– Adding more manipulations (translations)
– Adding better depth perception cues
Open Issues:
Depth Perception
• Static Depth Cues
Open Issues:
Depth Perception
• Remove Automatic
Depth Cues
Open Issues:
Depth Perception
• Remove Unavailable
Depth Cues
Open Issues:
Depth Perception
• Texture probably the
best
Without Grid
With Grid
Is Occlusion Really Impossible?
• Ad hoc modeling
– Automatic modeling during calibration
– Fiducials
• Real time depth
– Depth from stereo
• Something to be explored
Virtual Obstacles - Bees
• No consequences for using poor
strategy
• Bees behave like butterflies
• “Sting” if caught
• Penalty
– Lose life
– Butterflies escape
QuickTi me™ and a
TIFF ( Uncompressed) decompressor
are needed to see thi s pi ctur e.
Questions?
Old Slides from Past Talks
(In case people as a question I
have slides for.)
Butterfly Effect:
An Augmented
Reality
Puzzle Game
Marleigh Norton
Technical Advisor:
Blair MacIntyre
Person Walks Through
Game Area
Person Walks Through
Game Area
Person Walks Through
Game Area
Derive Walls from Path
Derive Walls from Path
Why this is Fun
• Fun in the same ways Rubik’s Cubes
are fun
• Caillois’s Four Categories of Games [1]
– agôn, alea, mimicry, ilinx
• MacTavish’s “astonishment at visual
and auditory technology” [2]
Open Issues
• Scoring
• Capture Interaction
• Depth Perception
Open Issues: Scoring
• Score based on efficiency, not speed
• Penalized for moving too much
– Accrue score as you walk
– High score bad
• Problem: High Score Traditionally Good
Open Issues:
Capture Interaction
• Location-Based Capture
– Capture butterfly by getting close enough
– Pros: Technically simple. No additional cost.
– Cons: Precise head tracking more important.
Less thematically appropriate. Less fun.
• Butterfly Net Capture
– Capture butterfly in a net
– Pros: More active, thematically appropriate
interaction.
– Cons: Requires an additional tracking system,
likely expensive.
Open Issues:
Depth Perception
• Judging depth in virtual worlds is difficult
– Overcoming this difficulty is part of the game, yet it
should not be impossible
• Problem: What can we do to help the player
understand the relative locations of the
butterflies?
• Research on Depth Perception yields several
techniques [4] [5] [6] [7]
Questions & Feedback
• Scoring – Efficiency
• Capture Interaction
– Location-based capture
– Butterfly Net Capture
• Depth Perception
– Which cues to use? More ideas?
• Other Feedback
References: Game Theory
• [1] Caillois, Roger, “Man, Play, and Games,”
trans. Meyer Barash, University of Illiniois
Press, 1961 (orig.1958) ch 2.
• [2] Mactavish, Andrew, “Technological
Pleasure: The Performance and Narrative of
Technology in Half-Life and other High-Tech
Computer Games”, in: Geoff King and Tanja
Krzywinska (eds.) Screenplay: Cinema/
Videogames/ Interfaces (London: Wallflower
Press, 2002) 33-50
References:
Depth Perception 1
• [4] Cutting, J. E., and P.M. Vishton, 1995. “Perceiving layout
and knowing distance: The integration, relative potency and
contextual use of different information about depth”, in
Perception of Space and Motion, W. Epstein and S. Rogers,
Eds. Academic Press, New York, 69--117.
• [5] Furmanski, Chris, Ronald Azuma, and Mike Daily.
“Augmented-reality visualizations guided by cognition:
Perceptual heuristics for combining visible and obscured
information”, in Proc. IEEE and ACM Int'l Symp. on Mixed and
Augmented Reality (ISMAR 2002) (Darmstadt, Germany, 30
Sept. - 1 Oct. 2002), pp. 215-224.
References:
Depth Perception 2
• [6] Sinai, M.J., W. K. Krebs, R. P. Darken, J. H. Rowland, and
J. S. McCarley, 1999. “Egocentric distance perception in a
virtual environment using a perceptual matching task”, in
Proceedings of the 43 rd Annual Meeting Human Factors and
Ergonomics Society, 43, 1256-1260.
• [7] Surdick, R. T., E. T. Davis, R. A. King, and L. F. Hodges,
1997. “The perception of distance in simulated visual displays: A
comparison of the effectiveness and accuracy of multiple depth
cues across viewing distances”, in Presence: Teleoperators and
Virtual Environments 6, 5 (October), 513--531.
References: DART
• [8] MacIntyre, Blair, Maribeth Gandy, Steven
Dow, and Jay David Bolter. "DART: A Toolkit
for Rapid Design Exploration of Augmented
Reality Experiences." To appear at
conference on User Interface Software and
Technology (UIST'04), October 24-27, 2004,
Sante Fe, New Mexico.
Implementation Status
• It’s playable!*
– Populate the world with a pre-defined set
of butterflies
– Butterflies can be captured
– Tallies of caught and remaining butterflies
are displayed
– Butterflies can be rotated about the axis
– Physical axis interface
* When all of the equipment is working.
The Screen
Feedback
•
•
•
•
Rotation Interface
Depth Perception
Score
Aesthetics
Rotation Interface
• Problem: Rotating about an axis? Huh?
– Suggested alternatives:
• Randomization
• Jar, Mini-version of gamespace
– (Rotation about a point)
• “Tornado Stick”
• Changes: Tornado Stick
– Axis/Tornado Stick only interface in which all
manipulations are valid moves
– Arrows pointing to butterflies’ final location
Depth Perception
• Problem: It’s hard to tell where the
butterflies are, especially when they’re
“behind walls”
– Complication: can’t change appearance of
butterfly based on presence of wall
(or can we…?)
• Changes: Add grid with drop shadows
– Also, it’s less of a problem when you’re
actually playing
Score
• Problem: for Efficiency-based score,
lower was better (confusing players)
– Suggestions
• Timer
• Subtract from High initial score
• No Score
• Changes: No Score
– Score isn’t really that important
– Was causing more problems than it solved
Aesthetics
• Problem: What will the game look like?
– Collecting the butterflies should be
compelling
• Changes: Aesthetics more fully fleshed
out
– Childlike graphic design with scanned
crayon textures
– Player is frog, eating butterflies
Frog has Butterfly in Sights
Frog Catches Butterfly
Frog Eats Butterfly
Future Work:
Finish Implementation
•
•
•
•
Sounds
Graphics
Animations
Refine Stick
– Are those the right buttons?
– Buttons or Switches or ???
– InertiaCube Issues with Ferris
Environments
Thanks to the Augmented
Environments Lab,
especially:
• Blair MacIntyre
• Steven Dow
• Enylton Machado Coelho
Questions?