Hear&There
An Augmented Reality System of
Linked Audio
Joey Rozier
Karrie Karahalios and Judith Donath
Sociable Media Group
MIT Media Lab
ICAD 2000 Conference - April 3, 2000
Overview
•
•
•
•
•
•
Project Purpose and Goals
Brief Scenario
Basic Augmentation Details
Hardware
Authoring and Playback
Security and Privacy
Purpose and Goals
• Augment Environment with Audio
– Augmented Reality (or Mixed Reality)
• Develop Social Augmentations
– Oral Histories
– Focus on Authoring
• Activation Networks - Linking
• Self-Contained and Portable
• Limit Use of Visual Cues
Example Scenario
• Augmented Boston Common
• Can hear audio “left behind” (maybe
history)
• Guided towards related audio
(moo...)
• Author can add his/her own sound
• Others are unaware
Augmentation Details
• The SoundSpot
– Basic unit of augmentation
– Any type of audio (voice, music,
whatever)
– “Placed” at a specific location
(longitude/latitude)
– Has a specified size
– Other audio properties (braids, etc.)
– User walking “into” SoundSpot hears it
Augmentation Details
• Example
Augmentation Details
• Must Determine User’s Location
• GPS
– Typical Accuracy: Order of 10m
– Accuracy of Ours: sub-meter
– Refresh Rate: 10Hz
– Drawback: Bulk
– Result: Media Lab Courtyard
Hardware
•
•
•
•
•
•
GPS
Compass
PalmPilot
Laptop
Headphones
Microphone
Authoring and Playback
• Focus on user authoring
– Most other systems do not
– Simple!
• Playback is still very important
– New author options lead to new
playback issues
Authoring
• Author
creates
SoundSpots
– In-Field
– Using GUI
Playback: Basics
•
•
•
•
User dons system
Very simple start-up
Moves through environment
Hears sound tied to location
Playback: Spatialized
Sound
• Each sound “appears” to be coming
from center of a SoundSpot
• Java 3D API
• Compass attached to headphones
• Do not deal with environment effects
(enclosed spaces, etc.)
• Future work will improve spatialized
sound
Playback: Two Modes
• Explorer
– All SoundSpots activated
– Can be very cluttered
– Ultimate control with listener
• Activation Networks
Playback: Activation
Networks
• For Linking and Organizing Sounds
• Author creates Link Categories
– e.g. My Family, About Media Lab
• Author associates Link Categories
with a SoundSpot
• User can activate Link Category
when standing in SoundSpot
• StartSpots
Playback: Activation
Networks Example
• StartSpot in Boston’s Public Garden
– Link Categories
• SMG: SMG SoundSpots
• Weeping Willow: interesting SoundSpots
about weeping willow trees
• My Favorites: my favorite SoundSpots
– When entering StartSpot, activate
“Weeping Willow”; now those “exist”
– Weeping Willow spots may have their
own LinkCategories
Playback: Directing the
User
• Palm Interface
– Easy, intuitive
– But uses visual information!
• Audio Beacons?
Security and Privacy
• Authentication for Authors
• Open System for Users
• Wireless Connection to Server?
Future Work
•
•
•
•
Finish Implementation!
More Portable System
Temporal Information
Other forms of audio augmentation
Contact and Contributors
• E-mail: jrozier@mit.edu
• http://smg.media.mit.edu
• Primary Contributors
– Judith Donath
– Karrie Karahalios
– Birendro Roy and Tuan Phan