Mobile Augmented Reality
Game Development
Jeff Chastine
Southern Polytechnic State University
Georgia State University, Mar. 27, 2013
Hi!
Overview
•
•
•
•
•
What got us here?
Paradigm Shift!
Case Study: Qualcomm
Challenges of AR Game Dev
If time: code review
A Little AR History
• 1968
• Ivan Sutherland
• Optical Combiner
A Little AR History
• 1992
• Tom Caudell
and
David Mizell
coin the
term
“Augmented
Reality”
A Little AR History
Mixed Reality
Real
Environment
Augmented
Reality (AR)
Augmented
Virtuality (AV)
Virtual
Environment
(VR)
• 1994
• Virtual Reality (VR) attempts to completely
synthesize
• Augmented Reality (AR) integrates virtual
objects into the physical world
A Little AR History
• 1996 – Rekimoto tracks in
6 DoF!
– x, y, z
– yaw, pitch, roll
A Little AR History
• 1997
• Azuma states three requirements:
1. Combines the real and virtual worlds
2. Interactive in real time
3. Registered in 3D
A Little AR History
• 1997
• Feiner et. al
create the
“Touring Machine”
• First mobile
augmented reality
system
A Little History
•
•
•
•
•
1999
ARToolkit
Free/Open Source
Kato/Billinghurst
Good for intro to AR
Visual Tracking
(ARToolkit)
A Little History
• 2000
• Landmark
breakthrough!
• AR with Duran Duran
Hardware Requirements
• HMDs/Displays
• Rendering
• Tracking
Display
• Tabletops
• Tablets &
Phones
• HMDs
– Optical (see-through)
• Stereo?
– Video (closed-view)
Tracking
•
•
•
•
GPS, HA-GPS, GPS/RTK
Acoustic (IS-900)
Electro-magnetic (Polhemus)
Visual (ARToolkit)
AR Examples
Scenarios
• Medical
• Scientific
Visualization
• Expert/Technician
17
Scenarios
• Medical
• Scientific
Visualization
• Expert/Technician
18
Scenarios
• Medical
• Scientific
Visualization
• Expert/Technician
19
AR in Education
Navigation
Commercial
Nightvision
Military
General Problems
• Tracking Accuracy
• Optical vs. Video
– Optical – High Resolution/FoV. Synching/Delay
issues, brightness issues
– Video – safety: camera vs. eye offset
• Registration Problems
– location
– lag in rendering
• Weight/Battery Life
• Interaction
– Techniques
– Physical objects
Other Issues: Interaction
Paradigm Shift!
• Note:
– Media is cool
– Mobile is cool
– Media+Mobile must be really cool
• Unity supports scripting
– Basic programming concepts
– Reinforces OO
– Javascript or C# (or Boo)
• Provides scaffolding to remove barriers (or at
least lower them)
Moving into Mobile
Layar
Nearest Tube
Moving into Mobile (2007)
Magic Books
Moving into Mobile
• ARToolkit on an iPhone (2008)
Moving into Mobile (2009)
I MAED A GAM3 W17H AR 1N 1T – (not really, it’s called AAAARRRRRGH!)
Moving into Mobile
Let’s make a Magic Book
• Mark Billinghurst et al.
• The hard part is finding a tracker…
Mobile AR Gaming
• Need a good tracker
– GPS + Gyros is NOT good enough!
– Qualcomm has released their free tracker!
• Need a good/cheap display
– Phone is a window into the virtual world
• Need a way to make games/render
+
=
Easy AR!
Not anymore…
AR Preprocessed Images
(pictures + tracking information)
Into the ImageTargetTextures folder
Into the Streaming Assets / QCAR folder
“Page” Textures
Tracking XML file and data file
Very Cool Unity Mobile API
• Input.acceleration
– Returns orientation as a Vector3
• Input.GetKeyDown(KeyCode.Mouse0)
– Screen tap event
• iPhoneUtils.Vibrate()
– Makes the phone vibrate
• GPS/Location
– Unity 3.2 or Java LocationManager
interface and native wrapper
AR Demo
Using accelerometers
A safe place to get started
• ARToolkit
–
–
–
–
Free
Written in C
Uses fiducials
http://www.hitl.washington.edu/artoolkit/
• What you’ll need:
–
–
–
–
A webcam
A printer
A compiler
Graphics card?
The Qualcomm AR SDK
• Free
• Works with Android
– Java
– JNI
– C++
• Runs on Qualcomm chipsets
– HTC Incredible
– Nexus One
– Others
• https://ar.qualcomm.com/qdevnet/
Others
• Dekko SDK
– No training!
• 13th Lab’s PointCloud
– Dynamic building of the scene
The Burden of AR Gaming
• Printing out fiducials
– Is this really “mobile”?
• User training
– Do your parents know what AR is?
– How do you interact in AR environments?
– Don’t block the tracker!
– Don’t shake the camera!
The Burden of AR Gaming
• How do you interact?
– Point and click?
– Tactile feedback?
• Will people get physically tired?
User Acceptance is Key
•
•
•
•
Tracking is a challenge
Lazy players
AR must “go beyond” ( and why AR?)
Where’s the killer app?
5 Common AR Mistakes
(if you don’t believe me)
1.
2.
3.
4.
5.
Poor education
Not investing in content
Different AR app for every campaign
AR and 3D for the sake of AR
Make content engaging
http://www.mobilemarketer.com/cms/news/software-technology/14435.html?goback=%2Egde_67494_member_226398138
The Future of AR?
• Lots of hype right now
• Project Glass?
Thank you!
Questions?