Augmented Reality
Overview and Current Systems
Motivation: A Brief Scenario
Motivation: A Brief Scenario
The Problems:



The surgeon can’t look at the patient and the screen
at the same time.
Looking at the screen while working on the patient
isn‘t ergonomic.
The screen is 2D whereas the patient‘s body is 3D.
Motivation: A Brief Scenario
The Vision



The information should be provided where it is
needed.
The user should be enabled to access and interact
with the information ergonomically.
The information should be in 3 dimensions like the
application’s subject.
Roadmap
Defining Augmented Reality
A Taxonomy of Reality and Virtuality
Technologies
Fields of Application
Current Systems
Defining Augmented Reality
Combining Real and Virtual
Interactive in Real Time
Registered in 3-Dimensions
Defining Augmented Reality:
Combining Real and Virtual
Attention:


Combining does not
neccesarily mean
adding.
Combining does not
only include visual
information (NaviProject).
Defining Augmented Reality:
Interactive in Real Time
„Using AR technology,
users can interact
with a combination of
real and virtual
objects in a natural
way.“
In „Confluence of Computer Vision and Interactive
Graphics for Augmented Reality“
Definining Augmented Reality:
Registered in 3-Dimensions
Lightning reacts to the
position of the tangible sun.
A Taxonomy
of Reality and Virtuality
The Reality-Virtuality Continuum
A Taxonomy of
Reality and Virtuality
Centricity: Exocentric or Egocentric?
A Taxonomy of
Reality and Virtuality
Control-Display Congruence


Question: How intuitive is the control?
In AR: When I move the real object to the right hand
side, in what direction does the virtual one go?
Conclusion:



AR is situated near the real-role of the continuum.
AR uses an egocentric viewpoint in most cases.
AR tries to maximize the control-display congruence.
Technologies
Real World
Virtual World
User Interface
User Interfaces
Visualization
Tracking
User
Tracking
AR-System
User Interfaces
Visual
Acoustic
Haptic/Tangible
User Interfaces: Visual
Gesture Recognition
Example:

MIT‘s KidsRoom:
Temporal integration
User Interfaces: Acustic
2 Parts


Speech Synthesis
Speech Recognition
Odyssey 2001‘s dream
has not come true, yet.

So, Acoustic UIs are
combined with other
modalities.
Example:

SHEEP
Insert!
User Interfaces: Haptic/Tangible
Joystick, 3D-Mouse
Tangible Objects

Examples:
CAR, SHEEP and the
Studierstube‘s PIP
Touch gloves
Visualization: Ways of Visualizing
Visualization Devices




HMDs
PDAs
Projection
Monitors
3-Dimensionality




Separeted displays
Polarized projection
Anaglyphic projection
Time division
Visualization:
Mixing Real and Virtual
Optical see through
Video see through


Chroma keying: Mixing with video hardware
Mixing in the Renderer
What about occlusion?



Tracking all real objects
Depthmaps construction via stereo images
Special Projection Techniques
Visualization: Challenges
Contrast
Brightness
Resolution
Field of View
Safety
Tracking: Ways of Tracking
Optical Tracking
Markers


IR-reflecting spheres
Cards
Types of optical
Tracking


Inside out
Outside in
Tracking: Ways of Tracking
Time of Flight


GPS, DGPS, Galileo
Ultrasound
Magnetic Field
Sensing
Inertial Tracking


Gyro Meters
Accelerometer
Special: Eye Tracking
Tracking: Challenges
Static Errors




Optical distortion
Improper parameter
configuration
Mechanical
misalignment
Strategies
Carefullness
Algorithms for
autocalibration
Dynamic Errors

Hardware related
delays
Software related delays

Strategies

Higher processing speed
Video stream matching
Rendering in advance
Predicting the future
Fields of Application
Annotating the Environment
Displaying Auxiliary Sensor Information
Visualizing Artificial Data
Annotating the Environment
Navigation

Columbia‘s
„Touring Machine“
Police and Military

BARS
„… the success of a military operation in an
urbanized environment depends crucially on
being able to provide navigation and
coordination information to the individual marine
level.“
The US-Army's Concepts Division of the Marine Corps Combat
Development Command
Annotating the Environment
Repair, Maintenance
and Production


BMW: The „Intelligent
Welding Gun“
Boeing: „Augmented
wiring“
Displaying Auxiliary Sensor
Information
Medical



University of North Carolina:
Ultrasound images of a fetus.
The heARt-Project: Minimal
invasive heart surgery.
MIT: MRI and EEG to support
neurosurgery.
Successfully used
Visualizing Artificial Data
Design

Studierstube‘s Virtual Showcase
Visualizing Artificial Data
Scientific
Visualization
Education

Studierstube: 3DConstruct Application
Current Systems
Libraries

ARToolKit
Frameworks


Studierstube
DWARF
Current Systems
DWARF a Sample Framework
DWARF, a network of distributed
dynamically cooperating services.



Distribution
Heterogenity
Modularity
Current Systems
DWARF a Sample Framework
Services



Needs
Abilities
Service Descriptions
Middleware


Service Manager
Visual Frontend
Current Systems
DWARF a Sample Framework
Subsystems






Tracking
Presentation
Interaction
Context
World Model
Application
The Future
HMDs in the near future?
AR in sports and television
Большое спасибо за Ваше внимание.
Vielen Dank für Ihre Aufmerksamkeit!
Thank you very much for paying attention.
Discussion
Questions?
Excursus



CAR, SHEEP and Co. as motion pictures.
A close look at DWARF: The Source Code of
a Service.
What does it cost? A sample HMD and a
sample Tracking System.