Input Devices:
Trackers, Navigation and Gesture Interfaces
Input Devices
What is Virtual Reality?
“A high-end user interface that involves real-time
simulation and interaction through multiple sensorial
channels.” (vision, sound, touch, smell, taste)
What is a tracker?
The special-purpose hardware used in VR to measure
the real-time change in a 3D object position and
orientation is called a tracker.
Input Devices
Virtual objects have 6 degrees
of freedom (D.O.Fs):
•three translations
•three rotations
3-D System of coordinates of a VR object
Input Devices
Trackers measure the motion of “objects” such as user’s
wrist or his head vs. a fixed system of coordinates.
Technologies to perform this task:
Magnetic trackers (used to be prevalent but less used today);
Ultrasonic trackers (least used);
Mechanical trackers (special cases);
Inertial/ultrasonic trackers (new). (volunteers?)
Vision-based trackers (new and geared to be favorite)
GPS
Input Devices
Input Devices
Tracker characteristics:
 Measurement rate – Readings/sec;
 Sensing latency;
 Sensor noise and drift;
 Measurement accuracy (errors);
 Measurement repeatability;
 Tethered or wireless;
 Work envelope;
 Sensing degradation
Price.
Input Devices
Tracker characteristics:
Accuracy
Resolution
Real object position
Tracker position
measurements
Input Devices
Tracker characteristics:
Tracker data
Real object fixed
position
Signal noise
Time
Input Devices
Tracker characteristics:
Tracker data
Real object fixed
position
Sensor drift
Time
Input Devices
Tracker characteristics:
Tracker data
Real object
position
Sensor latency
Time
Input Devices
Tracker characteristics:
Tracker Update Rate
#receivers = #tracked sensors
A: dedicated electronics for each sensor
B: single electronic unit
Input Devices
Mechanical Trackers
Definition: A mechanical tracker consists of a serial or
parallel kinematic structure composed of links interconnected
by sensorized joints.
Input Devices
Mechanical Trackers
Pros
 Use sensors imbedded in exoskeletons to measure position;
 Have extremely low latencies;
 Immune to interference from magnetic fields and large metal
objects;
Cons
 Limit the user’s freedom of motion;
 Can be heavy is worn on the body
Input Devices
Push 1280 (Fakespace Inc)
Stereo display
Input Devices
Exoskeleton
structure
Interface
With
computer
Input Devices
Magnetic Trackers
Definition: A magnetic tracker is a non-contact position
measurement device that uses a magnetic field produced
by a usually stationary TRANSMITTER to determine the real-time
position of a usually moving RECEIVER element
Input Devices
Magnetic Trackers
Use low-frequency magnetic fields to measure position;
Fields are produced by a fixed source;
Size of source grows with the tracker work envelope;
Usually the receiver is attached to the tracked object and
has three perpendicular antennas;
Distance is inferred from the voltages induced in the
antennas – needs calibration…
Input Devices
Magnetic tracker with the old Data Glove
Input Devices Fastrack magnetic tracker system
Electronic interface
Source
Stylus
Receiver
Input Devices
Long Ranger source for the tracker system
(Polhemus)
Source
Metal in lab double ceiling
Polhemus Long Ranger
tracking errors (Rutgers)
Magnitude of Error Vector / Moving Tripod
Magnitude of Error Vector (inches)
35.0
30.0
25.0
Err (54)
Err (60)
20.0
Err (66)
15.0
Err (72)
10.0
Err (80)
5.0
0.0
0
10
20
30
40
50
60
70
80
Transmitter-Receiver Distance (inches)
Tracking error as a function of tripod height
Liberty LATUS – A Polhemus Wireless
Tracker
Uses wireless sources, now called “markers”, each with
a different frequency;
Each source position is measured by a “receptor” within
8 ft (50 sq. ft). 1 receiver can track 4 markers.
The system can have up to 12 markers and up to 16
receptors
Sampling rate is 188 Hz up to 8 markers and
drops to 94 Hz from 9 to 12 markers
Markers are battery powered up to 2.5 hours and weigh
2 ounces each.
 For one marker and one receptor accuracy is 0.04 mm
and 0.0012 degree at 1 ft range and drops afterwards.
Communication from base unit is RS232 or USB
Cost – UP TO $64,000
source (marker)
sensor
controller
Input Devices
Motion Star wireless tracker (courtesy of Ascension Technology)
Input Devices
Wireless suit (Ascension Technology)
Sensors: 20/suit
100 updates/sec
3 meters range
from base unit
Resolution<2 mm
and <.2 degrees
Electronic unit
(2 hours battery life)
1.7 kg (3.8 lb)
Input Devices
• Two opposite ERTs 3.63.6 m
• Two side-by-side ERTs 1.8 4.2 m
• Readings weighted to the closer ERTs
• Several Base Stations for several users
Motion Star block diagram
Input Devices
Magnetic Tracker Calibration
Use mechanical measurements to reduce errors;
Sensor noise – variation in measurement with no
real object motion – solved by over-sampling;
Size of errors grow from source outwards;
Errors both in position and orientation.
Input Devices
Magnetic Tracker Errors
 due to ambient noise:
e ambient = Kn (d transmitter-receiver)4
 due to metal:
Kr (d transmitter-receiver)4
e metal = --------------------------------(d transmitter-metal)3  (d metal-receiver)3
Input Devices
Magnetic tracker accuracy degradation
Input Devices
Comparison of AC and DC magnetic trackers
 DC trackers are immune to non-ferromagnetic metals
(brass, aluminum and stainless steel)
Both DC and AC trackers are affected by the presence of
Ferromagnetic metals (mild steel and ferrite).
Both are affected by copper;
AC trackers have better resolution and accuracy.
AC trackers have slightly shorter range
Input Devices
Input Devices
Ultrasonic Trackers
Definition: A non-contact position measurement
device that uses an ultrasonic signal produced by a
stationary transmitter to determine the real-time
position/orientation of a moving receiver.
Input Devices
Ultrasonic Trackers
Use low-frequency ultrasound to measure position;
Sound produced by a fixed triangular source (speakers);
Number of sources grows with the tracker work envelope;
The receiver is triangular and attached to the tracked
object and has three microphones;
Distance is inferred from the sound time of flight;
Sensitive to air temperature and other noise sources;
Requires “direct line of sight”;
Slower than magnetic trackers (max 50 updates/sec).
Input Devices
Ultrasonic tracker (Logitech)
Input Devices
Large-volume ultrasonic tracker (Logitech)
Input Devices
Optical Trackers
Definition: A non-contact position measurement
device that uses optical sensing to determine the
real-time position/orientation of an object
Input Devices
Optical trackers: a) outside-looking-in; b) inside-looking-out
Input Devices
Outside-looking-in
LaserBIRD optical tracker
Input Devices
Inside-looking-out
LaserBIRD optical tracker
 The two beams are offset
 The three vertices are
computed based on known
offset, geometry and angular
velocity of the beams
The average of the three
vertices is computed and sent
to the host at 240 sets/sec
2 ms latency
Range is smaller than that
of magnetic trackers
Outside-looking in: Vicon MX
Uses 4 Mpixel cameras with
own 120 LED array (infrared, or
visible red). Accuracy 0.02 of a
pixel,
 Camera has real-time onboard
image processing (masking and
thresholding)
 Resolution 2352x1728 @ 160
fps
 8 cameras are connected to a
MX net unit which then
communicates with the PC
Input Devices
Outside-looking in: Vicon MX
MX Link connects several MX
Net units for more cameras
 To interface with other devices
like a force plate, sensing glove or
eye tracker – use a MX control
unit.
User wears reflective markers
(small spheres).

Avatar Animation
Techniques
Input Devices
Outside-looking in: The
Rutgers Arm
Used in rehabilitation
 Arm and shoulder
movements are tracked with
active markers and an
infrared camera
Low cost

Input Devices
Inside-out optical tracker advantages
The best accuracy is close to the
work envelope.
Very large tracking surface and
resistance to visual occlusions
(line of sight).
HiBall 3100 wide area tracker
The sensor advantages are:
 High sampling rate (2000 Hz, 1000Hz for 2 sensors, 500Hz for 4);
 High accuracy (0.4 mm, 0.02°) and high resolution (0.2 mm, 0.03°)
 Impervious to metallic or ultrasonic interference;
 Very large tracking area (up to 40 ft x 40 ft), small weight (6 oz).
HiBall Optical Sensor
HiBall Optical Sensor interior
6 photodiodes
6 optical lenses
Signal conditioning
electronics
(courtesy of 3rdTech Inc.)
HiBall 3000 tracker
on an HMD
Lateral effect
photo diodes
Types of VR Applications
Beacon array modules
(6 strips with 8 LED/strip)




Vision-based tracking for the xBox (Kinect)
The xBox “Project Natal” eliminates the remote.
Natural interaction, and nothing to wear or hold
Uses vision tracking and real-time scene recognition, based in part
software from 3DV Systems, an Israeli company.
Is based on the Time-Of-Flight (TOF) principle, which generates
distance (depth) information for each pixel or object captured by a
camera. Depth data is determined by emitting pulses of infra-red
light to all objects in the scene and sensing the reflected light from
the surface of each object.
Kinect Channel
How-To-Use-Kinect-Face-Tracking-SDK
Inertial Trackers
Christos Giannopoulos
Introduction
 Tracking technologies may be grouped into three
categories: active, passive, and inertial.
 While active and passive systems rely on the occurrence of
either produced or naturally occurring signals, inertial
systems are completely self contained, sensing physical
phenomena created by linear acceleration and angular
motion.
How it Works
 Inertial sensors determine orientation and position through
Newton's laws of motion. The two devices that are key to perform
inertial tracking are gyroscopes and accelerometers.
 Accelerometers measure linear acceleration vectors with
respect to the inertial reference frame
 Gyroscopes measure the rotation rate relative to changes in
orientation.
Properties
Advantages:
• Very low latency
• Prediction based on directly sensed motion derivatives
• High resolution
• Negligible jitter over entire range
• Unrestricted range
• No line-of-sight problems
• Portability: no source to set up
Disadvantages:
• Susceptible to noise & calibration
errors due to gravity field
• Position & orientation drift can
occur
Applications
Due to the versatility of inertial tracking systems, a
multitude of applications for its technology have been
developed including:
•
Subterranean Navigation

Inertial navigation technology provides a passive self-contained tracking and
location solution for GPS-denied environments.
•
Military Simulation & Training
•
Entertainment Applications (“Mo-Cap”)
•
Utilizes inertial tracking for “motion capturing,” which many 3D animation and
videogame studios use today (Motion Capture)
•
Medical Settings
•
Oil & Gas Discovery
Sun SPOT System

SPOT - Small Programmable Object Technology

Small wireless battery powered experimental platform
programmed almost entirely in Java, allowing regular
programmers to create projects that used to require
specialized embedded system development skills.
Future Development
 Aside from the constant attempt to make these devices
smaller, faster and more precise, an interest in improving
inertial tracker relies on the concept of Hybrid Tracking.
References
1)
http://www.ensco.com/products-services/gps-denied-geolocationnavigation/personnel-navigation.htm
2)
http://www.sti.nasa.gov/tto/Spinoff2005/ch_10.html
3)
http://www.xsens.com
4)
Foxlin, Eric. “Miniature 6-DOF inertial system for tracking HMDs”
5)
You, Suya. “Hybrid Inertial and Vision Tracking for Augmented Reality
Registration”
6)
Lang, Peter. “Inertial Tracking for Mobile Augmented Reality”
7)
Wormell, D. “Advanced Inertial-Optical Tracking System for Wide Area
Mixed and Augmented Reality Systems”
8)
Bandala, Manuel. “Wireless inertial sensor for tumour motion tracking”
9)
http://www.sunspotworld.com
Input Devices
Hybrid Ultrasonic/Inertial Trackers
No
interference from metallic objects;
No interference from magnetic fields;
Large-volume tracking;
“Source-less” orientation tracking;
Full-room tracking;
A newer technology.
Input Devices
But…
errors a lead to decreased
accuracy since x= a t2
2
Errors grow geometrically in time!
Gyroscope errors compound position errors;
Needs independent position estimation to reduce
“drift”;
Accelerometer
Input Devices
IS 900 block diagram
VC 2.1- book CD
IS 900 software block diagram
Base unit
Sonic Strips
I-cube
Tracker components (InterSense Co.)
Degrees of freedom: 6
Resolution: 1.5 mm RMS
Angular: 0.05o RMS
Update rate: 180 sets/s max
– one station
Down to 90 updates/sec
- for four stations.
Latency 4–10 ms
Max tracking area: 900 meters2
(300 strips, 24 hubs)
Tracker components (courtesy of Intersense Co.)
I-Cube
Accel./gyro
Ultrasonic
emitter
InterSense Stereo Glasses tracker (courtesy of Intersense Co.)
GPS Trackers and Virtual Reality
Global Navigation Satellite System (GNSS)
- Developed by the US DoD
- Only fully functional GNSS in the world
- Uses 24-32 medium Earth orbit satellites
- Transmits data via microwave signals
Basic Concept of Tracking
- Four or more satellites are used to determine x, y, z,
and t
- GPS satellites broadcast their navigation signals at 50
bit/s
• ~ 65 – 85 milliseconds for the signal to travel
from the satellite to the receiver
- The signal transmits:
• Pseudo-random code
• Ephemeris
• Almanac
- GPS receiver calculates it position by precisely timing
the signals from satellites
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Typical GPS Receiver Characteristics:
- Refresh Rate:
• Automotive Needs: < 5Hz
• Military Needs: 5Hz to greater than 100Hz
- Accuracy: 5 meters
- Wireless receivers
- Work Envelope: Worldwide, restricted to view of
satellites
- Price: 50 USD - 5000 USD
1Hz vs. 5 Hz GPS
Receivers
Sony PSP GPS Receiver
Reception Frequency: 1575.42Mhz
Refresh time: Approx. 1 second
Precision: 5 meters
Uses:
• Navigation System
• Stargazing
• Several GPS enabled games
Tourality
- GPS multiplayer game for your mobile phone
- Object is to race to a certain spot before others in the
game
JOYity
- GPS enabled games for the
Android
- YouCatch – GPS enabled
version of ‘Manhunt’
- Roads of San Francisco –
Scavenger Hunt
- City Race Munich – Tourist
trivia game
The GPS Revolution
- iNap
- Shop Savvy
- Google Earth
- Locale
- Goskywatch
- Safety Net
- Sit or Squat
- Twitter
Military operations in Urban
Terrain (MOUT)
- Developed at U. New
Orleans
-Uses a web browser with
CosmoPlayer
-Up to 4 team leaders are
rendered in the scene
-Lander et al 2000
Accelerometer
Ultrasonic
emitter
InterSense Stereo stylus tracker (courtesy of Intersense Co.)
Input Devices
Input Devices
Navigation and Gesture Input Devices
Navigation interfaces allow relative
position control of virtual objects
(including a virtual camera);
 Gesture interfaces allow dextrous
control of virtual objects and interaction
through gesture recognition.

Input Devices
Navigation Input Devices
 Are
the Cubic Mouse, the trackball and the
3-D probe;
 Perform relative position/velocity
control of virtual objects;
 Allow “fly-by” application by controlling
a virtual camera.
Input Devices
VC 2.2 book CD
The Cubic Mouse
Input Devices
Trackballs
Input Devices
The MicroScribe (Immersion Co.)
Input Devices
Gesture Input Devices
 Are sensing gloves such as:
- Fakespace “Pinch Glove”
- 5DT Data Glove;
- The DidjiGlove
- Immersion “CyberGlove”
 Have larger work envelope than trackballs/3-D probes;
 Need calibration for user’s hand.
Input Devices
Finger Degrees of Freedom
Input Devices
Hand work envelope vs. interface type
The Pinch Glove (Fakespace Co.)
- no joint measures, but contact detection
The Pinch Glove (Fakespace Co.)
Resistive sensor one per finger
P5 Glove
Wrist Tracking
Finger bending resolution 0.5 degrees
0.1 inch wrist tracking resolution at 3 ft range
60 sets/sec $20 to $89
http://www.vrealities.com/demo_hi.mov Movie Link
One optical fiber/finger
A)
Roll/pitch sensing
5DT Data Glove
5DT Data Glove Ultra
100 datasets/sec, 12 bit A/D flexion resolution,
wireless version transmits data at 30 m, needs calibration
The glove interface: a) five-sensor version; b) newer design
5DT Data Glove - continued
Two sensors/finger plus
abduction sensors
Two gloves use one Bluetooth 2.4
GHz transmitter on the user’s belt
The glove interface: a) 14-sensor version; b) wireless kit
5DT Data Glove
Glove has fewer sensors
than hand joints …
Needs to infer distal
joint flexion angle
5DT Data Glove
The coupling of intermediate and distal finger joints
Input Devices
5DT Data Glove
Input Devices
5DT Data Glove
Linear calibration method
The DG5
glove
 Uses one flex sensor/finger;
 Angles are obtained by measuring
voltages
 100 gestures/sec
 Sensor resolution 10 bit (1024 points),
 cannot measure individual joints
Needs calibration
 less expensive (about $495)
The new DG5
VHand glove 2.0
 Uses one bend flex sensor/finger+ accelerometers for position/orientation;
 Angles are obtained by measuring voltages
 25 gestures/sec
 Sensor resolution 10 bit (1024 points),
 USB or wireless connection
Incorporates roll, pitch, tracking, 0.5 degree resolution
Needs calibration
 less expensive ($585 wired $785 wireless) www.dg-tech.it
The CyberGlove
 Uses 18-22 linear sensors – electrical strain gauges;
 Angles are obtained by measuring voltages on
a Wheatstone bridge;
 112 gestures/sec “filtered”.
 Sensor resolution 0.5 degrees, but errors accumulate
to the fingertip (open kinematic chain);
 Sensor repeatability 1 degree
 Needs calibration when put on the hand;
 Is expensive (about $10,000)
The CyberGlove
(Vertex Co.)
VC 2.3 on book CD
Specifications
Pinch
Glove
P5 Glove
5DT Data Glove
Number of
sensors
Sensor type
Record/sec
7/glove
(2 gloves)
Electrical
NA
5
sensors/glove
(1 glove)
Ink film
Sensor
resolution
Communication
rate
Wrist sensors
1bit
(2 Points)
Wired
(19.2kb)
None
60
records/sec
5 or 14/glove
(1 glove)
Fiber-optic
100(5DT 5W),
200(5DT 5)
8bit
(256 Points)
Wireless(9.600k
b)
Wired(19.2kb)
Pitch
(5DT 5 model)
Weird
USB 1.0, 2.0
LED trackers
DG5 Glove
DG5 VHand
5 /glove
(1 gloves)
?(ink film)
100
25 (DG 5 VHand)
10 bit
(1024 points)
Wired (19.2kb)
Wireless (DG5
VHand)
Accelerometers
(DG5 Vhand)
CyberGlove
18 or
22/glove
(1 glove)
Strain gauge
150(unfiltere
d)
112(filtered)
0.5°
Wired
(115kb)
Pitch and
yaw