Communication via the
Skin: The Challenge of
Tactile Displays
Lynette Jones
Department of Mechanical Engineering,
Massachusetts Institute of Technology
Cambridge, MA
Spectrum of Tactile Displays
Sensory substitution
Human-computer interactions
Navigation/orientation
Visual impairments
Tactile mouse
TSAS (Rupert et al.)
Hearing impairments
CyberTouch
MIT tactile display
DataGlove
Data Glove EST
Vestibular (balance) impairments
Tactile belt
Tactile Displays - CTA ADA Focus
Utilize a relatively underused sensory channel to convey
information that is private and discreet
• Assist in navigation or threat
location in the battlefield
• Increase SA in virtual
environments used for training
• Enhance the representation of
information in displays
•Function as an alert
•Orientation and direction
information
•Sequential activation of
array – vector conveys
“movement” in environment
•Effective in environments
with reduced visibility –
enhances situation awareness
Displacement (dB re 1 um peak)
Torso-based Tactile Displays
Vibrotactile Sensitivity
40
Forearm
30
Abdomen
20
10
Finger
0
-10
-20
0.1
1
10
Frequency (Hz)
100
1000
Development of Tactile Display
• Actuator (tactor) selection and
characterization
• Development of body-based system
(configuration of display, power,
wireless communication)
• Perceptual studies – optimize design
of the display in terms of human
perceptual performance
• Develop a framework for creating a
tactile vocabulary – tactons
• Field studies – measure the efficacy
of display for navigation, identifying
location of environmental events,
and examine robustness of system
(e.g. impact of body armor)
Characteristics of the Actuators Evaluated
Pancake
Motor
R1
Rototactor
Length
Diameter (mm)
12.8
4.0
14
25.4
6.4
Mass (g)
0.87
1.6
3
Peak frequency
(Hz)
110
(at 4 V)
103
(at 8.8 V)
200
(at 10 V)
Voltage (V)
Rated maximum
current (mA)
Current (mA)
at 3.3 V
492
175
Cylindrical motor
150
125
100
75
50
25
254
0
Rated: 3
Range: 2-4
130
0.5
8
2.5-8.8
10
65
170
0.7
0.9
1.1
1.3
1.5
1.7
1.9
2.1
Voltage (V)
100
Pancake Motor
91
39
56
Frequency (Hz)
Peak accel.
(ms-2)
200
Frequency (Hz)
Cylindrical
Motor
Rototactor
C2 tactor
Pancake
motor
75
50
25
0
0.6
Cylindrical
motor
1.0
1.4
1.8
2.2
2.6
3.0
3.4
Voltage (V)
Tactaid
(Jones, Lockyer & Piateski, 2006)
Prototypes 2003-2007
Tactile Display - Final Elements
Core components - Pancake motors, Wireless Tactile Control Unit
Contact area - ~ 300 mm2 (encased in plastic)
Input signal – 130 Hz at 3.3 V, sinusoidal waveform
Power – 9 V battery or 7.2 V Li-ion rechargeable, 2200 mAh
Display – vest, waist band, sleeve
Visual Basic GUI
Actuator Evaluation – Frequencies and Forces
Frequency (Hz)
180
160
140
120
100
80
0
1
2
3
4
5
6
7
8
9
10 11 12 13 14
Motor number
2.5
Force (N)
2
1.5
1
0.5
0
0
Mechanical properties not affected
by encasing motors
1
2
3
4
5
6
7
8
9
10 11 12 13 14
Motor number
(Jones & Held, 2008)
Actuator Evaluation – Tactor spacing and Intensity
Mechanical testing of skin
Peak Frequency on Skinsim (Hz)
90
80
70
60
50
y = 0.3765x + 11.46
R2 = 0.979
40
30
20
10
0
0
Skinsim with accelerometers
50
100
150
200
250
Peak Frequency in Impedance Head (Hz)
(Jones & Held, 2008)
Transitions – MIT Tactile Display
ARL
Investigated
the efficacy of tactile and multimodal alerts on
decision making by Army Platoon Leaders (Krausman et al., 2005, 2007)
Analyzed the effectiveness of tactile cues in target search and
localization tasks and when controlling robotic swarms (Hass, 2009)
Evaluated Soldiers’ abilities to interpret and respond to tactile cues
while they navigated an Individual Movement Techniques (IMT) course
(Redden et al., 2006)
Measured the effects of tactile cues on target acquisition and
workload of Commanders and Gunners and determined the
detectability of vibrotactile cues while combat assault maneuvers were
being performed (Krausman & White, 2006; White et al., in press).
The MIT tactile displays have also been incorporated into multi-modal
platforms developed by the University of Michigan, ArtisTech in the
CTA test bed, and Alion MA&D for a robotics control environment.
Questions addressed – MIT Research
• Can tactile signals be used to provide spatial cues about
the environment that are accurately localized?
• How does the location and configuration of the tactile display
influence the ability of the user to identify tactile patterns?
• What is the maximum size of a tactile vocabulary that could be
used for communication?
• Which characteristics of vibrotactile signals are optimal for
generating a tactile vocabulary?
• Can a set of Army Hand and Arm Signals be translated into
tactile signals that are accurately identified when the user is
involved in concurrent tasks?
Localization of Tactile Cues for
Navigation and Orientation
Navigation
– Way-finding
– Location of events – real and
simulated environments
– Control of robots
Experiments
10 subjects in each
experiment
Each tactor activated 5
times (randomly)
Subject indicate location of
tactor vibrated
Waist
Back
Navigation – Tactile Belt – One-dimensional Display
Navel
1
8
Left
2
7
3
Right
Inter-tactor distance 80-100 mm
6
Identification of tactor location
Eight locations – 98% correct
(Inter-tactor spacing: 80-100 mm)
Twelve locations – 74% correct
(Spacing: 55-66 mm)
4
5
Spine
(Jones & Ray, 2008)
Percent correct
Localization – Two-dimensional Display
100
90
80
70
60
50
40
30
20
10
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16
Tactor number
Identification of tactor location
16 locations – 59% correct (40-82%)
Within 1 tactor location: 95%
Inter-tactor spacing: 40 mm vertical
60 mm horizontal
Darker the shading, the more
accurate the localization
(Jones & Ray, 2008)
Results
• Spatial localization becomes more difficult as the number of
tactors increases and the inter-tactor distance decreases
• Two-dimensional 16-tactor array on the back is unable to
support precise spatial mapping, for example between tactile
location and visual target –driving or to highlight on-screen
information
• One-dimensional array is very effective for conveying
directions
Questions addressed
• Can tactile signals be used to provide spatial cues about the
environment that are accurately localized?
• How does the location and configuration of the tactile
display influence the ability of the user to identify tactile
patterns?
• What is the maximum size of a tactile vocabulary that
could be used for communication?
• Which characteristics of vibrotactile signals are optimal for
generating a tactile vocabulary?
• Can a set of Army Hand and Arm Signals be translated into
tactile signals that are accurately identified when the user is
involved in concurrent tasks?
Location and Configuration of Tactile Display
•Tested vibrotactile pattern recognition on forearm and back
•Fabricated 3x3 (arm) and 4x4 (torso) arrays both controlled by
Wireless Tactile Control Unit (WTCU)
•Tactile patterns varied with respect to spatial cues (location),
amplitude (number of tactors simultaneously active) and spatiotemporal sequence.
Results
Tactile Patterns
A
3
3
3
2
2
1
1
B
1
1
1
2
2
2
2
1
3
3
3
Up
C
Subject Response
Actual
Pattern A
B
C
D
E
F
G
H
A
80% 0% 1% 8% 2% 1% 8% 0% E
B
1% 86% 6% 3% 0% 1% 3% 0%
C
2% 0% 95% 0% 2% 1% 0% 0%
D
0% 8% 0% 90% 2% 0% 0% 0%
E
0% 0% 1% 1% 96% 2% 0% 0%
F
1% 4% 0% 1% 5% 86% 2% 1%
G
0% 5% 3% 3% 0% 3% 84% 2% G
H
0% 1% 0% 1% 0% 0% 1% 97%
Group mean percentage of correct responses – averaged
across tactors – 89%
Down
1
2
3
1
2
1
2
D
3
2
1
3
3
2
1
3
3
2
1
Right
Left
1,3
2
1,3
2
1,3
2
F
Left, right, left
1,
3
1,
3
1,
3
2
2
2
Top, bottom,
top
1, 2,
3
Blink X-shape 3 times
H
1, 2,
3
Blink center 3
times
(Piateski & Jones, 2005)
Back -Tactile Pattern Recognition
A
B
C
D
4
4
4
4
1
1
1
1
1
2
3
4
4
3
2
1
3
3
3
3
2
2
2
2
1
2
3
4
4
3
2
1
2
2
2
2
3
3
3
3
1
2
3
4
4
3
2
1
1
1
1
1
4
4
4
4
1
2
3
4
4
3
2
1
Up
97%
E
Down
99%
1, 3
2, 4
1, 3
2, 4
1, 3
2, 4
1, 3
2, 4
Left, right,
left, right
100%
F
1, 3 1, 3 1, 3 1, 3
Right
Left
100%
100%
1,2, 3,4 G
H
1,2, 3,4
2, 4 2, 4 2, 4 2, 4
Top, bottom,
top, bottom
100%
Blink corners 4
times
99%
Blink single motor
4 times
100%
(Piateski & Jones, 2005)
Tactile Vocabulary – 15-20 Tactons?
A
4
3
2
1
C
B
1
2
3
4
1
H
4
Down
Up
I
D
3 2
J
1,3
1
E
1 2
3
4
Left
Right
K
L
F
1,2
7,8
3,4
5,6
ICorners vibrate
together four times
G
Each corner
vibrates twice
Right, left,
right, left
N
M
O 1,3
2,4
2,4
1,3
4,8 3,7 2,6 1,5
2,4
1,3
Outer corners
then inner twice
Single tactor
vibrates four
times
Left, right,
left, right
Middle
two rows
Bottom,
top, bottom,
top
Diagonal
vibrates four
times
2,4
Two corners
vibrate in
turn twice
100
95
Percent correct
Top, bottom,
top, bottom
90
Mean: 96%
85
80
75
70
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
Tactile pattern
(Jones, Kunkel, & Torres, 2007)
A
3
3
3
1
1
1
2
2
2
2
2
1
1
1
3
3
B
Up
3
2
1
3
3
2
1
3
3
2
1
1
2
3
2
1
2
3
1
2
C
Down
D
Right
Correct responses (%)
Tactile Pattern Recognition – Effect of Stimulus Set
Left
1, 2, 3
2
E
3
1
3
2
3
F
3
3
3
2
2
G
H
2
1
2
2
1
1
1
3
3
3
1
1
1
2
2
2
2
2
1
1
1
3
3
B
Up
E
1
2
3
2
1
2
3
1
2
C
Down
1,3
F
1,3
Left, right, left
G
2
1
3
3
2
1
3
3
2
1
1, 2, 3
H
1,3
2
2
2
2
2
2
1,3
Blink center 3 times
Top, bottom, top Blink X-shape 3 times
Experiment 1B
C
D
E
F
G
H
Mean correct response rate:
62% in Expt 1A 85% in Expt 1B
IT: 1.48 bits
IT: 2.15 bits
Left
1,3
2
B
Tactile pattern
3
D
Right
1,3
20
1,3
Experiment 1A
A
40
A
2
Blink X-shape 3 times
60
0
2
Up and left
Up and right
80
1,3
1,3
1
100
Left, right, left
Confusion matrix (Expt 1A):
A misidentified as F, whereas F misidentified as D – errors not symmetrical.
Tactile patterns that “moved” across the
arm more accurately perceived than those
that “moved” along the arm
(Jones, Kunkel, & Piateski, 2009)
Summary of Findings
vs back – both provide effective substrates for communication
Array dimensions – marked effect on spatial localization
Asymmetries in spatial processing on the skin
Need to evaluate patterns in the context of the “vocabulary” used
Tactile vocabulary size – absolute identification vs communication
Interceptor Body Armor - no effect on performance
Arm
Tap on shoulder
Saltation
Direction and orientation
Field Experiments
Five subjects participated
Eight patterns with five
repetitions
Familiarization with visual
analog initially
Brief training period outdoors
Navigation path
Navigation using only tactile
cues, without feedback
100% accuracy for 7/8 patterns
presented
Single error on 8th pattern
Demonstrated that navigation is
accurate using only tactile cues
as directions
(Jones, Lockyer & Piateski, 2006)
Questions addressed
• Can tactile signals be used to provide spatial cues about the
environment that are accurately localized?
• How does the location and configuration of the tactile display
influence the ability of the user to identify tactile patterns?
• What is the maximum size of a tactile vocabulary that could be
used for communication?
• Which characteristics of vibrotactile signals are optimal
for generating a tactile vocabulary?
• Can a set of Army Hand and Arm Signals be translated into
tactile signals that are accurately identified when the user is
involved in concurrent tasks?
Tactons (tactile icons)
Structured tactile messages that can be used to communicate
information.
These tactons must be intuitive and salient.
A
B
4
4
4
4
1
1
1
1
3
3
3
3
2
2
2
2
2
2
2
2
3
3
3
3
1
1
1
1
4
4
4
4
Move forward
C
Turn around
D
1
2
3
4
4
3
2
1
1
2
3
4
4
3
2
1
1
2
3
4
4
3
2
1
1
2
3
4
4
3
2
1
Turn right
E
3,4
1,2
5,6
7,8
Turn left
1, 3
2, 4
1, 3
2, 4
1, 3
2, 4
1, 3
2, 4
F
Arm horizontal
1, 3
1, 3
1, 3
1, 3
2, 4
2, 4
2, 4
2, 4
Arm vertical
1,2, 3,4
Assemble/rally
H
G
1,2, 3,4
Communication tactons
Stop at next cone
Hop
Navigation tactons
Tactons for
hand-based
communication
Frequency
Duration, repetition rate
Waveform complexity
(Jones & Sarter, 2008)
Tacton building blocks: Relevant properties of each variable
Frequency
Intensity
Waveform
Duration
Location
Range:
0.4-1000 Hz.
Optimal
sensitivity:
150-300 Hz1
Absolute
thresholds
across body
sites: 0.07-14
m at 200 Hz3
Relatively
insensitive to
waveforms:
sinusoidal,
triangular,
square wave5
Burst duration:
80-500 ms
(typical)
Differential
thresholds: 750%5
Localization
accuracy varies with
body site7
Body site
influences
perceived
frequency
Changes with
increased
voltage to a
single tactor
and with
number of
tactors
activated
Amplitude
modulation of
sinusoids
effective for
varying
roughness of
signals6
Pulse repetition
rate (create
temporal
patterns rhythms)
Inter-tactor spacing
and array
configuration
important
Number of
pulses:
1-5 (typical)
Localization
superior near
anatomical points of
reference (elbow,
spine)7
Differential
thresholds:
18-50%2
Differential
thresholds: 530%4
(Jones, Kunkel, & Piateski, 2009)
Arm and Hand Signals for Ground Forces
Identify a set of structured
tactile messages (tactons) that can be used to communicate information.
4
4
4
4
1
1
1
1
3,
5
3,
5
3,
5
3,
5
2
2
2
2
1
2
3
4
2,
6
2,
6
2,
6
2,
6
3
3
3
3
1
2
3
4
1,
7
1,
7
1,
7
1,
7
4
4
4
4
Increase speed
Take cover
Danger area
Attention
Halt
Advance or Move Out
1
2
3
4
4
4
4
4
1, 3
2, 4
1
2
3
4
3
3
3
3
1, 3
2, 4
1
2
3
4
2
2
2
2
1, 3
2, 4
1
2
3
4
1
1
1
1
1, 3
2, 4
Each tactile hand signal was designed to keep some of the
iconic information of the matching visual hand signal
Hand and Arm Signals – Tactile-visual mapping
Correct responses (%)
100
80
60
40
20
0
A
B
C
D
E
F
G
Tactile pattern
Mean (N=10) percentage of correct responses (35
trials per subject) when identifying the hand signal
with both the illustration and schematic available
(black - 98% correct) and with only the illustration
available (red – 75% correct).
(Jones, Kunkel, & Piateski, 2009)
Questions addressed
• Can tactile signals be used to provide spatial cues about the
environment that are accurately localized?
• How does the location and configuration of the tactile display
influence the ability of the user to identify tactile patterns?
• What is the maximum size of a tactile vocabulary that could be
used for communication?
• Which characteristics of vibrotactile signals are optimal for
generating a tactile vocabulary?
• Can a set of Army Hand and Arm Signals be translated
into tactile signals that are accurately identified when the
user is involved in concurrent tasks?
Field Experiments – Concurrent activities
Hand Signal Identification
Percent correct
100
80
60
Walking
Jogging
40
Cognitive task
91%
91%
93%
20
0
1
2
3
4
5
6
7
8
Hand signal
Increase
speed
Take
cover
Attention Assemble Advance
to left
Nuclear,
Danger
biological and
area
chemical
attack
Halt
(Jones, Kunkel, & Piateski, 2009)
Conclusions
 Vibrotactile patterns easily perceived on torso with little
training and single stimulus exposure
 Demonstrated feasibility of using sites that are nonintrusive and body movements are not impeded
 Shown that the ability to perceive tactile patterns is not
affected by concurrent physical and cognitive activities
 Directional patterns are intuitive and can readily be used
as navigational and instructional cues
 Two-dimensional arrays provide greater capabilities for
communication, but one-dimensional arrays are
effective for simple commands
Acknowledgements
Edgar Torres
Amy Lam
David Held
Christa Margossian
Katherine Ray
Brett Lockyer
Mealani Nakamura
Erin Piateski
Jacquelyn Kunkel
Research was supported through the Advanced Decision
Architectures Collaborative Technology Alliance sponsored by the
U.S. Army Research Laboratory under Cooperative Agreement
DAAD19-01-2-0009.
References
Jones, L.A., Kunkel, J. & Piateski, E. (2009). Vibrotactile pattern recognition on the
arm and back. Perception, 38, 52-68.
Jones, L.A. & Held, D.A. (2008). Characterization of tactors used in vibrotactile
displays. Journal of Computing and Information Sciences in Engineering,
044501-1-044501-5.
Jones, L.A. & Ray, K. (2008). Localization and pattern recognition with tactile
displays. Proceedings of the Symposium on Haptic Interfaces for Virtual
Environment and Teleoperator Systems, 33-39.
Jones, L.A. & Sarter, N. (2008). Tactile displays: Guidance for their design and
application. Human Factors, 50, 90-111.
Jones, L.A., Kunkel, J., & Torres, E. (2007). Tactile vocabulary for tactile displays.
Proceedings of the Second Joint Eurohaptics Conference and Symposium on
Haptic Interfaces for Virtual Environment and Teleoperator Systems, 574-575.
Jones, L.A., Lockyer, B., & Piateski, E. (2006). Tactile display and vibrotactile
pattern recognition on the torso. Advanced Robotics, 20, 1359-1374.
Piateski, E. & Jones, L.A. (2005). Vibrotactile pattern recognition on the arm and
torso. Proceedings of the First Joint Eurohaptics Conference and Symposium on
Haptic Interfaces for Virtual Environment and Teleoperator Systems, 90-95.