UNCLASSIFIED
ADVANCED DISPLAYS FOR DISMOUNTED WARFIGHTERS
21-22 SEPTEMBER 2010
UNCLASSIFIED
Outline
•
•
•
•
Introduction
Types of Advanced Displays
Current Displays and Lessons Learned
Occluded Helmet Mounted Displays
• Head Mounted Displays
• Fused Night Vision Goggles
• Conformational Displays
• Tactile Displays
• Flex Displays
• See-through Helmet Mounted Displays
• Augmented reality displays and Enhanced Cognition
• Conclusions
Introduction
The purposes for using displays are:
• to present or hold up to view
•
to provide information or graphics on a screen
• to provide a representation of information
Now just a minute, earlier
you said x equals 4!
x+3=5
X=2
Introduction
The purpose of advanced displays is to
immerse warfighters in the operation so
they can experience and convey critical
information from real-time data feeds in
an intuitive, recognition-based manner.
Introduction
Multifunction Displays (MFDs)
Definition- a display surface which, through
hardware or software, is capable of displaying
information from multiple sources, in several
different reference frames. It may display
different groups of data one at a time or in a
combined fashion.
Multifunction, not multiple displays
Types of Advanced Displays
Headmounted
Occluded
See-through
Body-worn or Carried
Formed
Configurable
HMDs
GMDs
Augmented Reality
Displays
PDAs
Tactile Displays
Fused NVGs
Notebooks
Flexible Displays
Current Displays and
Lessons Learned
Current Systems
FBCB2
Color Helmet
Mounted Display
The Soldier is the most difficult “combat platform” to interface with!
 One Size never fits all
 Soldiers have different opinions
 Soldier Acceptability is critical
AN/PVS-14 NVG
Current Displays and
Lessons Learned
Land Warrior
2005
Land Warrior
“Manchu”
2007
Land Warrior
“Strike”
2008/2009
Nett Warrior
2012/2017
Helmet
Mounted
Display
Headset &
Mic
GPS
Antenna
Battery
Soldier
Control Unit
Navigation Module
Soldier
Radio/
Antenna
Computer
•
•
•
Command & Control
- Soldier: Voice & Data
- Leader: Voice & Data
Key Attributes
- Weapon: Subsystem
- Radio: EPLRS
- Interoperability: w/ FBCB2
Size & Weight
- 11 components
- 16 pounds
•
•
•
The Future – Nett Warrior
All BCTs
•
Command & Control
- Soldier: No capability
- Leader: Voice & Data
Key Attributes
- Weapon: Dropped
- Radio: EPLRS
- Interoperability: w/ limited external
assets
Size & Weight
- 9 components
- 11 pounds
•
•
Command & Control
- Soldier: No capability
- Leader: Voice & Data
•
Key Attributes
- Weapon: Dropped
- Radio: EPLRS
- Interoperability: - UGV interoperability for
IED Defeat
- Air/Ground Integration thru SADL
•
Size & Weight
- 7 components
- 9 pounds
•
Evolution From Land Warrior To
Ground Soldier System (Nett Warrior):
Current Path to Modernization
Introduction
Command & Control
- Soldier: Interoperable with Soldier Voice & PLI
- Leader: Digital Voice & Data
Key Attributes
- Weapon: Dropped
- Radio: EPLRS/JTRS (P3I) and Open
Architecture
- Interoperability: increased w/ external sensors
(JBC-P and Fire Control Systems)
Size & Weight
- Minimum essential
- 10 pounds (Objective)
Current Displays and
Lessons Learned
Icons
Cognitive Walkthroughs with Target Audience Soldiers
Several Iterations of User Juries for the Ground Soldier System
• Ease of training
• Intuitiveness
• Speed
• Errors
Introduction
Limited Hardware Evaluations
Occluded Head Mounted Displays
•
•
•
•
•
•
Currently planned for many systems (NETT Warrior, Common Controller,
etc.)
Lightweight
Relatively high resolution and easier to read
Require less power than many portable devices
Less glare than many portable devices
Hands free
Occluded Head Mounted Displays
POTENTIAL PROBLEMS
•
•
•
•
•
•
•
•
•
•
Both real world and HMD imagery must be within the user’s depth of
field, dark focus and dark vergence
Restricted field of view can impair performance and adversely affect the
ability to see the “whole picture” and reduce SA
Binocular rivalry
Competition for the attention of the wearer/attentional tunneling which
adversely affects dual task performance
Eyestrain and blurry vision
As HMD wearers move their heads, displayed objects in front continue
to be in front and this is unnatural
Motion sickness, often explained as being caused by sensory conflict.
Interference with night vision devices and weapon sights
Fogging due to weather
Potential for placing the musculoskeletal system of the head and
neck under increased levels of stress
Occluded Head Mounted Displays
Helmet Mounted Displays
• Scaling Robotic Displays: Displays and Techniques for
Dismounted Movement with Robots
– Soldiers performed significantly worse with the GMD than
they did with the HHD on course completion times, driving
errors, and the number of times they drove off course.
– Soldiers also preferred the HHD to the GMD and rated the
workload with the HHD lower.
Occluded Head Mounted Displays
Helmet Mounted Displays
Dismounted Land Navigation Study
PTN Tactile System: GPS sensor, electronic compass,
processing unit, battery pack, eight tactors placed equidistant, on
a belt around the torso, worn over T-shirt
LAND WARRIOR HMD GPS SYSTEM
HANDHELD GPS PLGR
1. 24 Soldiers (N = 21)
2. 3 routes, each with 2 waypoints & endpoint, each approx 1800m
3. 3 systems: Tactile, PLGR, LW HMD
4. Order of routes and systems counterbalanced
Occluded Head Mounted Displays
Helmet Mounted Displays
Dismounted Land Navigation Outcomes
System % Reached
PTN
100
PLGR
100
LW
100
17
16.5
16
15.5
15
14.5
14
13.5
13
12.5
10
8
PTN
LW
4
PLGR
2
0
Targets
LW
PLGR
Nav Speed
Target Detection
6
PTN
Occluded Head Mounted Displays
Helmet Mounted Displays
Dismounted Land Navigation Soldier Feedback
Very good
7
Good
6
Somewhat good
5
PTN
Neutral
4
LWS
PLGR
Somewhat bad
3
Bad
2
Very bad
1
Easy to learn
Easy to use
Easy to tell where
located
Ease to stay on
Route
Accuracy of
guidance
Occluded Head Mounted Displays
Fused Night Vision Goggles
Urban Enhanced Night Vision Goggle
Enhanced Night Vision Goggle
Occluded Head Mounted Displays
Fused Night Vision Goggles
Scene Interpretation
OVERLAY COLOR
• Contrasting color for FLIR
2
image vs. green I image
improves target detection
speed and range.
Bonnett, Redden, & Carstens, 2003
• Contrasting color also
assists in differentiating
terrain characteristics.
Occluded Head Mounted Displays
Fused Night Vision Goggles
High Thermal
Target Detection
Optimal Mix of
Patrolling
I2 and Thermal
Depends on Purpose
Navigation
High I
2
Urban Enhanced Night Vision Goggles
(UENVG)
•
•
UENVG is a prototype system used to determine the impact of adding a
Short Wave InfraRed (SWIR) capability into the existing ENVG system.
SWIR
– works in darker conditions than I2
– can see objects with great clarity on moonless nights because
night sky radiance (nearly all in SWIR wavelengths) emits 5 to 7
times more illumination than starlight )
– is not visible to the human eye but interacts in a similar manner as
visible wavelengths
– has shadows and contrast and can see through glass
Occluded Head Mounted Displays
Flexible Displays
• Near term Objectives
–Compare indoor and outdoor sunshine
readability
– Collect Soldier opinions on the technology
• Recent Efforts
– Evaluations at Fort Benning (2009, 2010)
Handheld and Forearm
Instructors from Warrior Training Center
– Evaluation at Fort Bliss (2009)
• Benefits
Survey of 9 prototypes
– Much less glare, lighter, rugged
– Much less power consumption
– Longer battery life
Conformational Displays
Flexible Displays
Near-term: Rugged, Low Power, Compact,
Lightweight
Far-term vision: Novel form-factors
Enabled by
FDC’s Unique
GEN II (37x47cm)
Pilot Line toolset
and People
21 Partners Representing:
Display Technology
Manufacturing Tool Suppliers
Materials Developers
Defense Contractors
Conformational Displays
Tactile Displays
Scalability of Robotic Displays: An Evaluation of Controller Display Options
• Evaluated 3 options for TALON controller Display:
- 6.5 inch Split Screen
- 3.5 inch Toggle Screen
- 3.5 inch Toggle Screen plus Tactile belt
Results
• 3.5 Toggle Screen associated with slower
performance and higher workload
than either (A) or (C). No difference between
(A) and (C)
• Tactile belt enabled a smaller screen while
providing cues that supported performance.
Conformational Displays
Tactile Displays
3.5”Toggle Display
6.5” Split Screen
3.5” Toggle Display with Tactile Belt
Conformational Displays
Tactile Displays
• Subjective Measures
– Task difficulty and SA ratings were worse for the toggle
display.
– Some Soldiers preferred the toggle/tactile display
because they felt they were able to pay more attention to
the driving display while wearing it.
– Others preferred the split screen display because of the
wealth of information provided.
– A fixed position camera and latency cause some driving
difficulties.
– The egocentric GPS caused problems when Soldiers
glanced away from the screen.
See-through Displays
Augmented Reality Display
•
•
The Last 18 Inches: How can you improve perception, squad coordination, and
decision making in tactical, high stress urban operations?
Squads and platoons are not well served by current information systems
–
–
–
–
•
Most equipment today distracts during operations
–
–
–
–
•
Overloaded: too much 2D, 3D, video data
Confusing: irrelevant data, old, out-of-scale, inaccurate
Optimized for strategic levels, not tactical block-to-block, not flowing from bottom-up
Its getting worse: soldiers/vehicles as sensors, geospatial & human network models
Pain in the neck: heavy, cumbersome, ill-suited for on the move
Can’t make use when and where needed, intended for desktop
Need for “on the go” interfaces, computing, sensing
Independent module development, proliferation of multiple methods (the TV remote problem)
High stress tactical decision making
–
–
–
Hard to remember important stuff
Too much to pay attention to
Need tools to aid attention and memory
See-through Displays
Augmented Reality Display
Squads Need
Omniscience
&
Telepathy
within the City Block
“I’ve got
your back!”
• Extend superiority into short-range combat inside urban and jungle
environments.
– Extend collaborative planning, rehearsal, and execution capabilities from company
to squad level.
– Enable quieter, non-linear, distributed, increased op tempo, 3D operations (e.g.
take down building from 3 directions)
– Enable effective, quick dynamic replanning
– Reduce chaos, fratricide, avoid surprises
See-through Displays
Augmented Reality Display
 Make data useful to platoon/squad/soldier in context
 Geospatial registration of annotations
 Agent-based systems to recognize intent, state
 Tools to leverage models for preview, execution, debrief
 Relevance at the level of seconds and meters
 Make communication for squads and platoons more effective
 Integrate planning, execution, and debrief in one system
 Improve remote (higher command) understanding of local
ops/data/history by virtual experience
 Make interaction with information “on the go”
 Overwatch feeding/filtering information selectively
 Interface sensors respond to natural body actions, non-screen
interfaces
 Advances in smaller, lighter, lower power, higher resolution displays,
computing, sensing enable head/body worn approaches
 Make overwatch automatic
 Plan via analogy
 Fuse, filter, prioritize information automatically (learn)
See-through Displays
Augmented Reality Display
• Civilian technology offers examples to
learn from and leverage…
– Madden Board and first down line – graphics registered in scene
images for enhanced communication
– Nintendo Wii as “natural interaction”
– NFL offensive play caller – in booth above field – overwatch feeds info
to coach and QB
– Nascar crew chief communication to driver
– “Mission Impossible” controller feeding data and info to agents in
mission/field
– Blackberry, iPhone, iPod, HUD (Private Eye) displays and interfaces
– Social information systems like GoogleEarth, Facebook, Wikipedia,
wwmx.org, rich info sources with engaging interfaces
• GoogleEarth is an example of local users creating overlays on global 3D
base model maintained/updated centrally
Conclusions
• The Dismounted Warfighter is the most difficult
customer for displays.
• Display technology continues to advance and today’s
failures may be tomorrows successes.
• Displays should be chosen based on mission
requirements, echelon level and on environmental
considerations.
• Human factors considerations and experimentation are
critical for effective display design.