Design of Monocular Head-Mounted Displays for Increased Indoor
Firefighting Safety and Efficiency
Joel Wilson, Dan Steingart, Russell Romero, Jessica Reynolds, Eric Mellers, Andrew Redfern,
Lloyd Lim, William Watts, Colin Patton, Jessica Baker, and Paul Wright
UC Berkeley Mechanical Engineering Department, 2117 Etcheverry Hall, Berkeley, CA 94720
ABSTRACT
Four monocular Head-Mounted Display (HMD) prototypes from the Fire Information and Rescue Equipment (FIRE)
project at UC Berkeley are presented. The FIRE project aims to give firefighters a system of information technology
tools for safer and more efficient firefighting in large buildings. The paper begins by describing the FIRE project and its
use of a custom wireless sensor network (WSN) called SmokeNet for personnel tracking. The project aims to address
urban/industrial firefighting procedures in need of improvement. Two “user-needs” studies with the Chicago and
Berkeley Fire Departments are briefly presented. The FIRE project’s initial HMD prototype designs are then discussed
with regard to feedback from the user-needs studies. These prototypes are evaluated in their potential costs and benefits
to firefighters and found to need improvement. Next, some currently available commercial HMDs are reviewed and
compared in their cost, performance, and potential for use by firefighters. Feedback from the Berkeley Fire Department
user-needs study, in which the initial prototypes were demonstrated, is compiled into a concept selection matrix for the
next prototypes. This matrix is used to evaluate a variety of HMDs, including some of the commercial units presented,
and to select the best design options. Finally, the current prototypes of the two best design options are presented and
discussed.
1. INTRODUCTION
1.1 Background
Fires account for more deaths in the United States than all natural disasters combined. Between 1992 and 2001, an
average of 4,266 people died and 24,913 were injured due to fires, not including the 9/11 tragedy. There are about 1.9
million fires every year, and billions of dollars lost in property damage. The World Trade Center attack itself cost New
York City $33.4 billion in property damage1, and over 2800 lives, 350 of whom were firefighters2. Research is needed to
create preventative, reactive, and assistive firefighting systems. The McKinsey Report following 9/11 is just one Case
Study arguing for more effective rescue operations.
The 9-11 incident brought new incentive and focus to communications and rescue in fires and other emergency response
needs. Shortly after 9-11, Richard Nowakowski, Director of the Office of Emergency Communications of Chicago, reevaluated the city’s emergency response needs, and concluded that all buildings over eight stories should produce digital
floor plans as a basic information and communication tool for fire rescue teams. Researchers from Professor Paul
Wright’s lab at UC Berkeley visited the Chicago Fire Department (CFD) to gather more information for the project.
Over 50 firefighters and three fire chiefs were interviewed about firefighting tactics, procedures, and their thoughts on
what they would want in an information delivering Head-Mounted Display (HMD)3. The trip spawned the Fire
Information and Rescue Equipment (FIRE) project at the Berkeley Manufacturing Institute (BMI).
The FIRE team conducted further interviews with firefighters from the Berkeley Fire Department4. The findings of all
the interviews were then reviewed. It was found that firefighters must often make a best guess of where the fire started
and where it is traveling in a building by determining which alarm activated first (if possible), looking for smoke issuing
from windows, and by word of mouth from building occupants. From this information, they then guess where the safest
and most effective place to enter the building is. Further adding to the guesswork, they do not have maps with them
when they are in a building. This is because paper floor plans are impossible to carry and read while trying to fight a fire
and rescue victims. Instead, firefighters may navigate by unreeling a rope as they go, tying knots to mark important
locations, or marking doors with large crayons. When the smoke becomes thick, the “left-hand rule” is used: they drag
their left had along the wall so as not to become disoriented. Thermal cameras, when available, are also used to navigate
Helmet- and Head-Mounted Displays X: Technologies and Applications,
edited by Clarence E. Rash, Colin E. Reese, Proceedings of SPIE Vol. 5800
(SPIE, Bellingham, WA, 2005) · 0277-786X/05/$15 · doi: 10.1117/12.603590
103
smoke. These methods do not always work, as there have been incidents in which firefighters have become lost and
suffocated to death in thick smoke5.
1.2 FIRE project description
The intent of the FIRE project is to create hardware and software tools to improve
firefighting safety, efficiency, and effectiveness. The strategy is to use a wireless sensor
network (WSN) called SmokeNet to track firefighters in large buildings and give them
and the chief in charge, or Incident Commander (IC), important information about their
location, the fire, and their health. In this scenario, each firefighter has a computer
attached to his or her SCBA tank or in their jacket, in which is stored a set of floor plans
for the given building. A SmokeNet system preinstalled in the building tells the
computer which floor plan level to display on the HMD, e.g., the twenty-third floor. The
firefighter sees a map of the building-floor with a “You Are Here” symbol showing
location, in the lower right corner of their field of view (Fig. 1). Other symbols of a
different color and/or shape show locations of other personnel on the same floor plan.
Similar floorplans are seen on the IC’s laptop as he or she coordinates the fire with the
deployed fire crew. The IC sees a layered series of 2D floor plans for the entire building,
and can scroll up and down through the floor plans. For example, a six-story building has
six 2D floor plans stacked vertically (Fig. 2). Fire departments would already have floor
plans in an electronic format.
Floor plan
Figure 1: Concept
illustration of HMD
location in mask.
Figure 2: Concept illustration of the IC’s display of firefighter positions and fire conditions.
This system is aimed at large buildings such as high rises. The city of Chicago has mandated that all owners of buildings
over eight stories high must submit electronic floor plans as preparation for such a system. Thus, building floor plans
made of paper are redrawn in CAD and simplified for ease of navigation. This reduction of detail is crucial, as
firefighters must be able to read the plan at a glance. The main purpose of the floor plan is navigation, thus only basics
like walls, portals, room and floor numbers, standpipes, and stairways need to be shown. These plans would then be
loaded onto computers for the ICs and one PDA sized computer attached to every firefighter, so that each department
would have all the floor plans for its jurisdiction. If the building does not have a pre-installed SmokeNet, firefighters
would distribute ad hoc beacons on walls as intelligent breadcrumbs according to an intuitive deployment protocol (e.g.,
“place beacons approximately every 15- 20 feet”). This would create an evolving SmokeNet, where as more firefighters
place beacons throughout the building, the system would be able to track more personnel and the fire over a larger area.
2. USER NEEDS STUDIES
Much important design information was gained through user needs studies. Over 50 firefighters and three fire chiefs
from the Chicago and Berkeley Fire Departments were interviewed. Input was obtained on what they did and did not
want in an HMD. Chicago was interviewed first, after which two prototypes were built based on what was learned. The
prototypes were taken to the Berkeley Fire Department for a second round of interviews and feedback on the prototypes.
In the first round of interviews with Chicago, four different areas where information technology could be used to
enhance fire rescue operations were defined: environmental analysis, location technology, coordination and
104
Proc. of SPIE Vol. 5800
management, and health monitoring6. Environmental analysis refers to equipment on personnel that can be used to
describe the conditions a firefighter is encountering. This can include smoke, temperature, and chemical exposure.
Location technology refers to tracking firefighters as well as pinpointing victim location by means other than traditional
manned searches. Coordination and management involves automating and enhancing aspects of the incident command
system. Finally, health monitoring will allow ICs to better manage rotations of firefighters during extended operations,
as well as increase chances of rescuing incapacitated firefighters.
The firefighter’s HMD is a crucial part of the above four focus areas. The firefighters gave general remarks on what they
wanted. These included having a small, durable, low maintenance system that is easy to use and not too distracting to
the user. One suggestion was mounting the HMD low in the mask so that it would not block their vision when crawling
or be distracting when it is not needed.
The second round of interviews was with the Berkeley Fire Department at their fire tower, where drills are practiced.
The two prototypes shown in section 3 (Figs. 3 and 4) were demonstrated, each offering unique advantages and
disadvantages. The firefighters tried on the prototypes, viewed simple floor plan images of the fire tower, and gave their
comments on the designs. From this demonstration, user needs were compiled and written in terms of HMD
requirements. An example is shown in Table 1.
Table 1. HMD user needs assessment feedback from the first Berkeley Fire Tower demonstration.
User: Berkeley Fire Department
Products used: Draeger Panorama Nova mask with “FireEye” HMD
Type of use: frequent and hard use
Question
Typical Uses?
Statement
Mask protects my face from debris and hot
gases
Keeps out dangerous gases and allows me
to breathe easily.
Is used harshly, e.g., thrown into the fire
truck
Works in sprinklers, rain
Helps me navigate and fight the fire
Likes?
Allows me to clearly see where I’m going
Fits my face just right
Adjustable focus
Small
Lightweight
Lasts long enough for all emergency
responses
Easy to read
Relatively inexpensive
Can’t see bottom of screen
Screen not level
HMD screen sits level
Interferes with view of reality; distracting
HMD is small enough and positioned low such that it is not overly
distracting and does not block view
Electronics are robustly packaged for good durability and minimal
size
LCD and other electronics work in hot and cold extremes, ideally
–20 to 60 °C (in mask)
HMD allows for glasses insert
Electronics exposed- too delicate
Suggested
Mask and HMD operate properly after being repeatedly thrown,
dropped or shock loaded
Mask operates normally under water spray; HMD is water
resistant
HMD shows a floorplan, location of self, fire, and others, escape
routes
HMD does not compromise one’s view and does not make mask
fog up.
HMD does not affect the fit of the mask to face
HMD focus is adequately adjustable
HMD does not obstruct vision
HMD does not create additional neck or other fatigue or
significantly imbalance the mask
HMD has low power consumption and lasts for hours on batteries
HMD has adequate font and picture sizing that can be read at a
glance; high contrast and brightness; symbology is intuitive
HMD is inside the mask, giving protection of mask and no need to
flip down a display => hands free
HMD is affordable to fire department
HMD is adjustable such that all users can see all of the screen
Integrated inside the mask
Dislikes?
HMD Requirement
Mask shields face from thermal and projectile dangers. HMD
does not affect shielding ability; HMD is heat resistant
HMD does not negatively affect seal to outside air or breathing
Concerned that the LCD display won’t
work in winter
Can be used with glasses kit
Proc. of SPIE Vol. 5800
105
Improvements?
Can be easily adapted to individual user
needs
Easy to clean and solution cleanable
Zoom capability
Infra red
Orientation information
Floors need to be labeled
HMD can be positioned for either eye and adjusted for optimum
orientation
HMD can easily be solution cleaned
HMD UI has at least two easily switched zoom views
HMD compatible with infra red mask systems
HMD shows orientation by building sides 1 2 3 4 and escape
routes
HMD shows and automatically updates via SmokeNet the current
floor number
3. PREVIOUS BMI PROTOTYPES
The requirements of an HMD for firefighters are many and at times conflicting. For example, the unit must be of good
optical quality and very high durability, yet be low cost; it must display a variety of important information, and
command attention when necessary, yet be usable at a glance and not distract when unneeded. The design of prototypes
will be discussed in consideration of the firefighter’s needs.
The firefighters have high standards for their protection equipment. Self Contained Breathing Apparatus (SCBA)
facemasks were carefully retrofitted in integrating an HMD to maintain comfort, safety and familiarity. Mask integrity
cannot be jeopardized. The final products must be rugged designs with durable connections to an externally worn HMD
computer. In future tests, firefighters will be queried as to their preferences and redesigns will be based on their
feedback.
An external clip-on style HMD was the first prototype developed. It used a Poma wearable computer from Xybernaut.
The Poma has a see-through HMD with a 640x480 LCD. A housing was built in order to clip the Poma onto the outside
of a fire mask, or onto a firefighter’s helmet.
It was decided that the display would be better protected inside the mask. Firefighters who were shown the external
design commented that they would rather have it inside the mask where it will not be knocked off or become damaged
by extreme heat. Furthermore, they wanted it mounted low in their field of view (FOV), rather than extending down
from above. When the display is mounted in front of or just above the firefighter’s center of vision, it distracts them
when they are trying to stay low or even crawl to stay under smoke and hot gases.
A summer student named Jess Reynolds created the HMD shown in Figure 3. This
HMD was positioned low in the user’s FOV based on feedback from a demo with the
Poma. The 320x240 color AMLCD and its electronics are housed outside the mask,
and part of the optics are inside. The housing was designed with SolidWorks 2003,
and built on a Stratasys 1650 fused deposition modeling (FDM) rapid prototyping
machine. Using rapid prototyping has advantages of easy reproducibility and lower
cost in high volumes.
The major concern with this design is its protrusion through the face shield.
Approximately one-third of the unit is inside the mask for a compromise between
comfortable eye clearance and minimized snagging. This necessitated tampering with
the airtight factory seal. Having a sealed system is very important so that harmful and
hot gases do not enter the user’s respiratory system, and oxygen from the SCBA tank
does not leak out. Although a good seal can be made, it is more risky to have an HMD
mounted in this way. It also requires major and permanent modification to the
Figure 3: HMD is mounted
facemask. This is a problem if a fire department decides they want to be able to
through the face shield.
remove an HMD for certain tasks, such as cleaning the mask or using masks for nonfirefighting purposes. Finally, this design involves more manufacturing steps, leading to higher cost per unit.
106
Proc. of SPIE Vol. 5800
This HMD does not use as much space inside the mask, creating a less impinging, cleaner interface. Outside the mask, it
does not protrude as much as a design that is fully external, yet it does stick out enough to possibly hit something.
Furthermore, it is a permanent mount, so when in storage or lying on the floor of the fire truck, the HMD may be
frequently and forcefully knocked or squished by accident. This raises durability questions. The unit could easily be
broken off the face shield, leaving a large hole in the shield and rendering the facemask useless. Furthermore, a wire
extends out from the end of the HMD. This wire would require much reinforcement and protection so as not to melt,
become snagged, or break.
Eric Mellers, another summer student, took a different design approach to the previous designs. The inherent protection
of the mask was utilized by building a very small HMD that fits inside it (Fig. 4). All
of the electronics are left bare in order to minimize the space taken. They are held in
custom machined plastic housings, and the optics are held in a custom machined
aluminum housing. The HMD is mounted to the nosepiece of the mask. There is
already a hole in the side of the nosepiece, creating a natural mounting location. Two
more holes were punched for the PCB boards.
Particular attention was given to durability and
human factors in designing the wire routing. A
clever design is used to connect the LCD to the
control unit and power supply. Wires are routed
through the SCBA respirator, and copper bands
resting in grooves machined into the quick
disconnect allow firefighters to connect air and
Figure 4: Mellers’ internal
power supplies in one step (Fig. 5). With this
HMD with bare electronics.
system, firefighters would not have to do anything
more than they currently do in order to connect the HMD. The HMD is easier to don,
and any wires that could be snagged during an operation are removed.
Figure 5: Modified respirator
for quick HMD connection.
This HMD was demonstrated to firefighters, during which they were asked to give feedback on the design. They were
concerned about the durability of the delicate exposed electronics and connecting cables, a lack of user adjustability, and
difficulty in removing the unit for cleaning. Delicate wires connect the PCB boards, so nothing can be moved for user
adjustments. Removing the HMD for cleaning the mask risks breaking the fragile wires and cables, rendering the unit
impractical. Furthermore, the LCD and optics are mounted flush with the nosepiece, resulting in a tilted display that is
awkward to view. These issues would be dealt with in the next prototypes.
4. POTENTIAL COMMERCIAL HMDs FOR FIREFIGHTING
There are many groups working to improve firefighting and emergency response tactics, but few are developing a
dedicated HMD designed for the rigors of this work. Related work includes the SaabTech Group, which does HMD
research for a variety of applications7; project LISA in the GeoIT group at the Swiss Federal Institute of Technology,
applying wearable computing to firefighters8; the UC Berkeley Siren project on enhanced firefighter communications9, 10,
Chris Baber’s work on wearable computing for fire and emergency response11,12; the CMU VuMan HMD project13; TriSentinel’s first responder wireless communications work14; the LifeLine project for improved firefighter
communication15; and the HEMIND enhanced helmet system16. Some groups are involved with the Emergency
Response Technology (ERT) Program, a group sponsored by the Department of Homeland Security and NASA, and
administered by the National Technology Transfer Center. ERT’s number two of their Top 10 Technology Needs is an
integrated spatial recognition, tracking, health monitoring, and alerting system for emergency response17. Some of the
projects are run by companies modifying their commercially available HMDs, such as Xybernaut and Microvision. This
section will evaluate a selection of commercial HMDs that appear promising for firefighting.
MicroOptical has multiple small and lightweight HMDs, including the EG-8 and the SV-6. The EG-8 is a see-through
HMD integrated into a custom pair of clear eyeglasses. The FOV is small at 10°18, and the cost is high at $8,00019. The
SV-6 is an $1800 direct-view, focusable from 35 cm to infinity, with a 16° FOV. These HMDs could be mounted inside
of the protective face shield.
Proc. of SPIE Vol. 5800
107
The Kaiser Electro-Optics ProView SO35, developed for the Army Land Warrior Program, is a $9,500 direct view unit
with an SVGA+ (852 x 3 x 600 pixels) active matrix OLED from eMagin20. It has a wide operating temperature range of
-35 to +65 °C21, and excellent image quality. The display swings down in front of the eye, compromising balance and
situational awareness, requiring most users to stop and close one eye to read the display22. It may be too large to fit in a
mask, requiring special packaging for intense heat.
The Human Interface Technology Laboratory at the University of Washington developed the virtual retinal display in the
early to mid 1990s. A company called Microvision has taken this technology to the next step, now offering the Nomad
HMD for $4,000. An advantage of the Nomad is its use of a laser rather than a flat panel display. This in theory can
make the device quite small, light, and able to provide very high resolution and brightness. The laser can be adjusted
intense enough to easily see images against bright backgrounds, or dim for low light conditions. It is currently too bulky,
however, to fit within a firefighter’s mask, and probably not durable enough to survive outside the mask without extra
protection, considering an operating temperature range of only 0-45 °C23.
Interactive Imaging Systems sells an HMD called the Second Sight M2100. It runs off of CompactFlash and is designed
to connect with a PDA. It uses a color AMLCD with VGA resolution. It weighs only 2 ounces and has a built in
microphone, which could be useful for firefighters as an additional communication tool24. The M2100 appears to be
compact enough to fit inside a firefighter’s mask, giving a protective advantage over external systems.
Ingineo has a sunglasses mounted HMD called the Eyetop. One of its initial user groups is radio-controlled model
helicopter enthusiasts. A Kopin 320x240 color AMLCD is mounted on the right or left side of a pair of sunglasses. The
two main advantages of this unit are its compact size and low price of $40025. Price is a major concern for some fire
departments. It is small enough to fit inside a firefighter’s mask, and inexpensive enough that two or three of these can
be purchased for the price of other HMDs. It is light at 60 g, and the focus is adjustable for multiple users from 2 m to
infinity. The compromise for low price is lower resolution and poorer image quality than most other HMDs.
Liteye has designed two promising HMDs using OLED technology. The Liteye 400 has an SVGA AMOLED from
eMagin, and a 38 deg FOV26. It is too large to fit inside a firefighter’s mask, but it has been heavily ruggedized.
Additional heat protection may allow its use outside the mask but behind the fire helmet heat shield. A newer model
called the 500 will soon be available. This model has optics that allow see-through capability and high image quality. It
also uses an SVGA color AMOLED, and it may be small enough to fit inside a mask.
The BMW Technology Office and Design Works USA partnered to create a custom HMD for their BMW/Williams
Formula 1 racing driver, Ralf Schumacher. The design uses an AMLCD and a free-form surface (FFS) prism to project a
virtual image into the user’s eye27. Use of the FFS prism creates a compact and clean looking system. It is mounted in
the lower right corner of the driver’s FOV, which is where a firefighter would like to see the display.
The application is similar to firefighting in that the user must not be distracted from his or her work by the display.
Much of the time, the user will not need the display, and so it should not obstruct the view of the real environment. Yet
the display must be clear and simple to convey helpful information at a glance. For the racecar driver, a brief glance at
the display may alert the driver of upcoming track conditions, allowing accident avoidance. The ability to quickly
assimilate the information allows more concentration time on driving. Similarly, for the firefighter a glance down should
tell important information such as location and critical commands from the IC. With proper interface design, a brief
glance at the information presented could enhance situational awareness.
The above designs have been briefly evaluated for firefighting purposes. Given the firefighting requirements, the
BMW/Williams, Liteye 500, and MicroOptical EG-8 HMDs seem to best meet the requirements, primarily because they
fit within the facemask. The mask provides a great deal of protection from heat, water, and impacts. The HMDs could
be mounted low such that the user would look slightly down to see it, and up to see around it. This would make it less
distracting when not needed. Ultimately, modifying a currently available HMD to fit the confines of a facemask, and
repackaging for increased durability, will be the most cost effective method for meeting the design requirements.
108
Proc. of SPIE Vol. 5800
5. CURRENT PROTOTYPES
5.1 Concept selection
During the brainstorming process for the next round of prototypes, HMD characteristics from the user needs study were
compiled in a screening matrix used to weigh costs and benefits of different HMDs. Table 2 gives a screening matrix
comparing six different HMD concepts that were considered for prototyping. These HMDs were chosen based on
apparent promise as good candidates for firefighters. The top three were chosen for prototyping. There now exist more
HMDs, such as the Liteye 400 and 500, which appear promising but were not known of prior to creating the new
designs. Concept A, being a direct view similar to Mellers’ HMD but repackaged to be more rugged, is used as the
baseline to which the others are compared.
Table 2. Screening matrix of HMDs considered. HMDs are compared to each other, where A is the baseline.
Concepts
Direct View (similar to Mellers’ mask in Fig. 4) A
Projected OLED in mask
B
Microvision Nomad
Removed with fiber bundle image relay
C
D
OLED on/in face shield
See-through
E
F
Notes
SCREENING MATRIX
Concepts
Selection Criteria
A B C D E F
Cost
Ease of manufacturing
Ease of installation
0
0
0
-
0
+
0
- C: $4000 vs $500
- E: in face shield
0 B: face shield silvered
Adjustability
Low maintenance
0 0
0 0
+
0
-
0
0 C: external add on
0
Durability/Longevity
Comfort (wieght, eye strain, eye relief)
0 0
0 0
- - - 0 E: not removable for washing mask
+ + + + C,D,E: eye relief; C,F: eye strain
Safety
Optical quality
0 0
0 0
0
+
0
-
0
0
0
0
Compatibility with different masks
0
-
0
0
0
0
Sum +'s
Sum 0's
0 0
10 6
3
4
2
2
1
5
1
7
Sum -'s
0 4
3
6
4
2
Net Score
0 -4 0 -4 -3 -1
Rank
1 5
Continue?
Y N Y N N Y
1
5
4
3
Concept A was found to be one of the best choices due to its simplicity. Simplicity positively affects important areas
like cost, ease of manufacturing, and durability. Concept B would use a bright OLED microdisplay and projecting optics
to project an image onto the face shield. This may at first seem a good idea, but has the following problems. The face
shield is curved, complicating the optics needed to project the image. The shield would need to be coated to act as a
screen for the image. Finally, the shield is closer than the near point of the eye, meaning another lens would be needed
to focus the image. Overall, the system is too bulky and expensive.
Proc. of SPIE Vol. 5800
109
Concept C, the Microvision Nomad, has a cost disadvantage. Advantages include its ease of adjustability for
comfortable viewing, and its high image quality. According to Microvision, this system has the potential to become
smaller and less expensive; therefore it has been kept as a candidate. Concept D also rates high in comfort, but is low in
image quality. It would have the display, its electronics, and most of the optics removed from the mask and located
safely in a box on the firefighter’s back. A special fiber optic cable composed of 10,000-30,000 coherent fibers would
transport the image. The cable would enter the mask and have an eyepiece on the end for viewing. This system would
take up the least amount of space in the mask, creating a comfortable and clean looking interface. Each fiber would take
relay light from a few pixels, with a higher fiber count in the bundle giving a higher resolution. The problem is that even
with 30,000 fibers, the eye will see the ends of the fibers. The ends look like a honeycomb pattern, which can be
distracting. This is a minor issue when there are 30,000 fibers. Greater problems are the high cost of such cables,
placing a quick connect/disconnect in the optical cable that keeps image alignment of the thousands of fibers, and
questionable durability of such a fiber bundle. These cables cost many thousands of dollars, cannot be stepped on or
bent sharply, and are heat sensitive.
Concept E places a thin OLED or other display either within the plastic structure of the face shield or on its inner
surface. OLEDs are currently not thin enough to fit completely immersed within the thin plastic face shield, and it could
be expensive to do this. The curved surface requires a currently unavailable flexible or curved display. Optics would
then be needed to make the screen appear flat and far away. The manufacturing would be tricky and would probably
involve a separate plastic insert within the face shield. This may compromise the integrity of the shield, which is not an
option. Furthermore, if the face shield were to become scratched or damaged beyond repair, it and the display would
have to be replaced. This would increase cost significantly, whereas other systems are not as directly linked to the
mask’s durability. An OLED mounted on a flat surface on the inside of the face shield surface would solve many of
these problems. It would still be too close for the eye to focus on, thus a simple magnifier optic could be used. In this
case, the system becomes the direct view concept A with an OLED instead of an LCD. The OLED may be slightly
thinner, but the optics already fill the majority of the space with an LCD. The use of a Fresnel lens has been suggested,
as in the back window of an RV. They are thinner but have poor image quality, which is why they are generally used
only to focus light, not images, in overhead projectors and some lighthouses. Custom thinner optics may be too
expensive for fire departments unless they were produced in high volume.
Concept F is the see-through design. This is the optical design that most air force HMDs use, such as Apache helicopter
aviators. A see-through system is especially useful when one needs to frequently glance at the HMD while performing a
dynamic operation, such as flying a plane, driving, or walking. It is unclear whether firefighters would be better served
by a see-through system. One possibility is that firefighters will not usually be glancing at the HMD, as they do not need
one for many current operations. It may be used only in a critical situation where the information is vital, and an
occluded display may better relay information in extreme conditions such as against a bright fire background. On the
other hand, firefighters may end up using the HMD frequently while walking in order to navigate, making a see-through
system potentially better for obstacle avoidance. User testing will identify the pros and cons of each.
5.2 Design issues
In designing the most recent prototypes, many design issues were taken into account, and problems with the previous
units were addressed. A monocular design was again chosen for its simplicity, lower cost, relative ease of fitting inside a
mask, and potentially less distracting nature. The placement also remained the same, inside the mask in the lower right
corner of the user’s FOV. This is the design currently preferred by the firefighters with whom we have worked.
The next decision was whether the prototypes should be see-through, direct
view, or both. Both types were developed to determine which would be
more effective for firefighters. Moreover, the screening matrix showed that
both designs should be developed. An initial cost design target was set at
$500 or less per unit, not including the wearable computer. Two Ingineo
Eyetops were purchased as the base platforms for both designs. The Eyetop
was chosen for its small size, low weight, and low price. The internals are
shown in Figure 6. It also has adjustable focus, brightness, contrast, and
hue. It appears to be a durable unit, with the stipulation that the wires and
110
Proc. of SPIE Vol. 5800
Figure 6: The internals of the Eyetop,
showing the small Kopin AMLCD with
fragile connecting wires and cables.
cables are kept stationary.
The Eyetop has a Kopin 320x240 AMLCD microdisplay. Image resolution does not meet the human eye’s 1 arcmin of
resolution, but seems to be sufficient for reading at least 50’x50’ of a floorplan. This display has a brightness of 30 fL, a
focus range of 2m to infinity, a contrast of 100:1, a FOV of 12.8° horizontal by 9.6° vertical, and an operating
temperature range of 0-60 °C28. Although an OLED has a higher operating temperature capability and greater
brightness, its current much higher cost outweighs its benefits for early prototyping and most fire department budgets.
Custom packaging was designed and built for both prototypes from ABS plastic. The packaging is impact and waterresistant. It will help to protect the electronics from heat flashes, water and sweat that would quickly rust the electronics,
and vibrations and impacts that would eventually break the fragile electronics and optics. It is not yet, however, up to
firefighting standards. The packaging is designed to be injection molded, making it inexpensive to mass-produce.
An in-mask microphone and earpiece for radio communication were considered. The firefighters are facile with their
hand held radios, and are very hesitant to change a method that works. An in-mask radio, however, would free their
hands for other tasks. When talking on the radio, a button could be held down on their shoulder strap, and released when
talking aloud to teammates. An earpiece would be more effective than a speaker in overcoming the abundance of noise
present in an emergency response environment. Using one earpiece would be safer than two, allowing the firefighter to
maintain situational awareness with the uncovered ear.
5.3 Direct view HMD
The custom packaging was designed using SolidWorks 2003 and an FDM machine. Improvements have been made in
areas of durability, user adjustability, a level screen, and ease of installing and removing from the mask. The unit is
mounted to the nosepiece of a Draeger mask (Fig. 7).
Figure 7: The direct view SolidWorks model packaging at left, and the complete unit installed in a Draeger mask at right.
All of the electrical components and optics are securely packaged together inside a rigid injection moldable ABS plastic
casing. The casing closes with three screws for higher strength and security than a snap fit. There should never be a
need for the firefighters to disassemble the case. It is shaped to fit the inner contours of the mask, and to not overly
protrude into the user’s face. It is mounted low and aims up at the eye so as to minimize blocking of the outside world.
The unit mounts very close to the face and is light, which minimizes the amount of strain placed on the user’s neck due
to weight that is off-axis from the center of mass of one’s head.
The case remains level as it rotates on a disk mounted to the nosepiece. The disk can be rotated up and down so that
different users can make adjustments to see the entire screen. A wire exits out of a sealed hole in the rubber siding of the
mask. The hole is sealed with a plastic disk similar to the one on the nosepiece. The final prototype may have the wire
connection within the SCBA respirator as in Figure 5, rather than an exit hole. The unit is easily removed from the mask
(for cleaning the mask) by stretching the nosepiece hole around its disk, and the hole on the side of the mask around its
Proc. of SPIE Vol. 5800
111
disk. The wire can then be pulled through the hole on the side of the mask. The only modification made to the mask is
the hole punched in the side of the mask, which if adequately sealed should not compromise the integrity of the mask.
Two problems with this design are interference with mask’s glasses kit and difficulty in aligning the eye to the image.
The optional glasses kit that comes with this mask uses large and thick lenses that contact the HMD. If contact lenses or
thinner lenses are used, this will not be a problem. However, even with thinner lenses, the HMD is mounted so low and
close to the eye that one must look below the FOV of the glasses, unless the glasses are very close to the eye and cannot
be seen around. Contact lenses could potentially solve this problem. The U. S. Army had a similar problem with its
IHADSS monocular HMD for Apache pilots. It approved the use of contact lenses for these pilots29. However, the
National Fire Protection Association (NFPA) has restrictions on their use. Another solution is to shorten the HMD so
that there is more eye clearance. This, however, will require custom optics at a higher cost.
5.4 See-through HMD
Although units like the MicroOptical EG-8 could potentially work
well, the cost target per HMD is around $500. Thus, a less refined
but much less expensive see-through HMD has been designed. The
first design used a simple microscope based system (Fig. 8). The
entire system was designed using the program Optics Software for
Layout and Optimization - Educational Edition (OLSO-EDU).
Parameters of the system are given in Table 3.
Figure 8: A two-lens microscope, prism, and
beamsplitter system designed with OSLO-EDU.
Table 3. Some optical parameters for the first see-through HMD design.
Eye Relief
Exit Pupil
Total Magnification
Diameter
50.4 mm
6.8 mm
3.78
Display
Resolution
1.875 arcmin
FOV
10.7°
The microscope collimates the light rays, giving a virtual image at infinity. To create a see-through design, a prism
reflects the exiting light into a cube beamsplitter. The 50.4 mm exit pupil is long enough for this purpose. Proper image
orientation is achieved by including the prism. If the beamsplitter were used without the prism, the image would appear
backwards to the observer. Alternatively, the image on the LCD could be reversed. The exit pupil is quite small, as with
binoculars, which means the HMD must be adjustable such that each user can align his or her eye with the image. The
HMD must also be stable so that the eyepiece does not shake and vibrate when the user is running, which could cause a
momentary loss of the image.
To create a prototype of this system that will fit in the mask, it is less expensive to purchase and modify a complete
direct view HMD that has much of the necessary optics, electronics, and
software than to develop a fully custom system. A microscope-based design
creates a longer optical train that would have to be folded using mirrors to fit
in the mask. Therefore, an Eyetop HMD was purchased as for the direct view.
This unit uses a non-pupil forming optical system with a simple magnifier.
Upon inspection, the imaging quality appears to be adequate, with good
brightness and contrast, and the eye relief is about 5 cm. By rotating the
display upside-down, removing and replacing the LCD backwards, and
interfacing it with a beamsplitter, a see-through HMD is created for about
$500. This does not include the price of the casing and labor for assembly. A
SolidWorks rendering of the prototype is shown in Figure 9.
Figure 9: The see-through HMD uses
the same mount as the direct view, and
adds a cube beamsplitter for see-through
capability.
A 15 mm 50/50 cube beamsplitter was purchased from Edmund Optics and
mounted to the simple magnifier. Although this adds bulk to the HMD, it
creates a taller and thinner unit that fits better inside the mask. The casing is
positioned farther from the wearer’s face, allowing the glasses kit to fit. The display, however, is purposefully mounted
low in the user’s FOV, enough such that the lower rim of a user’s glasses kit obstructs part of the HMD image. As with
the direct view, contact lenses or thinner glasses would be needed for compatibility.
112
Proc. of SPIE Vol. 5800
A cube beamsplitter was chosen over a plate type because it is stronger and easier to mount. A plate beamsplitter would
be lighter, but would create more assembly difficulties. More importantly, they are very thin and could easily break,
causing a safety hazard. The cube beamsplitter is glass, making for a high quality optical element. Cost and weight
would be reduced with a plastic unit. The effect on image quality would be minimal and largely unnoticed due to the
low resolution of the LCD. The weight savings would place less strain on the user’s neck, and reduce the slight
imbalance of the mask. Therefore, future units will have a plastic cube beamsplitter.
Problems found with the cube beamsplitter are extraneous reflections and a dimmer image. Below the desired image, a
user will see a thin band of inverted images, caused by reflections from below. If the floor area has anything bright, such
as something white reflecting in the light, it may be especially distracting to the user. This has been temporarily solved
by occluding these image band areas. The dimmer image is a result of the 50/50% transmitting/reflecting beamsplitter.
As with the direct view HMD, it is difficult to align the image with the eye. If the eye is not aligned with the small
window in which the image appears, one will not see the entire image. The casing can be rotated and adjusted to align
up-down with many users’ eyes, but it needs more left-right adjustability. There is also concern that jarring or vibrations
could move the HMD enough to misalign the image. Finally, issues of binocular rivalry and attention costs vs.
awareness benefits should be investigated.
5.5 Future work
A large area of future work is the graphical user interface (GUI) design. The HMD will have a GUI that is simple and
effective. It will show firefighters only what they need to know, in a clear and concise manner. There will also be a
control system allowing some manual operations by the user. This control system will ideally be simple and hands-free.
Some GUI work has been done for firefighting, primarily for the IC. The IC currently uses a “grease-board,” a white
board with erasable markers and magnets. It does not have the potential functionality of a GUI with easily accessible
maps. The FIRE group at UC Berkeley has developed a prototype IC GUI, based on the interviews with the Chicago
Fire Department. Implementation of this system assumes tracking ability with a WSN infrastructure or other preinstalled
method, and affordable and rugged flat panel displays or laptops for the IC.
Control methods are ways for the firefighter to manually control what is shown on the GUI. The method should be
simple, fool proof, and ideally hands free. It is currently unclear how much control, if any, a firefighter should have over
the GUI. If no control is given and the system is fully automatic, it will have to be capable of showing the firefighter the
right magnification and section of floor plan for navigation and fire information. Having control over the map view
would mean being able to zoom in and out of and scroll across large floor plans when necessary. If too much control is
given, the HMD becomes a new task, which would reduce the efficiency of their operations and may add stress to an
already high stress environment. The firefighter may also accidentally put the HMD into a mode that is undesired, such
as switching from a map view to a biometric data or text messaging view. Time would be wasted in returning to the
desired mode. Therefore, it is beneficial to have as much automation in the system as possible.
A more compact hardware system is needed for the HMD computer, including electronics for the user control system,
and a node for wireless communication with SmokeNet. It would ideally be the size of a PDA. It must be very rugged,
able to protect inner components from heat, impacts, and water. It will ultimately be made out of steel, or a high impact
and fire retardent thick plastic, with insulation inside. It will be strapped or bolted to the firefighter’s SCBA backpack,
or placed in a protective pocket on their jacket. There will be a cable going from the box to the HMD in the mask. This
cable must be wrapped in a high strength protective sheath that will not allow the wire to break in the event of snagging
or melt in extreme temperatures. Snaggings will be minimized by guiding the cable against the firefighter’s pack straps
and jacket when possible.
6. CONCLUSIONS
The FIRE project at UC Berkeley is developing HMDs and wearable computing technology to increase safety and
efficiency in firefighting and emergency response. The most recent HMDs presented here are closer to fulfilling the
Proc. of SPIE Vol. 5800
113
demanding requirements of firefighters. This project also aims to improve and progress wearable computing, and may
be extended to wearable computing products for consumers.
If a system such as FIRE is implemented, proper training will be vital in creating a safe and effective operation. Further
demonstrations, creation of GUIs, minimized and rugged packaging, and user testing are the next steps in determining
the costs and benefits of this system to firefighters.
REFERENCES
1. U.S. Fire Administration, F.E.M.A., “Facts on Fire,” http://www.usfa.fema.gov/public/factsheets/facts.shtm, 2003.
2. McKinsey and Co. “Increasing FDNY's Preparedness,” http://www.nyc.gov/html/fdny/html/mck_report/toc.html,
2002.
3. Nowakowski, R., Private discussion, Manager of R & D, Chicago Office of Emergency Communications, 2003.
4. Green, C., Private discussion, Assistant Fire Chief, Berkeley Fire Department, 2004.
5. National Institute for Occupational Safety and Health (NIOSH), ”Death in the Line of Duty: Six Career Fire Fighters
Killed in Cold-Storage and Warehouse Building Fire – Massachusetts,” http://www.cdc.gov/niosh/face9947.html, 2000.
6. Steingart, D, Wilson, J., Redfern, A., and Wright, P., “Fire Information and Rescue Equipment: An IT Based
Coordination for Firefighting,” To appear in AugCog International, Vol. 1, 2005.
7. The Saab Group, “Head Mounted Displays, HMD,” http://www.saab.se/node5507.asp, 2004.
8. GeoIT, “LISA,” http://www.geoit.ethz.ch/research/lisa_en.html, 2004.
9. Jiang, X., Hong, J., Takayama, L., & Landay, J., "Ubiquitous Computing for Firefighters: Field Studies and
Prototypes of Large Displays for Incident Command," Proc. of CHI 2004, April 24-29, 2004.
10. Jiang, X., Chen N. Y., Hong, J. I., Wang, K., Takayama, L., & Landay, J. A., "Siren: Context-aware Computing for
Firefighting," Proceedings of Pervasive 2004, April 18-23, 2004.
11. Baber, C., Haniff, D.J., and Woolley, S.I., “Contrasting paradigms for the development of wearable computers,”
IBM Systems Journal 38 (4), 1999.
12. Baber, C., Haniff, D., Buckley, R., “Wearable Information Appliances for the Emergency Services: HotHelmet,”
Handheld and Ubiquitous Computing: First International Symposium, H.-W. Gellersen (Ed.), Karlsruhe, LNCS 1707,
pp. 314-316, Germany, September 1999.
13. VuMan project, http://www-2.cs.cmu.edu/~wearable/vuman.html, 2004.
14. Tri-Sentinel, http://www.trisentinel.com/, 2004.
15. Ceperley, D. et al., “LifeLine: Improved Communication and Informatics for Fire and Rescue Workers,” Computer
Society International Design Competition, 2002.
16. HEMIND project, http://www.hemind-project.org/project_summary.html, 2004.
17. ERT Program, http://www.nttc.edu/ertprogram//technologies/10_needs.asp, 2004.
18. Spitzer, M., Zavracky, T., Rensing, N., Hockman, P., Aquilino, R., McClelland, W., Zardeskas, J., “Eyewear-based
Displays for Personal Electronics,” Helmet and Head-Mounted Displays V, Proc. of SPIE Vol. 4361, pp. 27-33, 2000.
19. MicroOptical Inc., www.microopticalcorp.com, 2004.
20. KEO, Inc., www.keo.com, 2004.
21. eMagin Inc., www.emagin.com, 2004.
22. Hromadka, T. and Melzer, J., “Results of using helmet-mounted displays to control robots in Afghanistan,” Helmetand Head-Mounted Displays VIII, Proc. of SPIE Vol. 5079, pp. 222-231, 2003.
23. Microvision Inc., www.mvis.com, 2004.
24. Interactive Imaging Systems, Inc., www.iisvr.com, 2004.
25. Ingineo, www.eyetop.net, 2004.
26. Liteye, Inc., www.liteye.com, 2004.
27. BMW World, http://www.bmwworld.com/racing/f1/head-up_display.htm, 2003.
28. Kopin, Inc., www.kopin.com, 2004.
29. Rash, Clarence E, Helmet-Mounted Displays: Design Issues for Rotary-Wing Aircraft, SPIE, Bellingham, WA,
1999.
114
Proc. of SPIE Vol. 5800