User Experience and Acceptance of a Mixed Reality System in a
Naturalistic Setting – A Case Study
Susanna Nilsson ∗
University of Linköping
SE-581 83, Linköping, Sweden
ABSTRACT
This poster presents a qualitative user study investigating user
experience and acceptance of an MR application designed to give
instructions on how to start up a diathermy apparatus. The study
was conducted in a naturalistic setting on site at a hospital with
actual users and their equipment. The analysis of the results has
indicated that although there are several ergonomic issues to be
solved, the acceptance of an MR system in this user group is high.
As a result of the study, the MR system has been redesigned to
better fit the ergonomic needs of this user group.
1.1
Background and related work
The field of Augmented Reality (AR) and Mixed Reality (MR)
is a relatively new field in terms of commercially and publicly
available applications. As a research field it has existed for over a
decade with applications in diverse domains, such as medicine,
military applications, entertainment, technical support and
industry applications. One application area for MR systems is
instructional tools; Tang [7] describes an experimental evaluation
of AR used in object assembly; Zauner et al [9] describe how MR
can be used as an assembly instructor for furniture applications.
MR or AR can also be used to give instructions on how to operate
new or unfamiliar equipment. As an example, Feiner et al describe
a testbed for knowledge-based generation on maintenance and
repair instructions in AR [2].
Usability methods used for MR systems are mainly based on
usability methods used for graphical user interfaces in
combination with usability for Virtual Reality applications [8].
However there are some complications with this approach since it
is not based on the experiences of actual MR systems users and
there are few examples of studies on how users actually perceive
the system. There are no widespread examples of MR specific
usability criteria and maybe there is no need for such criteria – the
usability should instead be based on the task and goals of the user
in the actual context of use, and not based on the technicalities of
the system.
To address usability aspects of a helmet mounted video see
through MR system, a user study at a university hospital in a
midsized Swedish town has been conducted. The main objective
of the study was to find out whether the MR system would be
accepted as part of the technological support in a user group
working at a hospital.
2
Björn Johansson †
University of Linköping
SE-581 83, Linköping, Sweden
four did not. First the participants were interviewed about their
experience and attitudes towards new technology and instructions
for use. Then they were observed using the MR system, receiving
instructions on how to start up the diathermy apparatus. After the
task was completed they were again interviewed about the
experience. The interviews, as well as the observation were
recorded and the participants’ view through the video-see-through
MR system was also logged.
2.1
Equipment
The MR system in the study (see figure 1) includes a computer
(Fujitsu Siemens Tablet™), a helmet mounted display (Sony
Glasstron™) with a fire wire camera (Point Grey Research,
Dragonfly™).
Figure 1: The helmet-mounted MR system used in the study
A numpad (Targus™) was used for the interaction with the
user. The MR system uses a hybrid tracking technology based on
marker tracking; ARToolKit, [6], ARToolKit Plus [7] and ARTag
[3]. The software includes an integrated set of software tools such
as software for camera image capture, fiducial marker detection,
computer graphics software and also software developed
specifically for MR-application scenarios [4].
METHOD
The study was conducted on site at a university hospital and
involved eight participants, all employed at the hospital. Four of
them had previous experience with the diathermy apparatus, and
∗ susni@ida.liu.se
† bjojo@ida.liu.se
Figure 2: Participant receiving instructions from the
MR system, and interacting with the DA.
The users were given instructions on how to activate a surgical
diathermy apparatus, ERBE ICC-350 with associated instruments
and electrodes. In general, diathermy is a physical therapy for
deep heating of tissues with high frequency electrical current. The
ERBE ICC 350™ diathermy apparatus is used for mono- or
bipolar cutting and coagulating during invasive medical
procedures (it can be seen in figure 2 above). The instructions
were divided into two main steps; first status verification of the
patient, secondly start up of the DA.
The instructions have been rewritten and the clutteredness of the
display has been reduced by a change in the program. The
questions in the first step of the instructions now have to be
completely answered before the information in the next step is
shown. Other alterations to the visual display include redesign of
the animations, and removal of marker status indicating hands.
4
3
RESULTS AND CHANGES OF THE SYSTEM
It was found that all users but one could solve the task at hand
when aided by the instructions given in the MR system. The
interviewed responded that they usually prefer personal
instructions from an experienced user, sometimes in combination
with short, written instructions, but also that they appreciated the
objective instructions given by the MR system. The problems
users reported on related both to the instructions given by the MR
system and to the MR technology, such as problems with an
unstable image etc. Despite the reported problems, the users were
positive towards MR systems as a technology and as a tool for
instructions in this setting. A new hand-held MR system and a
new head mounted MR system have been developed based on the
comments from this study.
3.1
New prototypes
The study has resulted in a redesign of the head mounted MR
system as well as of a handheld MR system (see figure 3). The
head mounted MR system shown to the left in figure 3 below is
attached with an adjustable strap around the head instead of a
helmet (as its original design by Sony, but with a number of MR
specific components added). This will eliminate some of the
ergonomic problems that occur when using the MR system
indoors.
DISCUSSION OF THE RESULTS
Overall the study showed that the actual users from the medical
domain were positive towards MR systems as a technology and as
a tool for instructions. To combine interviews with video has been
valuable in the sense that it allows cross-checking of statements
from the participants as well as identification of obvious, but not
mentioned, usability problems. Interactivity is an important part of
direct manipulating user interfaces and also seems to be of
importance in an MR system of the kind investigated in this study.
Instructions on how to use new technological equipment, given by
a personal teacher is probably hard to replace. However, this study
has shown that MR can be used as a complement to personal
instructions, and quite possibly with improvements maybe even as
a replacement when needed.
Performing usability studies in a real work setting, with actual
users has also proven its value in this study. By confronting a
design for a new technology with tasks from real work rather than
laboratory settings allows problems that probably otherwise
would remain unknown to emerge. A product like the DA
represents an actual tool used for a high-risk task by skilled
professionals and is clearly a case where improved functionality
in terms of instructions is beneficial.
ACKNOWLEDGMENTS
This study is part of a collaboration project between
Department of Computer and Information Science at the
University of Linköping (IDA), and the Swedish Defence
Research Agency (FOI), funded by the Swedish Defence
Materials Administration (FMV). The MR system and associated
software has been developed by FOI.
REFERERNCES
[1]
[2]
[3]
[4]
Figure 3: To the left a head mounted MR system and to the
right a handheld MR system with easy access interaction
Another way of avoiding the problems associated with the
clumsiness of the helmet mounted system is to use a handheld MR
system, which does not need any adjustments like the headstrap.
A handheld system can freely be moved in relation to both the
user and the DA. The handheld MR system has some obvious
similarities with other handheld systems such as the AR system
used in MagicBook [1]. However, the proposed redesigned
handheld MR system has two interaction buttons on the handle
instead of one, which allows simple interaction to answer “yes”
and “no” questions as well as to proceed through the instructions.
The placement of two buttons next to each other with distinct
functionalities (a ‘yes-button’ and a ‘no-button’) may simplify the
interaction and will be tested in further user studies.
The problems with the instructions have resulted in a redesign
of the information presented in the head mounted display as well.
[5]
[6]
[7]
[8]
[9]
Billinghurst, M., Kato, H. and Poupyrev, I. The MagicBook: Moving
Seamlessly between Reality and Virtuality. IEEE Computer
Graphics and applications Volume 21, Issue 3, 2001
Feiner, S., MacIntyre, B. and Seligmann, D. Knowledge-based
Augmented Reality. Communications of the ACM vol 36, No 7, 1993
Fiala, M. ARTAG Rev2 Fiducial Marker System: Vision based
Tracking for AR. Workshop of Industrial Augmented Reality,
Wienna Austria Oct 4, 2005
Gustafsson, T., Carleberg, P., Svensson, P., Nilsson, S., and
Sivertun, Å. Mixed Reality Systems for Technical Maintenance and
Gaze-controlled interaction. FOI-R.1695-SE (Technical report ISSN
1650-1942) 2005
HITLAB, http://www.hitl.washington.edu/artoolkit/ (2006/04/27)
Schmalstieg, D. Rapid Prototyping of Augmented Reality
Applications with the STUDIERSTUBE Framework, Workshop of
Industrial Augmented Reality (IAR) Wienna, Austria Oct 4, 2005
Tang, A Comparative Effectiveness of Augmented Reality in Object
Assembly. Proc of the SIGHI conference on Human Factors in
Computing Systems April 5-10, Ft Lauderdale, Florida, USA 2003
Träskbäck, M. User Requirements and usability of Mixed Reality
applications. Licentiate Thesis, Helsinki University of Technology
2004
Zauner, J., Haller, M. and Brandl, A. Authoring of a Mixed Reality
Assembly Instructor for Hierarchical Structures. Proc of IEEE ACM
ISMAR 7-10 Oct, Tokyo, Japan 2003