A Comparison of Map vs. Text Directions for a Handheld Device in
a Campus Setting: A Pilot Study
Liz Atwater
Department of Psychology
George Mason University
Fairfax, VA 22030 USA
lizatwater@hotmail.com
Jason Burke
Institute for Systems Research
University of Maryland
College Park, MD 20742 USA
burkej@isr.umd.edu
Andrea Kirk
Department of Computer Science
University of Maryland
College Park, MD 20742 USA
kirkam@wam.umd.edu
December, 2001
ABSTRACT
This project explores some of the options available
for providing point-to-point directions. We are
testing part of the interface for a campus-wide system
to determine if there is any difference between mapbased directions and text-based directions. We are
also trying to determine if complexity of the route
(complexity defined as number of turns) affects users'
performance time and user satisfaction between the
maps and text. The results showed no statistical
difference between map or text directions, but that
there are some differences among the different
complexity levels. While the statistical results of this
experiment are inconclusive, this pilot study is
valuable for the many observations revealed through
the experiment that could prove useful in further
studies.
INTRODUCTION
Rover is a location-aware system currently under
development at the MIND Lab at the University of
Maryland. Its goal is to provide information relative
to user profile, device profile, location, and context
on handheld devices (here defined as small, portable
devices such as the Palm Pilot, Handspring Visor,
Hewlett-Packard Jornada, or Compaq i-Paq). This
kind of technology could be used for tourism and
commerce, among other applications. For example,
if a Rover system were implemented in a museum,
individual devices could be checked out as “pocket
tour guides” to provide information about exhibits,
maps of the museum, etc. When users approach an
exhibit, the device would display relevant
information, perhaps with audio output as well. It
could offer relevant information on similar exhibits,
based on previous topics in which the users have
shown interest. If the users want to visit one of the
suggested exhibits, Rover could provide a map to the
desired area. A system like this is implemented at the
Experience Music Project in Seattle, WA
(http://www.emplive.com). A handheld device called
the MEG (Museum Exhibit Guide) provides tour
information to visitors. Similarly, a Rover system
implemented in a large shopping mall could direct
users to specific stores, based on their input interests
or on their previous shopping behavior.
A key component of this kind of system is location
awareness, provided by a global positioning system
(GPS, see http://www.trimble.com/gps/), infrared,
radio frequency, or other means. Location awareness
could help users navigate to both specific, static
destinations, (for example, the library) and mobile
targets (your friend, who also has a Rover), through
both direct input of destination (“Show me a map
from here to 1143 Maple Road”) and destination
search (“Where is the cafeteria?”). Given such a
powerful and useful functionality, one of the first
questions from a user interface perspective is: how do
designers provide this navigation information to
users, through maps or through text?
To answer this question, we have created a scenario
for the use of a Rover system that spans the
University of Maryland campus, incorporating
buildings, roads, parking lots and sidewalks. The
users are set with the task of finding a friend who is
similarly equipped with a Rover unit, creating a
mobile target-destination search.
One of the key points of this scenario is that it deals
with pedestrian navigation. The vast majority of
point-to-point navigation services are meant for
motor vehicle users; they reference street names and
exit numbers. A pedestrian on the university campus
has few roads to be concerned with; most of the
traveling is done along sidewalks, or even along
“shortcuts” through open areas.
Under these
conditions, we are interested in determining which
method is more effective: maps or text.
Maps provide the navigator with a visual overview of
the route they are taking, and provide landmarks and
other references to help make navigation simpler. On
the other hand, text directions, especially those
provided by a shortest-route algorithm that a
navigation system would use, provide a high degree
of specificity. However, in an area that lacks street
names and large landmarks, creating clear point-topoint text directions becomes a challenge.
this experiment, the campus map-based directions are
very similar to the Mapquest format.
The advent of cheap GPS systems that support
Pocket PC devices have given rise to several
location-aware software packages.
One of the
simplest of these software packages is Microsoft
Pocket Streets. Pocket Streets merely indicates
where the user is in relation to the rest of the map.
The location indicator used in this experiment is
designed to resemble the green circle used in Pocket
Streets.
RELATED WORK
Analysis of Commercial Systems
Pharos’ Pocket PC Navigator (formerly known as
Ostia) was one of the first GPS systems for the
Pocket PC. It highlights the recommended route and
can offer voice directions. A third, and very popular,
car navigational system in use today is the Pocket
Copilot. Copilot gives combined map/text directions
and also supports voice instructions. Screenshots
from these three GPS systems are shown in Figure 2.
While outdoor walking-level direction generators are
still mostly in the research stage, there are several
driving direction generators that are commercially
available today. The most popular web-based
Background Research
A majority of the research on navigational displays
has been conducted in the aviation industry. Two
studies involve how to best display the closest airport
to a pilot in a possible emergency situation
(Williams, 1999 & Williams, 1999). In the first
experiment, Williams (1999) hypothesized that a
map-based presentation of the airport location would
be superior to the then current text-based presentation
because a map displays orientation information
directly. He believed that the time taken to mentally
compute an orientation from text causes delays that
could be dangerous in an emergency situation when
quick decisions are needed from the pilot. However,
Figure 1. Mapquest turn-based driving directions.
location-to-location map generator in use today is
Mapquest. Mapquest shades a recommended route
on a map and can give turn-by-turn directions as
shown in Figure 1. As the location awareness
services of Rover were not available at the time of
Figure 2. GPS-enabled driving direction systems.
2
it could be argued that text-based presentations are
superior because a map display can be more cluttered
than a text display and pilots have much experience
converting text orientation directions into a compass
direction.
Changing the display could cause
confusion to the pilot for these reasons. To test his
hypothesis, Williams (1999) designed an experiment
with three displays and used both pilots and nonpilots as his subjects. The three displays were text,
enhanced text (text with an orientation symbol), and
map. The map display had two treatments, a track-up
orientation (heading always at the top of the display)
and a north-up orientation (north fixed at the top of
the display). Williams hypothesized that the track-up
map would result in better performance because there
would not be a need to perform a mental rotation.
The subjects were asked to determine the relative
direction of the nearest airport using a flight
simulator and a GPS display. The two main
performance measures were orientation response time
and accuracy. Results showed that the subjects were
significantly slower and made three times more errors
when using the text-based display than with the mapbased display. Subjects were also more likely to
make errors with the north-up display than with the
track-up display.
in which type of direction, map or text, allow for the
fastest performance time and the least amount of
errors for navigating from one point to another.
Because the results in both of the Williams studies
show better performance with a map display, we
hypothesize that our subjects will navigate the routes
faster with the map directions than with the text
directions.
North-up vs. Track-up
An important design consideration for the Rover
device is the frame of reference to use for map-based
directions. In the area of flight navigation, Aretz
(1991) proposes two types of references that can be
used in navigation: the ego-centered reference frame
(ERF) and the world-centered reference frame
(WRF). For directions, ERF means that the locations
are described as relative to the user (left, right, etc.),
and WRF uses the user’s environment as the
reference frame (north, east, etc.).
Aretz (1991) finds that a track-up display is best for
representing the ERF tasks such as localization. The
study also shows that a north-up display is best for
representing tasks that require a WRF such as
reconnaissance.
In a related study that examined many of the same
variables, Williams (1999) tested subjects on
determining which of two airports was furthest from
a storm front. There were questions from the first
study that he hoped to answer in this experiment.
First, the previous experiment included a task that
was egocentric (readings relative to the user’s
location). Williams changed the task in the second
experiment to be a world-referenced one (readings
relative to the user’s environment); this could
possibly eliminate the advantage of a map display
because there would be no need to integrate bearing
and heading information. The second question was
the type of subjects used. In the first experiment, the
non-pilots were significantly slower than the pilots
and this may have confounded the results. In the
second study, only pilots were used. The same three
types of displays were used, as well as the two
treatments within the map display. The performance
measures were decision time and accuracy. Results
showed that significantly more decision errors were
made with the track-up condition.
This is a
contrasting result to the previous study. Overall,
pilots were faster making a decision with a map
display than with either of the other two displays, and
they were faster with the north-up orientation.
For Rover, a track-up display would rotate the map as
the user turns so that the user’s forward direction
corresponds to the top of the device’s display. The
north-up alternative would maintain a common
direction for the map display. It can be hypothesized
that a location-aware Rover device would work best
with a track-up display since finding a location is
primarily an ERF activity. Naturally, the text
directions are based upon an ERF. However, for this
experiment a common, north-constant direction is
maintained for all of the map displays. This decision
to use a WRF for the map display was made because
the Rover device does not yet support location
awareness, and it was felt that pseudo-track-up
directions could disorient and confuse the users and
lead to longer device-consultation times.
Route Modeling Theory
The routes in this experiment very nearly follow the
network model defined by Butz (2001). Nodes of the
graph represent real-world turns or intersection
locations.
Edges of the graph represent the
connections between nodes. Thus, the map and text
routes in this experiment are broken into the same
node-to-node sequential instructions.
Both of these experiments involve making a decision
from a display. In our experiment, we are interested
3
Butz (2001) explains that successful directions must
integrate environmental knowledge (such as
landmarks). “Landmarks at decision points are
needed to communicate a reorientation and/or path
progression, and are located at turns along the path”
(Butz, 2001). This reasoning supports our hypothesis
that subjects will perform faster and more effectively
with the landmark-rich map directions than with the
landmark-poor text directions.
ASSDLT =Average Standing Still Direction Loading
Time (on the device)
Time to Complete Task = TTW + TTSS
Where
TTW = TWCP + TWIP
TTSS = [(ASSOT)  (ND)] + [(ASSDLT)  (ND)]
In comparing the map and text implementations the
following arguments are made regarding above
equation:
EXPERIMENT
The test was a 2 x 3 design that compared map and
text directions for three different routes of low,
medium and high complexity, with complexity
defined as the number of key decision points.
Accordingly, the independent variables were
direction format and route complexity.
The
dependent variables measured were total task
completion time, consultation time (time spent not
walking; this could include consulting with the
device or the test administrators, or time spent
checking orientation against nearby landmarks), and
number of errors (incidents when the user had to ask
for help or strayed from the route for more than three
seconds). Each user navigated through all three
routes with either the map or text for all three,
making this design between subjects for format and
within subjects for complexity.
Format
Text
Map
Route Permutations
123
132
213
231
123
132
213
231
312
312
321
321

TTW will be much lower in the map
implementation because the greater number of
landmark aids will result in fewer errors
(lower TWIP) and more decisive progression
(lower TWCP).

TTSS will be lower in the text implementation
because of the ego-centered reference frame
(lower ASSOT).

ASSDLT will be longer for the larger files
needed in the map-based directions. However,
the time gains are marginal and will not weight
heavily in the overall time for task completion.

Increasing ND with the more complex routes
will result in higher TTSS and higher
probability for errors (TWIP).
These observations allow the authors to make the
following hypotheses:
Figure 3. Experimental design; each cell denotes a
trial that was run.
H0: (Null hypothesis) There is no statistical
difference between completion time, consultation
time, and number of errors between text and map
directions, regardless of complexity.
H1: Users will complete the tasks faster using map
directions.
H2: Users will make fewer errors using map
directions.
H3: Users will need less consultation time using text
directions.
H4: Completion time will rise with increasing route
complexity.
Predictive Model and Hypotheses
The predictive model must account for the fact that
some task activities can occur in parallel with others.
Our model tries to isolate the various factors that can
affect the task outcome. The amount of time that it
will take a subject to find a location using the handheld device can be modeled as follows:
TTW = Total Time Walking
TWCP = Time Walking Correct Path
TWIP = Time Walking Incorrect Path
Control for Bias
To eliminate any bias that could result from the
ordering of the routes, tests were run for each
permutation of the routes (see Figure 3). With trials
for text and map versions of all six permutations, a
total of twelve tests were performed. Furthermore,
TTSS = Total Time Standing Still
ND = Number of Directions
ASSOT = Average Standing Still Orientation Time
4
all three routes were approximately the same length
to eliminate large completion time differences
between routes.
to-point fashion, where the points were key decision
points, such as road crossings or sidewalk junctures
where a turn must be made. These critical points
were the same in both the map and the text versions
to ensure that both groups received the same
information. The map directions (see Figure 4)
feature a green circle to show the users’ current
location (ideally; for our implementation, the green
circle is the start location for the current step), a
highlighted line to show the path to follow, and a
darkened highlighted line to show the path already
traveled. The text directions (see Figure 5) show the
current direction in regular black text, and previous
directions grayed out. Each screen has one point-topoint step and a “Next” button for the users to hit to
obtain the next direction, as well as the distance
remaining to the goal, from the beginning of the
current step, to help users estimate how much further
they have to travel.
The routes were designed in such a way that subjects’
familiarity with campus would not allow them to
ignore the directions from the device and find the
destination by themselves. The endpoints were
placed at clear, logical meeting places, but at the
same time, these places would not be considered
major landmarks. Because these points were not
clearly identifiable from the directions, users had to
continually rely on the directions to complete the
tasks. The routes and directions are described in
more detail below.
Participants
Users were University of Maryland undergraduate
and graduate students from various fields of study
(see Appendix B). We chose twelve subjects to
control for route ordering bias, as described above.
Out of the twelve users, seven were male and five
female. They were distributed as evenly as possible
over the map and text conditions, to eliminate any
innate gender preferences for one format over the
other.
Pilot Tests and Results
Two pilot tests were run before the actual
experiment: one map trial and one text. These helped
the experimenters to refine the testing materials and
procedure. Changes made based on the pilot tests
were mostly minor: wording of certain questions in
the pre-task questionnaire, wording in the instructions
to the user, darkening of the map colors to improve
outdoor visibility, and clarification of some of the
text directions. It was also discovered that the device
that had been used in the pilot, a Hewlett-Packard
Jornada, did not provide good daylight visibility;
subsequently a Compaq i-Paq was acquired, which
had much better visibility outdoors.
Figure 4. Map Directions
Since the Rover infrastructure is not yet in place, the
Windows CE version of Microsoft Internet Explorer
was used to implement the interface, using HTML
files to display both the maps (.jpg images) and the
text. Instead of having a location-aware device, the
users were presented with the point-to-point
directions, and the testing team pointed out any
navigation errors as a substitute for a real locationaware system. In a true implementation of Rover, the
“Next” buttons would be unnecessary, since the
screen would update itself as the users walked along
their routes, showing the path and current position,
and making corrections or rerouting as needed.
Routes
The three routes were designed to be as natural as
possible, providing logical paths from a given point
to a destination. The most complex route, with seven
decision points, was 883 feet long; the mediumcomplexity route (five decision points) was 897 feet;
and the least complex route (three decision points)
was 893 feet.
The directions (which can be found in their entirety
in Appendices A1 and A2) were designed in a point-
5
preliminary orientation was finished, the users were
handed the device and the total completion time
clock was started.
Another timer, used for
consultation time, was only run when the users
stopped walking.
The testers accompanied the users during the test,
observing consultations and errors, including selfcorrected errors, and recording any comments from
the users. Once the destination was reached, the
times for that route were recorded, and the testers
escorted the users to the next start point.
After the routes were completed, users were asked to
fill out brief post-task questionnaires. These were
subjective satisfaction questions, using Likert scales
from 1 to 7 (see Appendix A6). The users rated the
difficulty of using the interface, based on the
beginning, middle, and end of their tasks, and the
general comprehensibility of the directions.
Figure 5. Text Directions
Expert User Results
An explanatory key accompanied both the text and
map directions. To ensure that the users would
understand the conventions used in the maps and text,
such as symbols and terminology, the keys were both
shown and verbally explained before each test.
In order to provide an understanding for the results
that might come from the most expert user, the
development team members undertook their own
experiment. Using the notation from the predictive
model, perfect users have TWIP and TTSS equal to
zero because they commit zero errors and can use the
device while walking at full speed. Therefore, the
completion time is reduced to the time walking the
correct path and it is the same for both map and text
directions.
The fastest times achieved by the
development team are 150 seconds for low
complexity, 152 seconds for medium complexity, and
146 seconds for high complexity. Note that these
times reflect only the slight differences in route
length (low = 893 ft., medium = 897 ft., and high =
883 ft.).
Procedure
Before beginning the test, users were read a short
explanation of the experiment (see Appendix A3),
asked to sign an experimental consent form (see
Appendix C), and given two pre-task questionnaires.
The first of these was a short questionnaire, asking
about the users’ major, knowledge of the campus,
usual means of transportation on campus, familiarity
with handheld devices and eyesight (see Appendix
A4). The other was the VZ-2 spatial ability,
available from the Educational Testing Service
(http://www.ets.org). Each question shows a series
of images, indicating how a square piece of paper is
folded and perforated (see Appendix A5). Below this
sequence, five unfolded pieces of paper are shown
with different arrangements of holes in each one.
The test takers are told to choose the image that
represents the final result of the perforated paper after
it is unfolded. Participants were asked to answer ten
of these questions within three minutes.
Problems
There were no procedural problems in the
experimental trials, but the major difficulty
throughout the whole study was the design of the
routes and directions.
As mentioned in the
introduction, most navigation systems are designed
for vehicles, not pedestrians. While driving, the
major landmarks are clearly named locations such as
streets and exits. In the case of pedestrians, however,
sidewalks do not have names and other landmarks,
such as buildings, may not be identifiable from the
pedestrians’ perspective. Without landmarks and
identifiers, the directions, especially the text
directions, were difficult to create.
The key, text or map, was explained to the user
thoroughly, and any questions were answered before
starting the actual experiment. Each of the route
destinations was marked with a small orange traffic
cone, one of which was shown to the users as a
sample of what to look for. After all of this
6
Completion Time
RESULTS
400
350
All data from this experiment are compiled in
Appendix B.
Time (s)
300
Participant Pre-Task Results
250
200
Map
150
Text
100
Information gathered from the pre-task questionnaire
and the VZ-2 spatial ability questionnaire was
analyzed with t-test statistics to determine if there
were any significant differences between the map and
text groups on several extraneous variables. The full
list of results can be seen in Figure 6 below.
50
0
High
Medium
Low
Route complexity
Figure 7. Completion Time Statistics
Variable
# of semesters at UMCP
Familiarity with campus
Walking speed
Yes to handheld use
Yes to corrected vision
Spatial ability
t value
1.4
1.1
1.1
0.5
0.0
1.1
Significance
0.2
0.3
0.3
0.6
1.0
0.3
Dependent Variable: Device Consulting Time
We defined device consulting time as the amount of
time a user was not walking and looking at the
directions for purposes of location orientation and
determining what the next step was. A two-way
ANOVA was conducted to determine if there were
any significant differences between the (1) map and
text groups, (2) different routes, and (3) to see if there
was a direction x route interaction. Results similar to
those in the previous section were found for this
dependent variable. The results are illustrated in the
graph below.
Figure 6. Pre-Task Statistics
Note: Significance levels need to be equal to or less
than 0.05 for there to be considered a significant
difference between two groups.
Dependent Variable: Completion time
There was a significant main effect for route (F
(2,30) = 6.0, p < .05). Scheffe’s post hoc analysis
shows that there was a significant difference between
the high complexity route and both the medium and
low complexity routes (mean difference = 31.0, 31.5,
respectively).
A two-way ANOVA was conducted to determine if
there was a significant difference between the
completion times for (1) the map and text groups, (2)
the different routes, (3) and to see if there was a
direction x route interaction effect. Results show that
there was a significant main effect for route (F (2,30)
= 5.8, p < .05). Scheffe’s post hoc analysis shows
that completion times for the high complexity route
were significantly different than both the medium and
low complexity routes (mean difference = 51.0, 58.3,
respectively).
Device Consulting Time
120
100
Time (s)
80
There was not a significant main effect for direction
type (F (1,30) = 0.7, p > .05). There was also not a
significant direction x route interaction (F (2, 30) =
0.4, p > .05). These results are illustrated in Figure 7.
60
Map
40
Text
20
0
-20
High
Medium
Low
Route Complexity
Figure 8. Device Consulting Time Statistics
There was not a significant main effect for direction
type (F (1,30) = 0.3, p > .05) nor was there a
significant direction x route interaction (F (2,30) =
0.1, p > 0.5).
7
number of semesters at UMCP, familiarity with the
campus, handheld device use, walking speed, having
corrected vision, and spatial abilities. From these
results we can be confident that the two groups were
equivalent with respect to variables that may have
had an effect on the outcomes of the experiment.
This lends support to the internal validity of our
experiment; there was sufficient control of
extraneous variables.
Dependent Variable: Number of Errors
An error was defined as when a user strayed off
course for more than 3 sec and needed to be corrected
and when the user asked us a question that they were
unable to answer for themselves.
A two-way
ANOVA was conducted to determine if there were
any significant differences between the (1) map and
text groups, (2) different routes, and (3) to see if there
was a direction x route interaction. Results similar to
those in the previous section were found for this
dependent variable. The results are illustrated in the
graph below.
The completion time statistics show that participants
statistically took longer to complete the high
complexity route than either the medium or low
complexity route. This partially confirms hypothesis
H4 (higher complexity increases completion time)
and therefore refutes the null hypothesis H0.
Number of Errors
3.5
Number of errors
3
The completion time statistics also show that there
was not a significant difference in the completion
times between the map and text group. This refutes
our hypothesis H1 that users would perform more
quickly with map directions compared to text.
2.5
2
1.5
Map
1
Text
0.5
0
-0.5
High
Medium
Low
Device consultation time statistical analysis shows
that users spent more time stopping and looking at
the device during the complex route than either the
medium or the low complexity route. It also
demonstrates that there was not a significant
difference in the amount of time spent solely looking
at the device between the map and text groups.
Further, this refutes hypothesis H3 (less consultation
time for map directions) because the map and text
groups spent similar amounts of time looking at the
device on the three different routes.
Route Complexity
Figure 8. Number of Errors Statistics
There was a significant main effect for route (F
(2,30) = 9.6, p < .05). Scheffe’s post hoc analysis
shows that there was a significant difference between
the high complexity route and both the medium and
low complexity routes (mean difference = 1.2, 1.2,
respectively).
There was not a significant main effect for direction
type (F (1,30) = 1.6, p > .05) nor was there a
significant direction x route interaction (F (2,30) =
1.2, p > 0.5).
The number of errors statistics shows that users
committed more errors during the complex route than
either the medium or the low complexity route.
These statistics also show that there was no
significant difference in the amount of errors between
the map and text groups. This refutes hypothesis H2
(fewer errors in map directions) because the map and
text groups committed similar numbers of errors on
the three different routes.
Post-Task Questionnaire
Users were asked to complete a brief questionnaire
after completing the three routes. The complete
questionnaire can be seen in Appendix A6. There
was no significant difference in the responses for
either the map and text groups (t-test, p > .05) or
between the two genders (t-test, p > .05).
It is important to note that this experiment is a pilot
study; we recommend that future studies in this area
use more subjects to produce statistically significant
results. With the small number of subjects used in
this study, the wide differences between individuals
may have disrupted any potential significant
differences, besides the main effects for route
complexity.
DISCUSSION
Interpretation of Results
The participant pre-task statistics showed that the
map and text subject groups were similar in respect to
The statistical analysis of the post-task questionnaire
further confirms our findings that there were not
8
many differences between the map and the text
directions. It is also interesting to note that males and
females had similar responses to our questions.
4.
Some users looked ahead to the next directions
too early and became confused.
Subject Comments
CONCLUSIONS
Some subjects made useful comments after the
experiment. These include:
This experiment probed some concepts for using a
hand-held device to navigate a campus environment.
Specifically, two implementations (map directions
and text directions) were evaluated against each
other. The statistical results for this comparison were
inconclusive. However, this pilot study serves the
purpose of spearheading further research into this
topic by providing insight to the problem and setting
up a model of experimentation.
“I was confused about meaning of ‘intersection’ for
sidewalks.”
“The map was easiest to read when major landmarks
were on the display.”
“Sometimes I had to turn the device around so that I
was traveling in the same direction as the map.”
Suggestions for Future Researchers
“The text terminology needs to be improved.”
The researchers made note of ideas for future
evaluation and lessons to be learned from. We feel
that incorporating location-aware capabilities into
this system should improve usability dramatically
through continuous feedback. Also, a track-up
display for map directions should help to decrease the
users’ orientation time. Full-fledged experiments in
the future should include a map-text hybrid solution
to account for differences in how users perform with
various forms of directions.
Also, any future
experiments would need to use a much larger number
of subjects in order to account for the high degree of
variance that is innate to user cognitive strengths and
weaknesses with location finding directions. Finally,
it is surmised that device learning had a large impact
upon these results, and future researchers should
account for this in their own experiments.
General Observations
It is believed that the subjects’ learning of the device
played a large role in the experiment. This was
especially inherent to the text version as the users
became familiar with the use of the terminology.
Many of the users exhibited their worst results in
their first route (regardless of the complexity or
implementation). It is felt that this effect had a
highly adverse impact upon the ability to get
statistically significant results.
Many of the errors (in both implementations)
occurred at the same locations. Notably, the starting
points to steps two and five in the high complexity
route and the beginning of the third step in the
medium complexity route seemed to generate a large
proportion of the errors. These are all points where
the users’ current sidewalk does not have a clear
dead-end or divide (see Appendices A1 and A2).
ACKNOWLEDGEMENTS
We would like to thank Dr. Ben Shneiderman and Dr.
Evan Golub for the guidance and insight that they
provided throughout the course of developing this
experiment. Special appreciation is due to Evan for
allowing us to borrow the hand-held devices that we
used in developing and implementing the experiment.
Also, we thank fellow students Ugur Kuter, Pankaj
Thakkar, and Cemal Yilmaz for reviewing a draft of
our work.
In general, there was a vast amount of variance in
how people used the device. Some observations are
listed below:
1.
Some people maneuvered the device around
while using the map in order to get track-up
bearings.
2.
While some users (in each implementation)
had a great amount of difficulty with the
directions, others managed flawlessly.
3.
Some users had difficulty
distances in the text version.
judging
REFERENCES
Aretz, A.J.
“The Design of Electronic Map
Displays.” Human Factors, 33, 1991, pp. 85-102.
the
9
Butz, Andreas, Baus, Jörg, Krüger, Antonio and
Lohse, Marco.
“A Hybrid Indoor Navigation
System.” Proceedings of IUI2001: International
Conference on Intelligent User Interfaces 2001, ACM
Press, New York, 2001, pp 25-32.
The Experience Music Project. “The Experience
Music Project.” Website. http://www.emplive.com.
2001.
Hanttula, Dan. “Lost & Found: Head-to-Head Pocket
PC GPS System Comparison.” SemperAptus.com
Reviews.
http://www.semperaptus.com/reviews/r080101.shtml.
2001.
Hanttula, Dan. “Verbal Directions to Anywhere from
Your
Pocket
PC.”
Pocket
PC
Review.
http://www.microsoft.com/MOBILE/pocketpc/revie
ws/gps.asp. 2001.
Hooper, Erika Yungkurth and Coury, Bruce G.
"Graphical Displays for Orientation."
Human
Factors. Vol. 36, 1994, pp. 62-78.
McPherson, Frank. “Pharos’ Pocket PC Navigator
with GPS and Voice Navigation.” Pocket PC
Review.
http://www.microsoft.com/mobile/pocketpc/reviews/
pharosnav.asp. 2000.
Microsoft Corporation. “Pocket Streets.” Website.
http://www.microsoft.com/pocketstreets/. 2001.
Towns, S., Callaway, C. and Lester, J. “Generating
Coordinated Natural Language and 3D Animations
for Complex Spatial Explanations.” AAAI–98:
Proceedings of the 15th National Conference on
Artificial Intelligence, 1998.
Trimble Navigation Limited. “All About GPS.”
Website. http://www.trimble.com/gps/. 2001.
Williams, K. W. “Comparing Text and Graphics in
Navigational Display Design.” Digital Avionics
Systems Conference, 1999. Proceedings. 18th,
Volume: C.2-7 vol.1, 1999, pp. 4.A.3-1 - 4.A.3-8.
Williams, K. W. “Navigational Display Design:
Displaying Nearest Airport Information.” The
International Journal of Aviation Psychology, 9(1),
1999, pp. 91-106.
10
APPENDIX A1: Screen Shots of Map Directions Used in Experiment
11
Map Directions
Map Key
Low Complexity Step 1
Low Complexity Step 2
Medium Complexity Step 1
12
Medium Complexity Step 2
Medium Complexity Step 3
Medium Complexity Step 4
High Complexity Step 1
13
High Complexity Step 2
High Complexity Step 3
High Complexity Step 4
High Complexity Step 5
14
High Complexity Step 6
High Complexity Step 7
15
APPENDIX A2: Screen Shots of Text Directions Used in Experiment
16
Text Directions
Text Key
Low Complexity Step 1
Low Complexity Step 2
Medium Complexity Step 1
17
Medium Complexity Step 2
Medium Complexity Step 3
Medium Complexity Step 4
High Complexity Step 1
18
High Complexity Step 2
High Complexity Step 3
High Complexity Step 4
High Complexity Step 5
19
High Complexity Step 6
High Complexity Step 7
20
APPENDIX A3: Experimental Instructions
21
Instructions to Participant
Imagine that you are meeting a friend on campus, but you can’t remember exactly where you
agreed to meet. There is not enough time to call or e-mail the friend, but luckily you have your
UMCP Rover with you. This is a location-aware device, meaning that the device knows where
on campus you are located (using a GPS system) and where your friend is located as well. The
UMCP Rover can give you directions from your current location to where your friend is waiting
for you.
In designing the interface for such a device, we are interested in knowing which kind of
directions get you where you need to be faster and with fewer errors – map or text directions. To
answer this question we have designed a task for you. You will be given a handheld device that
has directions from a starting point to an end point in either map form or text form. There will
be three sets of directions and the end points (supposedly where your friend is) will be marked
with a sign.
Because the infrastructure for the Rover system is not in place, we the researchers will act as part
of the “system”. If you make a wrong turn or stray off the path you are given and do not correct
yourself in a short amount of time, we will tell you that you are off course and to back up to the
previous step. We are available to help you if you are unable to determine a course of action, but
we are primarily observers. Using the Rover system is primarily a solitary activity involving just
you and the device, so please only ask for help if you are truly lost or confused.
22
APPENDIX A4: Pre-Task Questionnaire Form
23
Pre-task questionnaire
Participant # _____
Gender _____
Major _____________
1. How many semesters have you been taking classes at UMCP? ______
2. What percentage of the time do you navigate around campus using the following
modes of transportation?
Riding a bike ____
Driving ____
Walking ____
Taking a bus ____
3. How would you rate your familiarity with the UMCP campus?
1
2
Very
Unfamiliar
3
4
5
Neutral
6
7
8
9
Very
Familiar
7
8
9
Very
Fast
4. How would you rate your walking speed?
1
Very
Slow
2
3
4
5
6
Average
5. Have you ever used a handheld device before (e.g. Palm Pilot, pocket organizer,
pocket computer)?
a. If so, how many hours per week do you use it?
6.
Do you wear glasses or contacts to correct your vision? ____
24
APPENDIX A5: VZ-2 Spatial Ability Questionnaire
25
Question 1
Question 2
Question 3
Question 4
26
Question 5
Question 6
Question 7
27
Question 8
Question 9
28
Question 10
29
APPENDIX A6: Post-Task Questionnaire Form
30
Post-task Questionnaire
Participant # ____
1. How easy/difficult was it for you to determine how to begin your task?
1
Very
Difficult
2
3
4
5
Neutral
6
7
8
9
Very
Easy
2. How easy/difficult was it for you to determine what the next step was once you reached a
destination point?
1
Very
Difficult
2
3
4
5
Neutral
6
7
8
9
Very
Easy
3. How easy/difficult was it for you to determine if you had completed a step correctly?
1
Very
Difficult
2
3
4
5
Neutral
6
7
8
9
Very
Easy
4. How easy/difficult was it for you to determine that you had arrived at the destination point?
1
Very
Difficult
2
3
4
5
Neutral
6
7
8
9
Very
Easy
5. The directions given to me on the device were easy to understand.
1
2
Strongly
Disagree
3
4
5
Neutral
6
7
8
9
Strongly
Agree
5
Neutral
6
7
8
9
Strongly
Agree
6
7
8
9
Strongly
Agree
6. The tasks took too long to complete.
1
2
Strongly
Disagree
3
4
7. The directions were easy to read on the device.
1
2
Strongly
Disagree
3
4
5
Neutral
31
APPENDIX B: Raw Data
32
Raw Data
Breakdown of Subject Majors:
CS (2), ENSE (3), Philosophy, Undeclared, Biology, ENME, Math, Business, and BMGT
BMGT
8%
ENSE
26%
Business
8%
Math
8%
ENME
8%
CS
18%
Biology
8%
Philosophy
8%
Undeclared
8%
Figure 1: Breakdown of majors.
What people listed as their most frequent form of transportation around campus:
Walking (8), Biking (2), Driving (1), and Bus (1)
Bus
Driving 8%
8%
Biking
17%
Walking
67%
Figure 2: Breakdown of transportation
33
Completion and consulting times are in seconds.
Route Number References:
 Route 1: Low Complexity Route
 Route 2: Medium Complexity Route
 Route 3: High Complexity Route
Table 1: Subject
Subject #
1
2
3
4
5
6
7
8
9
10
11
12
Information
Gender
Male
Female
Male
Female
Female
Female
Male
Female
Male
Male
Male
Male
Direction type
Map
Text
Text
Map
Map
Map
Text
Text
Map
Map
Text
Text
Route order
123
132
123
132
231
321
312
231
213
312
213
321
34
Table 2: Pre-task Questionnaire and VZ-2 Spatial Ability Questionnaire Results
Subject # # semesters
1
2
3
4
5
6
7
8
9
10
11
12
11
2
4
7
11
8
3
4
2
3
3
10
Campus Walking Handheld Corrected Spatial
familiarity speed
use
vision
ability
9
7
No
No
9
3
6
Yes
Yes
7
8
6
No
Yes
9
8
5
No
Yes
9
9
6
Yes
Yes
8
8
7
No
No
9
8
7
Yes
No
4
9
5
No
No
5
8
6
No
No
9
7
6
Yes
Yes
2
7
5
No
Yes
3
8
5
Yes
No
8
35
Table 3: Task Data
Subject #
1
1
1
2
2
2
3
3
3
4
4
4
5
5
5
6
6
6
7
7
7
8
8
8
9
9
9
10
10
10
11
11
11
12
12
12
Route
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
Completion time
339
217
201
304
216
199
250
187
174
211
199
188
208
251
192
204
221
189
295
215
264
232
183
177
227
229
210
401
262
304
304
215
179
219
187
218
Consulting time
137
24
13
64
14
6
58
17
14
17
11
6
9
25
6
20
8
8
51
4
38
21
10
6
35
26
16
76
38
64
81
22
8
12
10
18
36
Errors
2
0
0
3
0
1
3
0
0
2
0
0
0
1
0
0
1
0
0
0
0
1
0
1
0
0
0
2
0
0
3
1
0
1
0
0
Table 4: Post-task Questionnaire Results
Subject #
1
2
3
4
5
6
7
8
9
10
11
12
q1
5
3
7
8
4
7
5
5
7
6
9
8
q2
7
8
6
7
7
8
9
9
8
6
8
8
q3
5
8
2
5
9
8
9
9
8
7
7
8
q4
9
9
9
9
9
8
7
9
8
8
8
8
q5
6
7
4
9
4
8
6
9
8
6
9
7
37
q6
5
2
4
1
1
3
5
3
5
7
7
2
q7
5
8
7
7
4
6
9
5
8
6
9
8
APPENDIX C: Experimental Consent Form
38
Experimental Consent Agreement
1. I have freely volunteered to participate in this experiment.
2. I have been informed in advance as to what my task(s) would be and what
procedures would be followed.
3. I have been given the opportunity to ask questions, and have had my questions
answered to my satisfaction.
4. I am aware that I have the right to withdraw consent and discontinue participation at
any time, without prejudice.
5. My signature below may be taken as affirmation of all of the above, prior to
participation.
PRINTED NAME
Student ID
39
SIGNATURE