Reflecting Our Users:
How Usability Testing Improved Our Web Page
Jane Nichols with Team 200 and the U-team, Oregon State University Libraries
Corvallis OR 97331
Methods
Introduction
Personas & Scenarios
Card Sorting
Think aloud protocol
OSU Libraries conducted two rounds of usability testing on its web site in Spring 2007 and Fall 2007. Methods
employed included card sorting, the use of personas and scenarios and the think aloud protocol. Results were used to
redesign OSU Libraries web site and to improve weaknesses of each design iteration. Usability results showed
continued points of confusion and revealed challenges with meeting users wide ranging needs.
Results Card Sort
Personas & Scenarios
Context
Team200 created and wrote
detailed Personas and Scenarios
for 6 types of key users: novice
student, expert student
(international), faculty researcher
and instructor, e-campus student,
community member and an on-call
librarian.
This work was made possible because of the close working relationship of two teams: Team 200 and the U-team. Team
200 oversees the OSU Libraries web presence and the U-team ensures the usability of the OSU Libraries and its
affiliated web sites.
The Usability Team developed
primary tasks from the personas
and scenarios and formed them
into usability tasks.
Round One
Round Two
Persona: “description of a specific
person who is a target user of “ a
site
Team200 facilitated the card sort and some members of the U-team
participated. Results showed that participants both mirrored library terminology
and suggested new phrasing. Common terms and phrases included: catalogs,
library services and collections (other, specific, library). Unique phrasing
included checking things out or stuff to check out, library partners, involving
you and other departments in building.
Participant groupings influenced the contents of the navigation menu. For
example, the Round 1 menu adopted the categories catalogs, services,
learning spaces and collections.
Results Round 1 Usability
Results Round 2 Usability
Many used an optimal path to find: a book, the
library hours, an e-journal, a database,
information about reserving a room to practice
a presentation and to chat with a librarian.
Many tasks were completed by using an
efficient path. Changes to the navigation menu
labels and the addition of explanatory text
helped with some tasks and did not present
new problems.
Hidden information was find your departmental
librarian and how to recommend a book for
purchase.
Scenario: description of target
users’ interactions with a site
Tasks completed successfully albeit with a long
path included finding the reference desk phone
number, a library tutorial, loan period
information.
Course reserves links were misleading
because it was not clear which was for
students to find a reserve and which for faculty
to place a reserve
Card Sort
Ten participants sorted 51
cards. The cards represented
links on the OSU Libraries
home page and navigation
menus.
Participants grouped the
cards into categories that they
created.
Team 200 used the results to
make decisions about the new
site architecture and
navigation.
The library hours placed in the middle of the
page were not visible to several participants.
Tasks done with ease were: find ILL, a
database, a DVD, a web page helpful to your
discipline or major; renew a book and chat with
a librarian.
The Find Other Resources link now explains
how to access magazines and newspapers,
this helped some.
Hours are more visible in the page footer.
Summit was moved to Find Books & Articles.
Find books link lists OSU Libraries catalog,
Summit catalog, WorldCat and other options.
Can still find even though not a navigation
menu option.
The scrolling news bar was not visible.
Continued Challenges & Points of Confusion
Finding journal articles continues to challenge novice users. There are several issues: where to start,
distinguishing between journal titles and article titles, how to get to the latest print journal issues in the
library catalog. A related issue is the continued confusion of having to select Journal Title to find
magazines and newspapers.
Although participants ultimately succeeded in finding a known journal article, there isn’t always a
quick, direct path. In Round Two, three routes were used: library catalog, databases and the search
box.
Usability Sessions:
Think Aloud Protocol
Round 1: 4 undergrads and 4 grads
completed 18 usability tasks, 3 faculty
did 8 tasks
Round 2: 4 undergrads and 4 grads
completed 22 tasks, 3 faculty did 13
usability tasks.
All participants completed 5
demographic & computer/library use
questions before doing the tasks and
an 8 question satisfaction post-test
questionnaire afterwards.
Timeline
Personas 2/2007
Task analysis & creation
Improving Weaknesses
Weaknesses in each round were improved by changing
navigation menu labels, having a navigation menu that
users could learn from and by providing more explanatory
text for novices.
Round 1 usability sessions 4-5/2007
Card sorting 8/2007
New Interface with left navigation bar & chat 9/2007
Round 2 usability sessions 11/2007-1/2008
The label Using the Library helped participants quickly find
information about policies and services that were
previously difficult to find. Several participants in Round
Two said they had seen links in the navigation bar during a
prior task and that helped them complete later tasks.
Participants took advantage of explanatory text to find
books, magazines and newspapers.
Details about the catalog are not clear: abbreviations like Lib Has, finding recent issues, material type.
and revealed challenges with meeting users wide ranging needs.
Participants who get lost sometimes stay lost. Typical problems are not finding journal articles or
confusing the Summit and the OSU Libraries’ catalogs.
OSU Libraries continues to balance providing enough explanatory text on the web page so new and
infrequent users can understand terms and find their way and so frequent users or those who know
what they need and want have efficient, clear paths to resources.