NC State University Libraries
Search Usability Testing
1
January 2010
ALA Midwinter Conference
Josh Boyer & Susan Teague Rector
Who We Are
Josh Boyer
 Associate Head, Distance Learning
and Research and Information Services




Susan Teague Rector
Web Design Project Librarian
Twitter: seteague
2
Today’s Agenda

Background

Usability Test Methodology

Testing Results

Implementation of Search

Search Usage

Next Steps
3
Background

Website Redesign January – August 2010 whose
main goal of site design was to emphasize and
streamline search across the site

Team of 9 librarians appointed to look at the
website’s search environment

Building materials:
Endeca-based catalog interface
 Summon
 Journal Titles list
 site search
 “QuickSearch”

4
www.lib.ncsu.edu/search/?q=xml
5
www.lib.ncsu.edu/search/?q=gis
6
How about Tabs?
7
Research Questions



How would users interact with a tabbed search
interface?
Would users choose to scope their searches by
preselecting a tab before beginning a search or would
they conduct all searches in the default “all” tab?
How successful would users be in selecting the correct
tab for the task?
8
Search Prototype
9
2 Search Prototypes
Model 1
Tabbed Search Results
Model 2
Non-Tabbed Search Results
10
Our Methodology

“Guerrilla” usability testing in the field

Participants recruited in the lobby of DH Hill Library

Testing took place over a 3 hour period


28 undergraduates, graduates and library staff,
participated in a round of guerrilla usability testing for
2 proposed search models
Each participant was asked to complete 2 tasks using
one of the search models; many participants
volunteered to answer more than 2 questions
11
Who Were the Participants?
Science/Engineering:

22 Undergraduates

3 Graduates
•
Aerospace Engineering

2 Library Staff Members
•
Animal Science/Biology

1 Non-Traditional Student
•
Biology
•
Biomedical Engineering
•
Chemistry
•
Civil Engineering
Represented:
•
•
Humanities:
•
•
English/Creative Writing
•
Computer Science
•
History
•
Electrical Engineering
•
Psychology
•
Food Science
•
Human Biology/Nutritional Science
Business:
•
Business
•
Integrated Management & Systems
•
Business Administration
•
Meteorology
•
Textiles Engineering


14 participants performed 46 tasks using Model 1
14 participants performed 38 tasks using Model 2
12
Observations

In both search models, in well over half the tasks,
participants pre-selected a search tab before
beginning their search.
13
Observations

Across both search models, in 45 of 60 tasks in
which users selected a tab, they selected the
appropriate tab for that task.
14
Observations

In both models, users rarely switched silos in the
search results once in a silo. Users tended to
stay in the same silo and refine searches within
those silos.
15
What Worked
Task No.
Task
1st Tab Searched
(Across Models)
1
n=9
Please tell me if the library has a
copy of the book titled ‘The Tipping
Point’ by Malcolm Gladwell
All (4)
Books (5)
3
n=9
Please locate information about
how much it costs to make
photocopies in the library
All (3)
Web site (5)
Books (1)
4
n=9
You are studying the effects of ADD
in the classroom for your
psychology class. Can you locate 2
books about helping students with
ADD succeed in school?
All (4)
Books (5)
8
n=8
You are in DH Hill Library and you
would like to locate an illustrated
encyclopedia about trees; where
would you find such a book at Hill?
All (3)
Books (4)
Web site (1)
9
n=7
Please locate information on how
to borrow a laptop from the library;
how long is the loan period?
All (1)
Web site (5)
Books (1)
10
n=7
Please locate the most recent issue
of the Journal of the American
Medical Association.
Journals (6)
All (1)
16
Problem Areas
Task No.
Task
1st Tab Searched
(Across Models)
2
n=9
Find 2 journal articles on how
tropical forest fires impact the
environment
Journals (4)
Articles (4)
All (1)
6
n=9
You are doing a lab on the
reproductive habits of fish in
your Bio class. Your professor
has recommended that you
use the Web of Science
database to find relevant
articles. How would you do
that on this site?
Articles (3)
Journals (1)
Website (1)
All (4)
7
n=8
You are writing a paper for a
course in Computer Science.
Find the 2001 article titled
‘Internet Under Siege’ by
Lawrence Lessig in the journal
called Foreign Policy.
Journals (3)
Articles (4)
All (1)
17
Other Issues



Even if users executed a successful search, they
often did not scroll down far enough in the search
results to find the answer
Almost all participants only scanned the first page
of search results for a quick answer to the task
Spelling was problematic
18
www.lib.ncsu.edu
19
Search Usage
20
Use of “All” tab (QuickSearch)
8.3%
43.8%
36.4%
2.4%
21
Articles – Full Text Downloads
17.3 %
increase
22
Next Steps


Conduct more usability testing with students.
Interview faculty about their research and how
they view our new site.
23
Questions?

Thank you!

Contact us:
susan_teague_rector@ncsu.edu
 josh_boyer@ncsu.edu

24