Cengage Learning
Opposing Viewpoints In Context
Usability Test
Hao Zeng, Mikko Tuomela,
Josh Keyes, Travis LaFleur
SI 622 Group 3
April 14, 2011
Cengage Learning
 Publisher of print and digital information services for
academic, professional and library markets
 Delivers customized learning solutions for colleges,
universities, professors, students, and libraries
Opposing Viewpoints In Context
(OVIC)
 OVIC is an online resource covering today’s hottest
social issues
 It helps students research, analyze, and organize broad
variety of data
 Used for conducting research, writing assignments,
debates and more
Test Recruiting
• Sent email to SI Informatics email list
• Gave general information about study along with how
much time it would take
• Promised them a $25 Amazon gift card after their
participation
• We aimed for 5-7 subjects
• Scheduled 7, 2 canceled so we tested 5 subjects
Usability Test Tasks
 4 Tasks
 15 minutes
 Client: focus on search and multimedia content
+
 Try to simulate normal operations that a general user will
usually do in the system
Usability Test Tasks
1. Find two articles about nuclear power, one in favor and
one opposed. Translate the second article into French
and send it to your French friend (hao.hci@gmail.com).
2. Find 2 radio broadcasts about the 2011 earthquake and
tsunami in Japan, save them to your personal favorites
in the system, and then start playing one of them.
3. Find a magazine article about unemployment and
download the article’s APA citation.
4. Find an article related to stem cell research from a peer
reviewed journal that was published in 2005.
Test Logging
• Logging sheet for each task
o From 2 to 4 subtasks
 Completed? Y / N
o General questions
 Surprised? Y / N
 Stuck?
Y/N
 Confused? Y / N
o Space for comments and
notes
• To be filled either by pen or
by computer (Word
document)
Moderation & Debriefing
• Introduction script
• Sign consent form
• Pre-test questionnaire
o Background, demographics, experience
• Begin tasks
• Encourage talk aloud
• Post-test questionnaire
o Reflect on experience
o Likes, dislikes, improvements
• Ask for questions
• Thank and dismiss
Recording
Location and Setup
• 1150B/1150C at North Quad
• 3-4 researchers present
• Audio recording by computer
Usability Test Findings and
Recommendations
Usability Test Findings: #1
The “Listen” appears on top of the
content of every document for people
with visual impairment;
But it causes significant confusion for
general users.
Usability Test Findings: #1
Usability Test Findings: #1
Usability Test Recommendations: #1
Recommendation: Rename to "Read the Screen" and move to Tools section.
Usability Test Findings: #2
Pro and Con Indicators Are Needed In
Search Results
• System is designed to give
users arguments from "both
sides" of an issue
• Sides of articles aren't readily
identifiable in search results
Usability Testing Recommendation #2
• Implement short summary to let user know
if articles is "Pro" or "Con" for the subject
that they are searching about
• "Pro" or "Con" summary could be anywhere
from one sentence up to a short paragraph.
• Summary text could be placed to left of
search results, just make column with
results in it a bit skinnier (which would be
ok)
Usability Testing Recommendation #2
Usability Test Findings: #3
Basic Search is used often, but is not fully satisfactory.
Usability Test Findings: #3
• Does not handle Google-style queries
"stem cell research 2005"
"Japan tsunami audio"
• Recommendation: improve search
algorithm to accept more parameters
(not just keyword)
Usability Test Findings: #3
Recommendation: add filtering options
from advanced search into results page
for basic search
Usability Test Findings: #4
Finding 4: Options in Advanced Search Are
Ambiguous
• Users were confused about what "Content Level"
means
• Because they don't know what content level means,
they can't make sense of the options that they can
select
• It frustrated users when they weren't able to tailor the
advanced search options clearly
Usability Test Recommendations: #4
• Cengage should name "Content Level" something
that is more descriptive of what the feature actually
filters
• Cengage could add a help section that describes all
of the features that are offered in the Advanced
Search
Thank you!
Questions?