COMP5427
Usability Engineering
Week 13: Tutorial
materials
Overview
Introduction (5 minutes)
• You questions (post on whiteboard, or simply tell tutor during the class)
1. Groups: share mini-assignment sample exam question 1 (10 mins .. 15)
2. Class: in turn groups contribute to whiteboard answers (10 mins .. 25)
3. Groups and class interspersed: activities around the mini-assignment
sample exam Question 2 (20 mins .. 45)
A selection of your questions (or take more time for #3) (10 mins .. 55)
Question 1 from the miniassignment
Consider the following graph discussed in lectures. It is from Jacob Nielsen’s
reports of the performance of evaluators in a Heuristic Evaluation of a banking
system. Each row is an evaluator (n=19) and each column is a flaw (n=16). Black
squares show where an evaluator found the problem. The rows are sorted with
most successful evaluators at the bottom.
a) (2 marks) How many evaluators are recommended for
Heuristic Evaluation as a discount usability method?
Justify your answer, referring to the graph above.
b) (2 marks) Why is Heuristic Evaluation described as a
predictive method?
c) (2 marks) State the three factors defining the severity of flaw. For each,
briefly state why it is important to consider.
1.
2.
3.
d) (2 marks) What is one key disadvantage of Heuristic Evaluation compared
with Think-Aloud evaluations.
e) (2 marks) What is one key advantage of Heuristic Evaluation compared
with Think-Aloud evaluations.
Activities around Question 2
from the mini-assignment
The screen shot below is for the web interface
for reading the class textbook at the
University of Sydney website. Suppose you are
evaluating the usability of this e-textbook
interface.
• a) (3 marks) Write one important abstract
task for evaluating the usability of this etextbook. Explain why this is an important
task.
• b) (3 marks) Write one important, welldesigned concrete task for the abstract task
in Part a. Write it in the form it would be
presented to a user in a think-aloud study.
• c) (4 marks) Explain how your concrete task
in Part b is well designed in terms of
avoiding leading the user. Illustrate your
answer by including an example of a variant
of the same task, but written to lead the
user.
a) (3 marks) Write one important abstract task for evaluating the
usability of this e-textbook. Explain why this is an important task.
Answers to critique:
Abstract task: Find a particular page by searching for a string.
Abstract task: Find a particular page by searching Page 17.
Abstract task: Mark up a page that has important information so you
can easily come back to it later.
b) (3 marks) Write one important, well-designed concrete task for the
abstract task in Part a. Write it in the form it would be presented to a
user in a think-aloud study.
Please write some concrete tasks for the above in groups and to
collaboratively critique them.
c) (4 marks) Explain how your concrete task in Part b is well designed in
terms of avoiding leading the user. Illustrate your answer by including
an example of a variant of the same task, but written to lead the user.
First write a bad task that does lead the user then a better version.
d) (3 marks) Write another important, well
designed concrete task for your abstract
task in Part a, taking care to chose it so
that it can contribute to good coverage of
the important tasks. Write it in the form it
would be presented to a user in a thinkaloud study.
e) (3 marks) Explain why this task in Part d
is an important task and how it contributes
to good coverage of the important tasks.
f) (4 marks) Explain why the task in Part d
is well designed in terms of being
understandable to the users. Your answer
should state key assumptions you make
about the user’s mental model and your
reasoning for making these assumptions.
As you discuss these in your groups, come
up with counter examples that do not add
to the coverage (e) and that examples that
illustrate poorer task design (f)
Properties of good concrete tasks
They are concrete
a user understands what they need to do after reading them: the instructions need to be clear, easily
understood provide all the information the user needs, different people should interpret them in the
same way
You can judge success
There should be a clear start and clear end at which point you can assess whether the user was able to
do the task successfully, needed help, or could not do it.
There should be a clear outcome.
They do not lead the user
Tell WHAT to do, not HOW to do the task.
Avoid any words that appear at the interface.
They are relevant
Users of your real system would be expected to do them
The set of tasks gives good coverage
Frugal: each task tests different things (unless you explicitly want to study repeat use)
Effective: each task covers an important aspect
They are at the right level of difficulty
Start with easy tasks
Ensure you time the test and tell the user ahead of time an estimate of the duration
They are respectful and avoid offending the user
Humour is dangerous
13