Heuristic Evaluation
YASER GHANAM
Roadmap
 Introduction
 How it works
 Advantages
 Shortcomings
 Conclusion
 Exercise
Introduction
 Introduced by Nielsen as a discount usability
method.
 Early in the design or during implementation.
 Given:
 A prototype or a working system
 A set of usability heuristics
 A few evaluators
 Come up with:
 A usability evaluation of the system
Introduction
Usability
Heuristics
The
system
Evaluators
Usability Evaluation
How it works - Procedure
1
• Get the heuristics
2
• Get the system ready
3
• Get the evaluators
4
• Do the evaluation
5
• Compile the results
6
• Conduct severity rating
7
• Develop an action plan
Get the heuristics
 Heuristics are system dependent.
 Nielsen’s heuristics proved reliable & representative.
 Feel free to add more heuristics but not many.
 Feel free to drop irrelevant ones.
Usability Heuristics
 Visibility of system status
Usability Heuristics
 Match between system and the real world
Usability Heuristics
 User control and freedom
Usability Heuristics
 Consistency and standards
Greenberg, S., Overview of Heuristic Evaluation, http://pages.cpsc.ucalgary.ca/~saul/
wiki/uploads/CPSC681/Heuristic.ppt, accessed October 10, 2007.
Usability Heuristics
 Error prevention
Usability Heuristics
 Help users recognize, diagnose, and recover from
errors
Usability Heuristics
 Recognition rather than recall
Usability Heuristics
 Flexibility and efficiency of use
 Shortcuts:
Ctrl


C
Normal mode vs. Advanced mode
Don’t make it an alternative
Copy
Usability Heuristics
 Aesthetic and minimalist design
Usability Heuristics
 Help and documentation
Ability to
use system
Good
Usability
No/minimal
docs
How it works - Procedure
1
• Get the heuristics
2
• Get the system ready
3
• Get the evaluators
4
• Do the evaluation
5
• Compile the results
6
• Conduct severity rating
7
• Develop an action plan
Get the system ready
 Prototype: novel application or interface
 No redesign required
 Less maintenance later
 Working system: replacement study or competition
 Prepare typical scenarios: task analysis
How it works - Procedure
1
• Get the heuristics
2
• Get the system ready
3
• Get the evaluators
4
• Do the evaluation
5
• Compile the results
6
• Conduct severity rating
7
• Develop an action plan
Get the evaluators
 HE is a group effort.
Nielsen, J., and Mack, R.L. (Eds.),
Usability Inspection Methods,
p25-62. John Wiley & Sons,
New York, NY (1994).
Get the evaluators
 The more the better? Not necessarily
 Rule of thumb: 3 to 5 evaluators
Nielsen, J., and Mack, R.L. (Eds.),
Usability Inspection Methods,
p25-62. John Wiley & Sons,
New York, NY (1994).
Get the evaluators
 Evaluators’ expertise
 Novices: better be potential users of the system
 Usability experts: more effective
 Double experts (both in usability and the domain): the best to
get but very expensive
 Session manager: facilitates the evaluation session &
aggregates reports
 Observers: provide help to evaluators
How it works - Procedure
1
• Get the heuristics
2
• Get the system ready
3
• Get the evaluators
4
• Do the evaluation
5
• Compile the results
6
• Conduct severity rating
7
• Develop an action plan
Do the evaluation
 Evaluators get the heuristics & scenarios.
 Navigate through the system twice.
 Inspect screens, dialogues, forms, messages and
menus in the system.
 Categorize any problem under one of the heuristics:
should give specific explanation.
 Can make comments beyond the heuristics.
 Report in writing or verbally to the observer.
Do the evaluation
 Observers answer evaluators’ questions.
 especially domain-specific questions.
 but, without influencing judgments.
 IMPORTANT:
 Inspection should be done individually.
 Evaluators are not allowed to communicate.
 Session takes 1 to 2 hours.
How it works - Procedure
1
• Get the heuristics
2
• Get the system ready
3
• Get the evaluators
4
• Do the evaluation
5
• Compile the results
6
• Conduct severity rating
7
• Develop an action plan
Compile the results
 Aggregate evaluators’ reports.
 Eliminate duplicate entries.
 Merge different problems.
 Output: One report of all usability problems found
by the evaluators.
How it works - Procedure
1
• Get the heuristics
2
• Get the system ready
3
• Get the evaluators
4
• Do the evaluation
5
• Compile the results
6
• Conduct severity rating
7
• Develop an action plan
Conduct severity rating
 Evaluators are aware of all usability problems.
 Severity determined by:
 frequency of occurrence
 impact on user
 persistence
0: Not a
problem at all
1: Cosmetic
problem
2: Minor
problem
3: Major
Problem
4: Catastrophic
problem
How it works - Procedure
1
• Get the heuristics
2
• Get the system ready
3
• Get the evaluators
4
• Do the evaluation
5
• Compile the results
6
• Conduct severity rating
7
• Develop an action plan
Develop an action plan
 Evaluators, facilitator & the design team meet.
 Discuss problems .
 Suggest solutions.
 Consider organization’s priorities.
 Take decisions: fix major not minor problems, delay
release, replace interface … etc.
Advantages
 Discount usability methods: few gives many.
 Easy to teach.
 Fast to conduct.
 Cheap.
 Can be used in early design.
 High benefit to cost ratio: i.e. 48
Shortcomings
 Reduced set of heuristics is very broad and general
 Usually does not involve end users
 Finds many minor problems causing a “false alarm”
sometimes.
 Not suitable for in-depth usability testing or critical
systems.
Conclusion
 A discount usability engineering method.
 Best for time-constrained, budget-limited projects.
 3 to 5 evaluators follow 10 heuristics.
 Finds many problems in a short time.
 Does not replace other usability methods.
Thanks for listening
?
?
Questions
?
?
Exercise
Nielsen, J. (1993). Usability Engineering, 273-274. Academic Press.