Evaluation approaches
Text p 340 - 347
Text p 590- 595
Evaluation approaches
• Usability testing
• Field studies
• Analytical evaluation
• Combining approaches
• Opportunistic evaluations
Usability testing
• Dominant approach in the 80’s
• Involves measuring typical users
performance on typical tasks
• Note the number and types of errors
that users make and record the time
it takes them to complete the task
• Recorded on video/audio with
interactions often recorded
• User satisfaction questionnaires and
interviews are also used to elicit user
opinions
• Test environment and format of
testing is controlled by the valuator
Field studies
• Done in natural setting with the aim of
understanding what people do naturally and
how products mediate their activities
• Help identify opportunities for new technology
• Establish the requirements for design
• Facilitate the introduction of technology or how
to deploy existing technology in new contexts
• Evaluate technology
Analytical evaluation
• 2 categories:
– inspections
• heuristic evaluation
• Walkthroughs
– theoretical based models
• keystroke model
• Fitt’s Law
Analytical evaluation
Heuristics
• Based on common sense knowledge and usability
guidelines
• Originally developed for screen based applications
• Adapted to make new sets for evaluation web based
products, mobile devices and computerised toys
• Care needs to be taken as designers are sometimes led
astray by findings from heuristic evaluation that are mot
as accurate as they first thought
Analytical evaluation
Cognitive Walkthroughs
• Involve simulating a users problem solving
process at each step in the human computer
dialog and checking to see how users progress
from step to step in these interactions
• Focus on evaluation designs for ease of
learning
• Other walkthroughs developed: pluralistic
Analytical evaluation
Predictive models
• Have been primarily used for comparing the
efficacy of different interfaces for the same
application
• Eg the optimal arrangement and location of
features on the interface base
• The keystroke model provides numerical
predictions of user performance
• Fitt’s Law predicts the time it takes to reach a
target using a pointing device
Characteristics of approaches
Usability
testing
Field
studies
Analytical
Users
do task
natural
not involved
Location
controlled
natural
anywhere
When
prototype
early
prototype
Data
quantitative
qualitative
problems
Feed back
measures &
errors
descriptions
problems
Type
applied
naturalistic
expert
Evaluation approaches and
methods
Method
Usability
testing
Field
studies
Observing
x
x
Asking
users
x
x
Asking
experts
Testing
Modeling
x
Analytical
x
x
x