ComputerPrototypesand
FormativeUsabilityTests
MattiasArvola
2
Today
•
•
•
•
•
•
•
•
Types of prototypes
Fidelity
Computer prototypes
Tools for building computer prototypes
Preparing prototypes for testing
Setting up formative tests
Evaluation of mental models
Think-aloud protocols
3
Prototype(Merriam-Webster)
• an original model on which something is patterned : archetype
• an individual that exhibits the essential features of a
later type
• a standard or typical example
• a first full-scale and usually functional form of a new
type or design of a construction (as an airplane)
4
Whyprototype?
• Good UX does not come from nowhere. It is based on
many hours of work
• Designing and re-designing in series of user testing
sessions
• Iteration is the mother of quality
5
Human-CenteredDesign(ISO9241-210)
1.Planthe
human-centred
process
Meetsthe
requirements
2.Understand
thecontextof
use
5.Evaluate
against
requirements
3.Specifyuser
&business
requirements
4.Produce
design
soluAons
ideaLon
revisions
detailing
Uncertainty/PaOerns/InsightsClarity/Focus
ideaLon
revisions
detailing
Whatitisandnotis
https://uxmag.com/articles/what-a-prototype-is-and-is-not
• A simulation of the final product. It is not THE final
product
• Interactive mockup with any degree of fidelity. It does
not always look like the final product
• Test the smoothness and consistency of the flow of the
product.
• Provide insight into user interaction
• Test the feasibility and usability before writing code
• Make unexpected discoveries and innovations
8
9
Evolutionaryorthrow-awayprototypes
10
WhatPrototypesPrototype
FuncLon
Integrated
ConstrucLon
Look&Feel
11
LoFiandHiFiPrototypes
• Low Fidelity: Often paper-based. Used to test a design
with users and predict big problems at an extremely
low cost.
• High Fidelity: Interactive and computer-based. Used
to convince management and other stakeholders that
you have thought the design through and are on the
right path. Cost can be rather high, and they can
create to large expectations from customers.
• Mixed Fidelity: Some parts are HiFi and others LoFi
13
Fidelityinwhatrespect?
• Interaction: Transitions, animations, system
responses
• Data: Real or realistic data
• Width in functionality: Overview of e.g. withdrawal,
deposit, account current, payments
• Depth in functionality: Details of e.g. withdrawal
• Visual: Sketchy or pixel perfect
14
15
16
Howmuchshouldweimplement?
• Build it if you can
• Fake it if you cannot
• If nobody can building, then your design is wrong
17
Whyandwhatofcomputerprototypes
• Bring a prototype with you to get more out of a
stakeholder meeting
• Response times, states of graphical objects and
animations
18
WizardofOz
• Gives the users the impression that they are working
on a real system when there is no such real system
• Controlled completely or partially be a wizard that
looks at what the user is doing and then provides the
system response
• Makes it possible to test advanced AI to se if and how
it should be constructed
• Can also be used to build a corpus of interactions for a
natural language user interface
far from the participant, helping to make the
Overkamp,T.,&Holmlid,S.(2015).MarionetteprototypingforevaluatingconceptualUbiCompapplicationsintheircontext.In
manipulations non-obtrusive for the participant.
ParticipatoryInnovationConference2015,TheHague,TheNetherlands.
WizardofOz
Figure 1. Imagined wireless interaction with a ubicomp object. The
object sends information directly to the user’s device.
19
Figure 1. Imagined wireless interaction with a ubicomp object. The
object sends information directly to the user’s device.
Figure 2. Marionette setup of the ubicomp interaction. For the user,
the information seems to be coming from the object, whereas in
reality, the researcher sends this information to the user’s device.
Thus, in the Marionette Prototyping method we create a
Figure 3. The setup for the researcher following along on the walkthrough.
The tablet on the left, for manipulating the user-device in the
manipulator role, and the notepad on the right, for taking notes in the
researcher role.
InteractiveComputerPrototypes
• Looks and behaves like the final product
• Test the feel and detailed interaction
• It takes an effort to make changes in it (compared to
paper prototypes) and a bug can put the entire test
session to a halt
22
23
IterativeandEvolutionaryPrototypes
• Built to both test a design and to build on for the
future
• Made in the development environment, but preferably
with a GUI builder
• Such environments have component libraries: easy to
use standard widgets
• Easy to fixate on standard components and widgets
24
ThreeBasicKindsofPrototypingTools
• Hot spot-based tools that take existing wireframes /
mock-ups and add interactivity
• E.g. InVision, POP, Flinto, Keynote, Powerpoint
• States-based tools with UI components
• E.g. UXPin, Axure RP, proto.io, Pixate, Atomic,
Indigo Studio, Fluid UI
• Code tools that take and load dynamic data: E.g.
Framer, LiveCode, Interface Builder, Visual Studio
Express, Qt Creator, Noodl
proto.io
Principle
Indigo
Studio
Protoshare
Pixate
HotGloo
Solidify
Origami
Framer
UXPin
Marvel
Form
Webflow
Justinmind
Fluid
Briefs
InVision
LiveCode
Axure
Keynotopia
Atomic
QTCreator
Prototyping
onPaper
Noodl
InVision
•
https://www.youtube.com/watch?v=zNoBmjg-NnQ
26
UXPin
•
https://www.youtube.com/watch?v=NM30LmppB5U
27
LiveCode
•
https://www.youtube.com/watch?v=PeYBSFoiOqI
28
29
PrototypingToolswithProsandCons
• For example:
• http://www.cooper.com/prototyping-tools
30
Furtherreading:
• http://www.inuse.se/blogg/tre-proptptypverktyg-duinte-far-missa/
• http://www.cooper.com/prototyping-tools
• https://medium.com/@msds_branding/5-fantasticux-prototyping-tools-part-1-3a4d05f8801f
• http://ms-ds.com/our-thinking/insights/5-best-uxprototyping-tools-part-two
• https://uxmag.com/articles/comparing-popularlayer-based-and-code-based-prototyping-tools
• https://uxmag.com/articles/comparing-fourpopular-page–based-interactive-prototyping-tools
Formativeusabilitytests
32
T-prototypes
• It’s a prototype – Not the Mona Lisa
• Set a deadline
• Prototype only what you need
33
Preparetestscenarios
Rettig,1994
• Draw on your user research on typical situations of
use
• Construct the prototype to support a few task
scenarios
• Reasonably small set of functions but enough to allow
meaningful tests
• Make sure you use realistic scenarios and data: test
them on someone
• If possible: balance the order
• Make a pilot run!
34
Agoodtaskscenario
Lewis&Reiman,1993,1994
• Specific
• Describes a complete job
• Test how different features of the interface work
together
• Example from Anneli Friberg, LiU Library
•
"Du letar efter boken Skriva uppsats med kvalitativ metod av Johan
Alvehus. Hitta den och ta reda på om den finns att låna på Campus
Norrköpings bibliotek.”
•
”You are looking for the book Successful Writing for Qualitative
Researchers by Peter Woods. Find it and determine if it is available
to borrow at the library at Campus Norrköping.”
35
Selectyourusers
Rettig,1994
• Representative in terms
of for example:
– Education and
training
– Computer knowledge
– Domain familiarity
– Typical tasks
• Try to get the whole
range of users
• Surrogate users
– Sometimes hard to
get
– Politics involved
– Ethics
• Avoid friends and
family and other clearly
biased people
36
Rolesthatyou
rotateintheteamRettig,1994
• Greeter: Make
participants feel at ease
and introduce them.
Hand out test screener
• Facilitator: Gives
instructions,
encourages the user to
express thoughts during
the test, keeps track of
time
• Observers: take notes
on index cards. One
observation per card.
Ideas for solutions on
the same card
37
Threephases
Rettig,1994
A session typically lasts about an hour, depending on
the size of the prototype
1. Getting ready
2. Conducting the test
3. Debriefing
Gettingready(andsomeethics)
• Greetings, introductions,
refreshments, and icebreaking
• Set expectations and
explain:
– introduce the
prototype
– the equipment
– what you will do with
the data
– think aloud testing
38
• Assure anonymity
• Make sure that they
understand they can quit
at any time
• Clarify that you are testing
the design and not testing
the participants
• Setting up the camera to
record the interaction
with prototype but not
people’s faces
• Ask if there are any
questions
39
Uncoveringtheinitialimpressionand
mentalmodelGreenbergetal2012
• What is this product/system/website for?
• What is your first impression?
• Then briefly introduce the design and ask people to
describe their understanding, in detail, of every
design element on the screen
• If their explanations doesn’t match your intention you
have a problem in people’s initial mental model that
you need to address in a redesign
40
Conductingthetest
• The facilitator hands
written tasks to the
user. Tasks should be
clear and focus on what
the users want to
achieve but describe not
how.
• One page for each task,
one task at the time
• Facilitator encourages
people to think aloud if
they stops talking:
– What are you
thinking right now?
– What questions are
on your mind?
41
Theobservers
• May sometimes interject a question
• Never laugh, gape, say ”a-ha”, nudge each other, or
display any reaction
• Intimidation and humiliation can ruin the
relationship with the participant and spoil the test
• Don’t ever explain or defend your design to the users
42
Debriefing
• Discuss interesting behaviours that you would like the
participant to explain without accusing the user
• Ask your final questions
• Gather impressions and participant’s thoughts on the
design
• Ask if the participant has any questions
• Express thanks
43
Informedconsent(moreethics)
• You need to assure that
your participants
understand
– that they agree to
participate in the study
– that it is voluntary to
participate
– that they agree to
immediately raise concerns
or discomforts during the
session
– that their interaction is
recorded for research
purposes
– how you will use the results
and the recordings
– that they will be
anonymised
• Consider using a form
• Confirmed consent with
minors include inform
consent from their legal
guardians
44
Pre-testquestions
usability.gov
•
•
•
•
•
•
•
Male/female
Age
Income
Education
Work title
Location of workplace
Knowledge of
competing sites or
products
• Computer habits
(platforms, activities,
browsers, websites and
applications often used)
• Language skills
• Domain knowledge
• Technical knowledge
• Contact info
45
Observationprotocol
Usability.gov,Greenbergetal,2012
• Pathways users take
• Degree of success on the
task scenario
0. not completed
1. completed with difficulty
2. easily completed
• Look for places where
participants have
troubles
• Look for patterns of
behaviour that tells you
if the product was
understood correctly
• Review it afterwards on
the video
46
Debriefingquestions
Usability.gov
Overall impression?
What did you like best?
What did you like least?
If you got to decide
what would be the first
thing to improve?
• Do you feel that there is
anything missing?
Probe for content and
functions
•
•
•
•
• If you were to describe
it to a colleague in a
sentence or two, what
would you say?
• Final comments or
questions?
47
Analysingtheresults
Rettig,1994
• Print screen shots from the prototype and arrange
them on a big table
• Pile the notes near the relevant interface component
• Divide the labour of going through the piles to
summarise and prfioritize the problems
• Go through the piles and agree on suggested changes
48
Redesign
• Sketch down solutions and affix them directly on the
relevant part of the paper prototype
• Use that as a basis for creating a revised design
49
Tips
https://uxmag.com/articles/how-to-perform-your-own-lean-mobile-usability-testing
• Focus on human
connection
•
Empathy and practice
• Let them fail
•
Explain the task in as few
words as possible, and then
sit back and observe.
• Answer questions with
questions
•
User: ”What do I do next?”
•
You: ”What do you think?”
• Focus on behaviours,
not opinions
•
Behaviour is consistent,
opinions are not
• Ask valuable questions
•
How was your experience?
•
How was your experience
doing task X?
•
Which parts of the
experience stood out to
you?
50
Commonmistakes
• Not making pilot runs
• Not having specific
tasks with a clear start
and a clear end state
• No tasks written down
• Not varying the order of
tasks
• Not explaing how the
test works
• Telling the user what to
do when they hesitate
• Explaining or defending
the design
• Laughing, encouraging,
nodding, or expressing
other reactions to users
actions.
• Not having wellprepared pre- and posttest questions
• Not getting informed
consent
51
TestingwithChildren
hOp://www.microsoi.com/usability/UEPosLngs/p9-hanna.pdf
•
•
•
Make sure their parents know what
is going to happen and that they
have giving their permission.
Adapt to the child since
preschoolers have a short attention
span.
Let the children explore the device
to their own interests, instead of let
them perform a series of tasks.
•
Observe closely their behaviour.
•
Children this age often have
difficulty expressing their likes and
dislikes in words.
•
A session will last on average about
30 minutes, but schedule extra time
for play and exploration. Give
yourself plenty of time in between
children.
•
Engage small talk with the children,
to establish a relationship.
•
Tell the children about the system
and that it’s not finished yet. Tell
them why is is important to get their
feedback.
•
During the test it’s important to
keep the children’s attention. When
that’s not possible find a way to do
something different for a bit and
come back later to the test.
•
After testing, reward children by
commenting on how helpful they
were. Explain to them that all of
their hard work helped you with.
UsabilityReports
(usability.gov)
• Thera are templates and examples for usability report
contents
•
http://www.usability.gov/how-to-and-tools/resources/templates/
report-template-usability-test-long.html
•
http://www.ida.liu.se/~TDDD58/assignments/report1.pdf
•
http://www.ida.liu.se/~TDDD58/assignments/report2.pdf
• But use an academic report structure
•
http://www.ida.liu.se/edu/ugrad/thesis/instructions/
Exjobb_anvisning_150313.pdf
•
http://www.ida.liu.se/edu/ugrad/thesis/instructions/
Exjobb_instruction_eng_150313.pdf
52
53
• Academic report outline
•
Title page
•
Abstract
•
Introduction – what is the
problem/question and why
is it important?
•
(Background – what is the
context?)
•
Theory – what have others
done?
•
Method – what did you do
and why?
•
Results – what happened?
•
Discussion – what does it
mean?
•
Conclusions – what are the
take home points in
relation to problem/
questions and purpose
•
References
•
(Appendices)
• Chapters in parentheses
are optional
• Sometimes Background
and Theory is placed as
subheadings of
Introduction
• Sometimes Conclusions
are placed as last
subheading in Discussion
54
Insummary
•
•
•
•
•
•
•
•
Set up test task scenarios
Prototype enough but not more
Choose prototyping tool according to your needs
Make a pilot test
Introduce participants
Be clear on the roles when conducting the test
Debrief participants
Analyse results and work out consequences for
redesign
mattias.arvola@liu
@mattiasarvola
www.liu.se