Overview of User Interface Design,
Prototyping, and Evaluation
(or, All of HCI in 80 Minutes)
Jason I. Hong
January 18, 2007
User Interface Hall of Fame or Shame?
•
•
IE5 page setup for printing
Problems
– codes for header & footer
information
• requires recall!
• recognition over recall
• no equivalent GUI
– help is the way to find out,
but not obvious
Human-Computer Interaction (HCI)
Human
•
– the end-user of a program
– the others in the organization
•
Computer
– the machine the program runs on
– clients & servers, PDAs, cars, microwaves
•
Interaction
– the user tells the computer what they want
– the computer communicates results
(input)
(output)
HCI Approach to UI Design
Tasks
Organizational &
Social Issues
Design
Technology
•
Humans
Other considerations we won’t look at
– Business models, level of fun
Why is HCI Important?
•
•
Major part of work for “real” programs (~50%)
Bad user interfaces cost:
– money (reduced profits, call centers)
• WiFi Alliance: 30% of WiFi boxes returned
– reputation of organization (e.g., brand loyalty)
– time (wasted effort and energy by users, rework)
– lives (Therac-25)
Why is HCI Important?
•
Privacy and Security
–
–
–
–
•
phishing scams
accidental disclosures (ex. location info, cookies)
difficulty diagnosing the situation (intrusion detection)
intentionally circumventing security mechanisms
User interfaces hard to get right
– people are unpredictable
– intuition of designers often wrong
– need good design methods
Four Myths about Good Design
•
Myth 1: Only experts create good designs
– experts faster, simple and effective techniques anyone can apply
•
Myth 2: We can fix the user interface at the end
– good design is more than just user interface
– having right features, building those features right
•
Myth 3: Good design takes too long / costs too much
– simple and effective techniques that can reduce total
development time & cost (finds problems early on)
•
Myth 4: Good design is just cool graphics
– graphics part of bigger picture of what to communicate & how
Outline
•
Overview of the Design Process
– Design
– Prototyping
– Evaluation
•
User Models
– Affordances
– Mappings
– Metaphors
Who Builds User Interfaces?
•
A team of specialists (ideally)
–
–
–
–
–
–
–
–
–
graphic designers
interaction / interface designers
information architects
technical writers
marketers
test engineers
usability engineers
software engineers
users
Design
Prototype
Evaluate
Design
•
Design is driven by requirements
– focus on the core needs, not how implemented
– e.g., Nokia N80 not as important as “mobile” app
• might be multiple ways of achieving your goals
•
A design is a simplified representation of the
desired artifact
– text description of tasks
– screen sketches or storyboards
– flow diagrams / outline showing
task structure
– executable prototypes
Write essay
start word processor
write outline
fill out outline
Start word processor
find word processor icon
double click on icon
Write outline
write down high-level ideas
.
.
.
Web Design Representations
Site Maps
Schematics
Storyboards
Mock-ups
Usability Goals?
According to the ISO:
The effectiveness, efficiency, and satisfaction with
which specified users achieve specified goals in
particular environments
•
This does not mean you have to create a “dry”
design or something that is only good for novices –
it all depends on your goals
Usability Goals
•
•
•
Set goals early and use them to measure progress
Goals often have tradeoffs, so prioritize
Example goals:
– Learnable
• faster the 2nd time & so on
– Memorable
• from session to session
– Flexible
• multiple ways to accomplish tasks
– Efficient
• perform tasks quickly
– Robust
• minimal error rates
• good feedback so user
can recover
– Pleasing
• high user satisfaction
– Fun
•
Cognitive abilities
– perception
– physical manipulation
– memory
•
Organizational / job abilities
– what skills required
– who they talk to
•
You already know too much
•
You already know too much
– easy to think of self as typical user
– easy to make mistaken assumptions
•
Keep users involved throughout the design
– understanding work process
– getting constant feedback
•
User-centered design mind-set
– thinking of the world in users’ terms (empathy)
– not technology-centered / feature driven,
think of benefit to users
Outline
•
Overview of the Design Process
– Design
– Prototyping
– Evaluation
•
User Models
– Affordances
– Mappings
– Metaphors
Example of Design Failure
•
BART “Charge-a-Ticket” Machines
– allow riders to buy BART tickets or add fare
– takes ATM cards, credit cards, & cash
Example of Design Failure
•
BART “Charge-a-Ticket” Machines
– allow riders to buy BART tickets or add fare
– takes ATM cards, credit cards, & cash
•
Problems (?)
– no visual flow (Where do I start? Where do I go next?)
– one “path” of operation
• ticket type -> payment type -> payment -> ticket
– BART Plus has minimum of $28, no indication of this
until after inserting >= $1
•
can’t switch to regular BART ticket
– too many instructions
– large dismiss transaction button does nothing
Lessons from the BART machine
•
Can’t we just define “good” interfaces?
– “good” has to be taken in context of users
• might be acceptable for office work, not for play
• infinite variety of tasks and users
– guidelines are too vague to be generative
• e.g., “give adequate feedback”
•
How can we avoid similar results?
– “What is required to perform the user’s tasks?”
Task Analysis
•
Find out:
– who users are
– what tasks they need to perform
•
Observe existing work practices
Create scenarios of actual use
•
This lets us try new ideas before building software!
•
– Get rid of problems early in the design process
while they are still cheap to fix!
Task Analysis Questions
•
•
•
•
•
•
Who is going to use the system?
What tasks do they now perform?
What tasks are desired?
How are the tasks learned?
Where are the tasks performed?
What’s the relationship between user & data?
Task
Analysis
Task Analysis Questions (cont.)
•
•
•
•
•
What other tools does the user have?
How do users communicate with each other?
How often are the tasks performed?
What are the time constraints on the tasks?
What happens when things go wrong?
Who?
•
Identity
– in-house or specific customer is easy
– need several prototypical users for broad product
•
•
•
•
Background
Skills
Work habits and preferences
Physical characteristics
Who (BART)?
•
Identity?
– people who ride BART
• business people, students, disabled, elderly, tourists
•
Background?
– may have an ATM or credit card
– have used other fare machines before
•
Skills?
– may know how to put cards into ATM
– know how to buy BART tickets
Who (BART cont.)?
•
Work habits and preferences?
– Some people: use BART 5 days a week
– Others: first time use
•
Physical characteristics?
– varying heights -> don’t make it too high or too low!
What Tasks?
•
Important for both automation and new functionality
Understand relative importance of tasks
•
Example: on-line billing
•
– small dentists office installed new billing system
– assistants became unhappy with new system
– old forms contained hand-written margin notes
• e.g., patient A’s insurance takes longer than most
Where is the Task Performed?
•
•
•
•
•
•
•
•
Office, laboratory, point of sale?
Effects of environment on users?
Users under stress?
Confidentiality required?
Wet, dirty, or slippery hands?
Soft drinks?
Lighting?
Noise?
What is the Relationship Between Users & Data?
•
Public data?
– Open government records, public web sites
•
Personal data?
– Ex. health records, bank records
– always accessed at same machine?
– do users move between machines?
•
Common data?
– used concurrently?
– passed sequentially between users?
•
•
Remote access required?
Access to data restricted?
What Other Tools Does the User Have?
•
More than just compatibility
How user works with collection of tools
to get things done
•
Example: automating lab data collection
•
–
–
–
–
–
how is data collected now?
by what instruments and manual procedures?
how is the information analyzed?
are the results transcribed for records or publication?
what media/forms are used and how are they handled?
How Often Do Users Perform the Tasks?
•
•
Frequent users remember more details
Infrequent users may need more help
– even for simple operations
– make these tasks possible to do
•
Which function is performed
– most frequently?
– by which users?
– optimizing system for these tasks will improve
perception of good performance
Involve Users to Answer
Task Analysis Questions
•
Users help designers learn:
– what they do and how they do it
•
Developers reveal technical capabilities
– builds rapport and ideas of what is possible
– users can comment on whether ideas make sense
•
How do we do this?
– observe & interview prospective users in work place!
•
Heated arguments with others on design team
– Over what tools, skills, and knowledge users have
– These tend to generate lots of heat, little light
•
Go out to real users and get real data from them
– find out what they really do
– how would your system fit in
•
Are they too busy?
– buy their time (t-shirts, coffee mugs)
– find substitutes (medical students)
Can’t we just ask users what they want?
•
•
Not familiar with what is possible with technology
Not familiar with design constraints
– Budget, legacy code, time, etc
•
•
•
Not familiar with good design
Not familiar with security and privacy
Sometimes users don’t know what they want
– Ex. Remote controls
•
Contextual inquiry is an important method for
understanding users’ needs
– Also, attitude vs actual behavior
Contextual Inquiry
•
•
•
Go to the workplace & see the work as it unfolds
People summarize, but we want details
Keep it concrete when people start to abstract
– “We usually get reports by email”, ask “Can I see one?”
Contextual Inquiry
•
•
Facts are only the starting point, you want a
design based on correct interpretations
Validate & rephrase
– share interpretations to check your reasoning
• Ex. “So accountability means a paper trail?”
– people will be uncomfortable until the phrasing is right
Conducting a Contextual Inquiry
•
Use recording technologies
– notebooks, tape recorders, still & video cameras
•
Structure
– conventional interview (15 minutes)
• introduce focus & deal with ethical issues
• get used to each other by getting summary data
– transition (30 seconds)
• state new rules – they work while you watch & interrupt
– contextual interview (1-2 hours)
• take notes, draw, be nosy! (“who was on the phone?”)
– wrap-up (15 minutes)
• summarize your notes & confirm what is important
Selecting Tasks
•
•
Choose real tasks users face
Have a mixture of simple & complex tasks
– easy tasks (common or introductory)
– moderate tasks
– difficult tasks (infrequent or for power users)
•
Good tasks are fundamental to good usability
– does your system support key tasks desired by users?
– how well does your system support these?
– we’ll see more of tasks in evaluation
Using Tasks in Design
•
Write up a description of tasks (informal ok)
– tells a cohesive and plausible story
– run by users and rest of the design team
• do they make sense?
•
Note that task descriptions focus on what, not how
Manny is in the city at a club and would like to call
his girlfriend, Sherry, to see when she will be arriving
at the club. She called from a friend’s house while
he was driving, so he couldn’t answer the phone. He
would like to check his missed calls and find the
number so that he can call her back.
Using Tasks in Design (contd.)
•
Okay to have some freeform tasks
– Ex: “purchase tickets for a movie you want to see”
• navigation, reviews, shopping cart, etc
– specific tasks good for understanding usability,
freeform tasks good for understanding usefulness
Outline
•
Overview of the Design Process
– Design
– Prototyping
– Evaluation
•
User Models
– Affordances
– Mappings
– Metaphors
Why Do We Prototype?
•
•
Quickly experiment with alternative designs
Get feedback on your design faster
– fix problems before code is written
– saves time and money
•
Keeps the design centered on the user
– must test & observe ideas with users
Fidelity in Prototyping
•
•
Fidelity refers to level of detail
High fidelity
– prototype looks like the final
product
•
Low fidelity
– artist’s rendition with many
details missing
Low-fi Sketches & Storyboards
Low-fi Sketches & Storyboards
Ink Chat
Why Use Low-fi Prototypes?
•
Traditional methods take too long
– sketches  build prototype  evaluate  iterate
– don’t want to program for weeks or months before feedback
•
Simulate the prototype
– sketches  evaluate  iterate
– sketches act as prototypes
• designer “plays computer”
• other design team members observe & record
•
Kindergarten implementation skills
– allows non-programmers to participate
– helps make sure everyone on the team is together
Qualities of Lo-Fi Prototypes
Informal visual representation
•
–
–
–
–
•
communicates “unfinished”
encourages creativity
faster to create
higher-level feedback
•
Formal visual representation
– communicates “finished”
– inhibits creativity (detailing)
– slower to create
Advice: avoid high-fidelity tools until necessary
The Basic Materials
•
•
•
•
•
•
•
Large, heavy, white paper (11 x 17)
5x8 in. index cards
Post-its
Tape, stick glue, correction tape
Pens & markers (many colors & sizes)
Overhead transparencies
Scissors, X-acto knives, etc.
ESP
Constructing the Model
Constructing the Paper Prototype
•
Set a deadline
– a few hours or 1-2 days
– don’t think for too long - build it!
•
•
Draw a window frame on large paper
Put different screen regions on cards
– anything that moves, changes, appears/disappears
•
Ready response for any user action
– e.g., have those pull-down menus already made
•
Use photocopier to make many versions
Advantages of Low-fi Prototyping
•
Takes only a few hours
– no expensive equipment needed
•
Can test multiple alternatives
– fast iterations
• number of iterations is tied to final quality
•
Almost all interaction can be faked
Outline
•
Overview of the Design Process
– Design
– Prototyping
– Evaluation
• A lot here applies to evaluating both
paper prototypes and real systems
•
User Models
– Affordances
– Mappings
– Metaphors
Preparing for a Test
•
Select people representative of target users
– minimize use of friends or family
– job-specific vocab / knowledge
– approximate if needed
• medical students, engineering students
– use incentives to get participants
•
Prepare tasks that are:
– typical of the product during actual use
– ideally, same defined tasks from beforehand
•
Practice to avoid “bugs”
Conducting a Test
•
Roles:
– greeter – puts users at ease & gets demographic data
– facilitator – only team member who speaks
• gives instructions & encourages thoughts, opinions
– computer – simulates response, w/o explanation
– observers – take notes & recommendations
•
Typical session is 1 hour
– preparation, the test, debriefing
•
Make a recording or take good notes
Instructions to Participants
•
Describe the purpose of the evaluation
– “I’m testing the product; I’m not testing you”
•
•
•
Tell them they can quit at any time
Demonstrate any equipment
Explain how to think aloud
– tell us what they are trying to do
– tell us questions that arise as they work
– tell us things they read
•
•
•
Explain that you will not provide help
Explain the basic concept of the UI, but not too much
Describe the task
– give written instructions, one task at a time
Conducting a Test
•
Do a debriefing at the end
– Were tasks realistic?
– What parts made sense? Confusing?
– Any features missing?
Ethical Considerations
•
Sometimes tests can be distressing
– users have left in tears
•
You have a responsibility to alleviate
–
–
–
–
–
•
make voluntary with informed consent
avoid pressure to participate
let them know they can stop at any time
stress that you are testing the system, not them
make collected data as anonymous as possible
Often must get human subjects approval (IRB)
Deciding on Data to Collect
•
Two types of data
– process data
• observations of what users are doing & thinking
– bottom-line data
• summary of what happened (time, errors, success)
• independent and dependent variables
Which Type of Data to Collect?
•
Always focus on process data first
– gives good overview of where problems are
•
Bottom-line doesn’t tell you where to fix
– just says: “too slow”, “too many errors”, etc.
•
Hard to get reliable bottom-line results
– need many users for statistical significance
Using the Test Results
•
Sort & prioritize observations
– what was important?
– lots of problems in the same area?
•
Summarize the data
– make a list of all critical incidents, positive & negative
– try to judge why each difficulty occurred
•
What does data tell you?
– UI work the way you thought it would?
– missing features?
Using the Results (cont.)
•
Update task analysis & rethink design
– rate severity & ease of fixing critical incidents
– fix both severe problems & make the easy fixes
•
Will thinking aloud give the right answers?
– not always
– if you ask a question, people will always give an answer,
even it is has nothing to do with facts
• panty hose example
– try to avoid specific questions
Measuring Bottom-Line Usability
•
Situations in which numbers are useful
– time on task
– # tasks successfully completed
– # errors
• define in advance what these mean
•
Do not combine with thinking-aloud. Why?
– talking can affect speed & accuracy
Measuring Subjective User Preference
•
Can rate system on a Likert scale
– Example: The user interface was easy to use
•
1 - Strongly disagree
•
2 - Disagree
•
3 - Neither agree nor disagree
•
4 - Agree
•
5 - Strongly agree
– Can be hard to be sure what data means
•
novelty of UI, not realistic setting …
•
Can also get some useful data by asking
– what they liked, disliked, where they had trouble, best part,
worst part, etc. (redundant questions are OK)
Comparing Two Alternatives
•
Between groups experiment
– two groups of test users
– each group uses only 1 of the systems
•
Within groups experiment
A
– one group of test users
• each person uses both systems
• can’t use the same tasks or order (learning)
– best for low-level interaction techniques
•
•
Between groups requires many more participants
than within groups
See if differences are statistically significant
– assumes normal distribution & same std. dev.
B
Reporting the Results
•
•
•
Report what you did & what happened
Images & graphs help people get it!
Video clips can be quite convincing
– Especially for convincing other engineers
Discount Usability
•
Reaction to “expensive” user tests
– Meant to be cheap, fast, effective
•
Walkthroughs
– put yourself in the shoes of a user
– like a code walkthrough
•
•
•
•
Low-fi prototyping
Action analysis (GOMS)
On-line, remote usability tests
Heuristic evaluation
Outline
•
Overview of the Design Process
– Design
– Prototyping
– Evaluation
•
User Models
– Affordances
– Mappings
– Metaphors
Design of Everyday Things
•
By Don Norman (UCSD, Apple, HP, NN Group)
Design of everyday objects illustrates problems
faced by designers of systems
•
Explains conceptual models
•
– doors, washing machines,
digital watches, telephones, ...
•
Resulting design guides
-> Highly recommend this book
Conceptual Models
•
•
Mental representation of how object works
and how interface controls affect it
People may have preconceived models
that are hard to change
– (4 + 5) vs. (4 5 +)
– dragging to trash?
• delete file but eject disk
•
Interface must communicate model
– visually
– online help and documentation can help,
but shouldn’t be necessary
Affordances as Perceptual Clues
•
Well-designed objects have affordances
– clues to their operation
– often visual, but not always (e.g., speech)
Affordances as Perceptual Clues
Siemens Pocket PC Phone
Pen input, no keypad
Handspring Treo
Pen input/keypad input
Affordances as Perceptual Clues
•
Poorly-designed objects
– no clues or misleading clues
French artist Jacques Carelman
Crazy design for a screw punch!
Refrigerator
freezer
fresh food
Problem: freezer too cold, but fresh food just right
Refrigerator Controls
Normal Settings
Colder Fresh Food
Coldest Fresh Food
Colder Freezer
Warmer Fresh Food
OFF (both)
A B C D E
C and 5
C and 6-7
B and 8-9
D and 7-8
C and 4-1
0
7
6
5 4
3
What is your conceptual model?
A Common Conceptual Model
A B C D E
7
6
5 4
cooling
unit
3
cooling
unit
independent controls
Actual Conceptual Model
A B C D E
cooling
unit
7
•
•
6
5 4
3
Now can you fix the problem?
Possible solutions
–
–
make controls map to user’s model
make controls map to actual system
Design Model & User Model
Design Model
User Model
System Image
•
•
Users get model from experience & usage
– through system image
What if the two models don’t match?
Conceptual Model Mismatch
•
Mismatch between designer’s and user’s
conceptual model leads to…
– Slow performance
– Errors
• And inability to recover
– Frustration
– ...
Notorious Example
A Personal Example
Make Things Visible
•
Refrigerator (?)
– make the A..E dial something about percentage of
cooling between the two compartments?
•
Controls available on watch w/ 3 buttons?
– too many and they are not visible!
•
Compare to controls on simple car radio
– #controls = #functions
– controls are labeled (?) and grouped together
Map Interface Controls
•
•
Control should mirror real-world
Which is better for dashboard speaker
front / back control?
Dashboard
Map Interface Controls
Map Interface Controls
Four Key Ideas
Summary
•
Overview of the Design Process
– Design
– Prototyping
– Evaluation
•
User Models
– Affordances
– Mappings
– Metaphors
(Task Analysis, Contextual Inquiry, Tasks)
(Paper prototyping)
(Lab tests, Heuristic evaluation)
Using the Data You Learn
•
Say who the users are (use personas or profiles)
Example Persona
Name:
Patricia
Age:
31
Occupation:
Sales Manager, IKEA Store
Hobbies:
Painting
Fitness/biking
Taking son Devon to the park
Likes:
Emailing friends & family
Surprises for her husband
Talking on cell phone with friends
Top 40 radio stations
Eating Thai food
Going to sleep late
Dislikes:
Slow service at checkout lines
Smokers
Using the Data You Learn
•
Say who the users are (use personas or profiles)
– personas do not have to be a real person, but should be
based on real facts and details
– design can really differ depending on who the target is
– provide names (easier to reference)
– characteristics of the users (job, expertise, etc.)
•
Might have one persona for each class of users
– helps the design team think in terms of the users
•
Keep in mind we already use personas
– “I wouldn’t like that”
– “My mom wouldn’t be able to use that”
1 Minute Break
•
Good design matters in all
areas of our lives
•
The little things really do
matter
•
A designer’s proposed
changes to airport
screenings
Metaphor
•
Definition ?
– “The transference of the relation between
one set of objects to another set for the purpose
of brief explanation.”
•
Lakoff & Johnson, Metaphors We Live By
– “...the way we think, what we experience, and
what we do every day is very much a matter of
metaphor.”
– in our language & thinking - “argument is war”
• he attacked every weak point
... criticisms right on target
... if you use that strategy
•
We can use metaphors to leverage existing
conceptual models
Desktop Metaphor
•
Suggests a conceptual model
– Not really an attempt to simulate a
real desktop
– Leverages existing knowledge
about files, folders, trash
– A way to explain why some
windows seemed blocked
•
The “user illusion”
Example Metaphors
•
Global metaphors
– personal assistant, e-wallet, e-mail, digital library
•
Data & function
– rolodex, to-do list, calendar, documents
•
Collections
– drawers, files, books, newspapers, photo albums
What Happens When Things Go Wrong?
•
How do people deal with
– task-related errors?
– practical difficulties?
– catastrophes?
•
Is there a backup strategy?
Discount Usability Engineering
•
Reaction to excuses for not doing user testing
– “too expensive”, “takes too long”, …
•
Cheap
– no special labs or equipment needed
– the more careful you are, the better it gets
•
Fast
– on order of 1 day to apply
– standard usability testing may take a week or more
•
Easy to use
– some techniques can be taught in 2-4 hours
Example Heuristic
•
H2-8: Aesthetic and minimalist design
– no irrelevant information in dialogues
Example Heuristic
•
H2-9: Help users recognize, diagnose, and
recover from errors
– error messages in plain language
– precisely indicate the problem
– constructively suggest a solution