Lecture 2:
Discovering what people can't tell you:
Contextual Inquiry and Analysis
Methodology
Brad Myers
05-863 / 08-763 / 46-863: Introduction to
Human Computer Interaction for
Technology Executives
Fall, 2012, Mini 2
© 2012 - Brad Myers
1
Happy Halloween!

Take 2 candies!
© 2012 - Brad Myers
2
Resolve Devices for Assignments

On the GoogleDoc
© 2012 - Brad Myers
3
Some Usability Methods














 Questionnaires
Contextual Inquiry
Contextual Analysis (Design)  Surveys
 Interaction Relabeling
Paper prototypes
 Log analysis
Think-aloud protocols
 Focus groups
Heuristic Evaluation
 Card sorting
Affinity diagrams (WAAD)
 Diary studies
Personas
 Improvisation
Wizard of Oz
 Use cases
Task analysis
 Scenarios
Cognitive Walkthrough
 Cognitive Dimensions
KLM and GOMS (CogTool)
 “Speed Dating”
Video prototyping
 …
Body storming
Expert interviews
© 2012 - Brad Myers
4
Contextual Inquiry and Analysis/Design

One method for organizing the development process
We teach it to our MS and BS students
Seems to be very successful

Hartson-Pyla text: Chapters 3-6




(doing things in a different order than text)
Also described in this classic book:


H. Beyer and K. Holtzblatt. 1998. Contextual Design: Defining
Customer-Centered Systems. San Francisco, CA:Morgan
Kaufmann Publishers, Inc. ISBN: 1558604111.
http://www.incent.com/
© 2012 - Brad Myers
5
User Research Methods
& the different fields they come from





Questionnaires, Interviews
 Social Psychology
Focus Groups
 Business, marketing technique
Laboratory studies
 Experimental Psychology
Think-aloud protocols
 Cognitive Psychology
Participant/observer ethnographic studies
 Anthropology
© 2012 - Brad Myers
6
Contextual Inquiry & Analysis/Design

Contextual Inquiry


A kind of “ethnographic” or “participatory design”
method
Combines aspects of other methods:


Interviewing, think-aloud protocols,
participant/observer in the context of the work
Afterwards: Contextual Analysis (HartsonPyla term)


Beyer-Holtzblatt call it “Contextual Design”
Also includes diagrams (“models”) to describe
results
© 2012 - Brad Myers
7
“Contextual Inquiry”



Interpretive field research method
Depends on conversations with users in the
context of their work
Used to define requirements, plans and
designs.


Discover the real requirements of the work
Drives the creative process:


In original design
In considering new features or functionality
© 2012 - Brad Myers
8
Context


Definition:
 “The interrelated conditions within which
something occurs or exists”
Understand work in its natural environment
 Go to the user
 Observe real work
 Use real examples and artifacts

“Artifact”: An object created by human workmanship
Interview while she/he is working
 More reliable than asking them
Context exists even when not a “work” activity




Use “work” here just to mean “doing something”
Can be home, entertainment, etc.
© 2012 - Brad Myers
9
Elements of User's Context: Pay
Attention to all of these









User's work space
User's work
User’s workarounds
User's work intentions
User's words (language used)
Tools used
How people work together
Business goals
Organizational and cultural structure
© 2012 - Brad Myers
10
Why Context?

Design complete work process



Integration!


Fits into “fabric” of entire operations
Not just “point solutions” to specific problems
Consistency, effectiveness, efficiency, coherent
Design from data


Not just opinions, negotiation
Not just a list of features
© 2012 - Brad Myers
11
Key distinctions about CIs
Interviews, Surveys, Focus
Groups
Contextual Inquiry
Summary data & abstractions
Ongoing experience &
concrete data
What customers say
What users do
Subjective
Objective
Limited by reliability of human
memory
Spontaneous, as it happens
What customers think they want
What users actually need
© 2012 - Brad Myers
12
Who?

Users



Between 6 – 20
Representative of different roles
Note: may not be people who will be doing the
purchasing of the system


E.g., if for an enterprise; public kiosk
Interviewers: “Cross-functional” team





Designers
UI specialists
Product managers
Marketing
Technical people
© 2012 - Brad Myers
13
Where?

Design is a group activity

Shared across different groups

Useful to have a designated, long-term space
for the project team

Interviews at user site
© 2012 - Brad Myers
14
Partnership




Definition:
 A relationship characterized by close cooperation
Build an equitable relationship with the user
Suspend your assumptions and beliefs
Invite the user into the inquiry process
© 2012 - Brad Myers
15
Why is Partnership Important?






Information is obtained through a dialog
The user is the expert.
Not a conventional interview or consultant relationship
Alternative way to view the relationship:
Master/Apprentice
The user is the “master craftsman” at his/her work
You are the apprentice trying to learn
© 2012 - Brad Myers
16
Establishing Partnership





Share control
Use open-ended questions that invite users to talk:
 "What are you doing?"
 "Is that what you expect?"
 "Why are you doing...?"
Let the user lead the conversation
Listen!
Pay attention to communication that is non-verbal
© 2012 - Brad Myers
17
Standard Contextual Inquiry:
Work-based Interview
Use when:
 Product or process already exists
 Or a near competitor’s
 User is able to complete a task while you observe
 Work can be interrupted
© 2012 - Brad Myers
18
Reasons for variation on the
standard work-based interview

Different goals




Designing a known product
 Know the competition
Addressing a new work domain
 Study what replacing
Designing for a new technology
Types of tasks that make work-based inquiry
impractical



Intermittent – instrument or keep logs
Uninterruptible – video and review later
Extremely long – point sample and review
© 2012 - Brad Myers
19
Some Alternative Contextual
Inquiry Interview Methods

For intermittent tasks
 In-context cued recall
 Activity logs

For uninterruptible tasks
 Post-observation inquiry
For extremely long or multi-person tasks
 Artifact walkthrough
New technology within current work
 Future Scenario
Prototype or prior version exists
 Prototype/Test drive



© 2012 - Brad Myers
20
Interview Recording and Note-Taking

Do record interview



Video recordings
Screen capture software with laptop microphone for user
When to take notes?



Note taking can help you pay closer attention
Notes lead to faster turn-around
Do not let it interfere with interviewing


Usually would use a second person
How to record?




What the user says – in quotes
What the user does – plain text
Your interpretation – in parentheses
© 2012 - Brad Myers
Write fast!
21
Analysis


In the moment:
Simultaneous data collection and analysis during
interview
Post interview:


Using notes, tapes, and transcripts
Analysis by a group:





Integrates multiple perspectives
Creates shared vision
Creates shared focus
Builds teams
Saves time
© 2012 - Brad Myers
22
Defining the Tasks

In a real Contextual Inquiry, user decides the
tasks


But you still must decide the focus



Investigate real-world tasks, needs, context
What tasks you want to observe
That are relevant to your product plan
But for Assignment 1, you will have to invent
some tasks
© 2012 - Brad Myers
23
Test Tasks


Task design is difficult part of usability testing
Representative of “real” tasks



Appropriate difficulty and coverage




Should last about 2 min. for expert, less than 30 min. for novice
Short enough to be finished, but not trivial
Tasks not humorous, frivolous, or offensive
Easy task first, progressively harder


Sufficiently realistic and compelling so users are motivated to
finish
Can let users create their own tasks if relevant
But better if independent
Remember: Not asking their opinions
© 2012 - Brad Myers
24
Initial Questions for the Users

Find out the context through initial questions




When would you normally do this kind of task?
Who would be involved in making the decisions?
What would influence any decisions?
How would you know what to do?

What information would you use to help decide?
© 2012 - Brad Myers
25
Test Script

Useful to have a script



Should read instructions out loud


Make sure say everything you want
Make sure all users get same instructions
Ask if users have any questions
Make sure instructions provide goals only in a general
way, and doesn’t give away information


Describe the result and not the steps
Avoid product names and technical terms that appear on the
web site


Don’t give away the vocabulary
Example:

“The clock should have the right time”;
not: “Use the hours and minutes buttons to set the time”
26
© 2012 - Brad Myers
Example of CI

Video of sample session with a eCommerce site:
http://www.cs.cmu.edu/~bam/uicourse/EHCIcontexualinquiry.mpg

Issues to observe




Interview of work in progress, in “context”
Actual session of doing a task
 Not an interview asking about possible tasks, etc.
 Note that focusing on expert behavior & breakdowns
Questions to clarify about routine, motivations
 Why do certain actions: need intent for actions
 Notice problems (“breakdowns”)
Notice what happens that causes users to do something
(“triggers”)
 E.g. appearance of error messages, other feedback, external
events (phone ringing), etc.
© 2012 - Brad Myers
27
Screen shots of important points in video
http://www.cs.cmu.edu/~bam/uicourse/EHCIcontexualinquiryScreens.ppt
© 2012 - Brad Myers
28