by Xianghua Ding
Hoang Minh Ho Dac
1

2

Asking Users:
 Interviews
 Questionnaires
 Asking Experts:
 Heuristic evaluation
 Walkthroughs
• Cognitive
• Pluralistic

3
 Interviews
• Developing questions
• Planning interviews
• 4 Types of interviews
• Data analysis and interpretation
 Questionnaires

4
Developing Questions
 Avoid long questions
 Avoid compound sentences


“How do you like this cell phone compared with previous ones that you have owned?”
“How do you like this cell phone? Have you owned other cell phones? If so, how do you like it?”
 Avoid using jargon
 Avoid leading questions

“Why do you like this style of cell phone?”
 Be alert to unconscious bias

Planning an Interview - Steps





Introduction
 To introduce himself, explain the purpose and get consent
Warmup session
 Using easy, non-threatening questions
Main session
 To ask prepared questions from easy to difficult.
Cool-off period
 a few easy questions
Closing session
 To thank the interviewee and to clear up the scene


5
Make the interview as pleasant as possible
The golden rule is to be professional

6
Planning an Interview - Advice
 Dressing
 In a similar way to the interviewee if possible
 Prepare a consent form and ask the interviewee to sign in
 Make the equipment work
 Make sure your recorder works and know how to use it in advance .
 Record answers exactly

7
4 Types of Interviews
 Unstructured interviews
 Structured interviews
 Semi-structured interviews
 Group interviews

Unstructured Interviews


Both interviewer and interviewee have control and can steer the topic


More like a conversation
Focus on particular topic but go into depth
Questions are open


No predetermined content and format
Interviewee is free to answer questions as fully or as briefly as she wished
 Need a plan to make sure the main things to be covered
8

Unstructured Interviews-
Advantage & Disadvantage
Advantages
 Generate rich data
Disadvantages




Time consuming
Ethical issues
Impossible to replicate the process
Difficult to analyze all data
9

Structured Interviews
 The interviewer has the most control
 Pose predetermined , closed questions
 The study is standardized
 The same questions are used with each participants

Useful when study goal’s are clear and specific questions can be identified
10

Semi-structured Interviews




Combines features of structured and unstructured interview
Use both closed and open questions
Has a basic script for guidance
 so the same topics are covered
Probe are device to get more information

“Do you want to tell me anything else?”
 Be aware not to preempt an answer
 “You seem to like this color”
 Accommodate silence
 Prompt the person to help her along
11

Semi-structured Interviews -
An example





Which websites do you visit most frequently?
 <Answer several but stresses that se prefers uci.com>
And why do you like it?
 <Answer>
Tell me more about X?
 <Answer>
Anything else?
 < Answer>
Thanks. Are there any reasons that you haven’t mentioned?
12

Group Interviews
 Involve a small group guided by a interviewer to facilitate discussion
 Focus group



Normally 3-10 people are involved
Participants are representative of a certain type of users; they normally share certain kind of characteristics
Allows diverse and sensitive issues to be raised
13

Group Interviews –
Advantage & Disadvantage
Advantages





Method is readily understood
Findings appear believable
Low-cost
Quick results
Easily be scaled
Disadvantages


Facilitators need to be skillful
Difficult to get people together in a suitable location and time
14

Data Analysis & Interpretation
 Quantitatively
 For structured interviews
 Qualitatively
 For unstructured Interviews
 A coding form may be developed
 Comments may be clustered along themes and anonymous quotes used to illustrate points of interest
 Tools such as NUDIST
15


An alternative technique for getting users’ opinions
 Can have closed and open questions
Strengths :
 Distributed to a large number of people
 Provide evidence of a wide general opinion
16




Make questions and instructions clear
If possible, ask closed questions and offer a range of answers
Include a “no-opinion” for questions that seek opinions


General questions should precede specific ones
Group related items



Specify age as a range
Different versions for different population
Balance between white space and compactness


If scales are used, the range should not overlap
The ordering of scales should be intuitive and consistent
17

1. State your age in years
2. How long have you worked here?
(check one only)
1 yr
2 yrs
3 yrs
> 3 yrs
3. How long have you use the
Internet? (Check one only)
< 1 yr
1-3 yrs
3-5 yrs
> 5 yrs
4. Do you use the Web to:
Purchase goods
Send email
Visit chatrooms
Find information
5. How useful is the Internet to you?
18

 Checkbox
 Used for demographic or background data
 Respondents check an appropriate box or circle a response
 Ranges
 used for getting opinions
 2 types
1. Likert Scales
2. Semantic Differential Scales
19

Instruction:
In the following questions, 1 represents strongly agree and 5 represents strongly disagree. Please check only one.
 The company website is helpful:
1 2 3 4 5
 The website color is annoying:
1 2 3 4 5
 Should we mix positive questions with negative questions?
 What is the best rating scale ? (e.g. odd like 1-3, 15,… or even)
20

Instruction: for each pair of adjectives, place a cross at the point between them that reflects the extent to which you believe the adjectives describe the home page. You should place only one cross between the marks on each line.
 Attractive
 Clear
 Helpful
|_|_|_|_|_|_|_| Ugly
|_|_|_|_|_|_|_| Confusing
|_|_|_|_|_|_|_| Unhelpful
21
 How to calculate the total score?

Two important issues
1. How to reach a representative sample of people
2. How to ensure a reasonable response rate
 With small number fewer than 20, 100% is often achieved
 With larger populations, 40% return is generally acceptable
Solutions
•
Tell people it is OK to complete just a part
•
Include stamped, self-addressed envelope
•
Explained why you need the questionnaires
•
Assure anonymity
•
Contact users
•
Offer incentives
22

23
 Email
 Can be targeted to specific users


Quick response
Limited to text
 Web-based







Flexible graphical design
Errors could be corrected easily
Immediate data validation
Less time for data analysis
Low cost for copying and postage
Have random samples of respondents
Response rate may be lower than paper form

24
1.
2.
3.
Devise the questionnaire on paper, following the guidelines
Identify a random sample of population. Avoid biased or convenience sampling




Turning the paper questionnaire into a webbased version error-free interactive
Accessible and readable from all online users
Identification information handled confidentially
User-test before distributing

• What do you think about the questionnaires at this website? Are they good or bad? Why?
http://www.perseusdevelopment.com/surveytips/sampl esurveys.html
25

26
 Display data graphically (e.g. bar charts)
 Often simple statistics are needed (number of participants, percentage of responses…)
 Identify any trends, patterns or relationship between responses

27

28

29

Asking Expert
 Heuristic evaluation




Introduction
Core heuristics
Doing heuristic evaluation
Heuristic evaluation for web sites
• An example
• Guidelines of website
 Walk through


Cognitive Walkthrough
Pluralistic walkthrough
30

Heuristic Evaluation –
Introduction
 An inspection technique in which experts evaluate whether user-interface elements conform to a set of heuristics
 Closely related to design guidelines
 Different sets of heuristics for different products
31

Core Heuristics

Nielsen’s










Visibility of system status
Match between the system and the real world
User control and freedon
Consistency and standards
Help user recognized, diagnose, and recover from errors
Error prevention
Recognition rather than recall
Flexibility and efficiency to use
Aesthetic and minimalist design
Help and documentation
32

Doing Heuristic Evaluation






Briefing session
Experts are told what to do
A prepared script is useful as a guide
Ensure each person receives the same briefing


Evaluation Period
Experts independently inspecting the product, using heuristics for guidance.
•
•
At lease 2 passes
Give a feel of the flow of the interaction and the product scope
Focus on specific interface elements

Debriefing session
Discuss their findings, prioritize the problems, and suggest the solutions
33

Heuristic Evaluation for
Websites – an Example
 http://www.lib.uci.edu/
34

Heuristic Evaluation for
Websites – an Example

Heuristics tailored from Nielsen’s original set


Internal consistency
• Is the logo, format, text , font or usage of terms consistent?
Minimizing the user’s memory load


Layout
• Is it compact? Is the page layout meaningful? Is there too much text on the page?
……..
35

Heuristic Evaluation for
Websites – an Example
 Findings about the websites



The formatting of pages and presentation of logos are consistent of website
Some forms require users to recall instead of recognition
• http://www.lib.uci.edu/services/workshops/isform.
html
The layout is kind of complicated, kind of too much text
• http://www.lib.uci.edu/online/ebooks.html
36

Heuristic Evaluation for
Websites - Guidelines
37
 Navigation





Avoid orphan pages that are not connected to the home page, which lead users into dead ends
Avoid long pages with excessive white space that force scrolling
Provide navigation support
Avoid non-standard link colors
Provide consistent look and feel
 Access


Avoid complex URLs
Avoid long download time that annoys users
 Information design




An alternative to heuristic evaluation
To predict users’ problems without doing user testing
How ? - Walk through a task with the system and recognize usability problems
 2 types:
1. Cognitive walkthroughs
• No user participation
2. Pluralistic walkthroughs
•
Users, experts, specialists, designers, developers… are involved
38

1.
Users’characteristics are identified
2.
The walkthrough team come together for task analysis
3.
The walkthrough team go through each task, answering
•
•
• the three questions :
Will users know what to do?
Will users see how to do it?
Will users understand from the feedback whether their action are correct or not?
4.
Record what cause problems, why and how serious they are to users
5. Revise the design to fix the problems
39

What to Evaluate: Java Home Page
Task : to find the tutorial for Java 3D
Typical users : students, developers
Go through each step to complete the task
1.
At Java Home Page
 Will users know what to do ? – yes, to search the Tutorial Section
 Will users see how to do it ? - yes, click on the left submenu Tutorial
 Will users understand the feedback ? – yes, it leads to the List of
Tutorials
2.
At The List of Tutorials
…
40

1.
Choose a task to evaluate.
Take a series of screenshots to complete that task.
2.
Each member of the team looks at the screen pictures and writes down the sequence of actions they would take to move from one screen to another
3.
First, users present their suggest of actions
Next, experts present their findings
Last, developers comment
4.
Go back to step 1 with another task
41

Suppose you are a member in the walkthrough team.
Purpose: to evaluate the web site of Sea World
Step 1
 Chosen task: to find the location and open hours
 Scenarios:
There are 2 screens in the path to find Sea World’s location and open hours
42

1
Step 2:
•
You are shown the above 2 screens in the path to complete the task
•
Can you say what action will lead you from the first screen to the second screen? (don’t consult with other members in the team)
2
Step 3:
•
•
Do other members in the team agree with you?
If yes, what do you think about the interface? If no?

Cognitive
•
Focus on users’ problems in detail
•
• No user involved in evaluation.
Do not need a working prototype
•
Time-consuming
•
Has narrow focus, only useful for certain types of systems
Pluralistic
Strong focus on users’ tasks
•
Multidisciplinary evaluators with user participation in evaluation
•
Difficult to arrange time and
• location for the walkthrough team
Time-consuming
•
Explore only a limited number of tasks
44

45