Usability and Eye Tracking
Marco Pretorius
Usability Manager & Researcher
UNISA: School of Computing
Agenda
• Introduction
– What is usability?
– User-centered design
– Eye tracking
• Benefits and ROI
• UNISA usability laboratory
• Studies and results
• Other Interesting figures and examples
• Questions
Introduction
What is usability?
• Lack of usability:
– Play video
What is usability?
• Usability means making products and
systems easier to use, and matching them
more closely to user needs and
requirements
• ISO9241: Usability is the
–
–
–
–
–
–
effectiveness
efficiency and
satisfaction
with which specified users
achieve specified goals
in particular environments
What is usability?
•
•
•
•
•
•
Effectiveness
– Can users achieve what they need by using the
product?
Ease of learning
– How fast can a user who has never seen the interface
learn to use it?
Efficiency of use
– How fast can users complete the task?
Memorability
– Can users remember enough to reuse the interface
effectively?
Error prevention
– Can users complete tasks without making errors?
Satisfaction
– How much does the user like using the system?
User-centered design
• ISO 13407: User-centered design is
characterised by:
– the active involvement of users and a
clear understanding of user and task
requirements
– an appropriate allocation of function
between users and technology
– the iteration of design solutions
User-centered design
• Goal
– To develop easy-to-use products
– that lead to increased user satisfaction
– meet your organizational or business objectives
• Difference between other design
philosophies
– user-centered design tries to optimize the user
interface around how people can, want, or need
to work
– rather than forcing users to change how they
work to accommodate the system or function
User-centered design
• ISO13407
• 4 essential activities in UCD
http://www.upassoc.org/usability_resources/about_usability/
what_is_ucd.html
User-centered design
• 1. Specify context of use
– Identify the people who will use the product, what
they will use it for, and under what conditions
they will use it
User-centered design
• 2. Specify requirements
– Identify any business requirements or user goals
that must be met for the product to be successful
User-centered design
• 3. Design
– Producing designs and prototypes
– This part of the process may be done in stages,
building from a rough concept to a complete
design.
• 4. Evaluation
User-centered design
– The most important part of this process is that
evaluation - usability testing with actual users - is
as integral as quality testing is to good software
development
User-centered design
• The stages are carried out in an iterative
fashion
– Cycle repeated until the project's
usability objectives have been attained
– Critical - participants accurately reflect
the profile of your actual users
– Talk directly to the user at key points in
the project to make sure the site will
deliver upon their requirements
User-centered design
• Methods used:
– Requirements and data gathering
• Interview
• User observation
• Contextual inquiry
• User group meeting
• Focus groups
• JAD sessions
• Surveys
• Bulletin boards/discussion groups
• Web logs
User-centered design
• Methods used:
– Usability testing
• Task walkthroughs
• Performance-based tests
• Heuristic evaluations
• Preference tests
• A-B testing
What is usability?
• Usability testing involves
– measuring the performance of users on
tasks with regard to
•
•
•
•
the ease of use,
the task time, and
the user‟s perception of the experience
of the product, software application, website
or systems
– Based on Performance
– Purpose: Feedback
What is usability?
• What Usability testing is not
– Not market research – gathering opinions
– not just a milestone to be checked off on the project
schedule
– not finished when the last participant leaves
• Involves systematic observation under controlled
conditions to determine how well people can use the
product
– watching people trying to use something for its
intended purpose
• Consider the findings, set priorities, and CHANGE
the prototype or site based on what happened in the
usability test
Eye tracking
• A technique to determine eye movement
and eye fixation patterns of a person
• The human eye moves by alternating
between
• Saccades
• Quick movement of the eye
• Move focus from one area to the next
• Fixation
• Time spent looking at the newly found area
Benefits and ROI
Benefits and ROI
•63% of software projects exceeded their estimates
because (Lederer and Prassad ‟92)
•Frequent request for changes by users
•Overlooked tasks
•Users‟ lack of understanding of their own tasks
•Insufficient user <> analyst communication
•80% of software life cycle costs occur during
maintenance phase (Pressman „92)
•80% of maintenance comes from unforseen/unmet user
requirements (Martin „83)
•60% of maintenance phase is due to re-work because
user requirements were not clear at start (Standish „83)
Benefits: usability
•Benefits for an organisation:
•Save on development and maintenance by:
•Saving on development costs
•Decreasing the development time
•Reducing maintenance costs
•User-centred design
•Usability testing very early and throughout
development
•With a rigorous user-centered methodology, you
can ensure successful, on-time delivery – and
avoid the rework that's necessary later if you
don't talk to users in the first place
Benefits: usability
•Benefits for an organisation:
•Increase Total Revenue by increasing the
number of:
• Transactions
• Conversion and hence turnover
• Returning customers
• New customers through word of mouth
advertising
Benefits: usability
•Benefits for an organisation:
•Stimulate the use of the website by:
•Increasing the success ratio for a visitor
•Decreasing the number of errors and show
stoppers
•Increasing efficiency for users (less time to
complete a task)
•Increasing users satisfaction
Benefits: usability
•Benefits for a user:
•Understand a site faster and achieve their
goals faster
•Have a positive experience using the
website
•Develop trust in the company
•Perform tasks without help from telephone
or e-mail
•http://www.2c.nl/en/what_is_usability/advantages_usability_test.php
Benefits: combination
•Usability
•All
and eye tracking:
the benefits of traditional usability results
•Eye
tracking can add new and interesting insights
•Provide
insights that are not available from
traditional usability testing methods
•Scan
paths; Pattern of fixations
•Time
spent looking at various display elements
•Insight into deployment of
•Participant strategies
visual attention
Benefits - Examples:
•Human Factors International
•Staples.com
•67% more repeat customers
•31-45% reduced drop-off rates
•10% better shopping experience
•80% increased traffic
•Increased revenue – sales up by 491%
•Utility company in Canada – Intranet
•Training hours from 8 hours to 15 minutes
•Energy company in US – Intranet
•300 support calls per day to ZERO
UNISA usability lab
UNISA usability lab
• A usability laboratory is a state-of-the-art
facility designed to support the observation
of HCI
• Users are brought into a controlled
environment, in which they are asked to do
specific tasks within specific timeframes
• Evaluators
– observe the problem(s) the participant might have
– videotape the participant
– analyse the data
UNISA usability lab
• Observer room
• Participant room
• Separated by one-way mirror
• Tobii 1750 eye tracker
• Audio and Video Recording equipment
• Event logging, eye tracking software
Typical data collected
• Live video recordings
– Screen
– Mouse and keyboard movements
– Facial expression
• Audio
• Eye tracking video with cursor
• Eye tracking data files
• Post-test questionnaire
• Monitoring of tasks
Usability Measures
•
Effectiveness
•
•
•
•
Efficiency
•
•
•
Task completion rate
Number and percentage of tasks completed with
and without assistance
Error rate recovery
Task completion time
Real-time events
Satisfaction
•
Post-test questionnaire
Eye tracking measures
•
Number of fixations
•
Fixation duration
•
Number of fixations on each Area of Interest
•
Number of gazes on each Area of Interest
•
Time to 1st fixation on target Area of Interest
•
Scan path
Studies and results
MyUnisa – Assignment Submission Tool
• Participant Profile
– Full-time UNISA students who have to submit
assignments online
– 10 participants
– 5 male, 5 female
– 7 different languages
– 5 expert, 5 non-expert Web users
MyUnisa – Assignment Submission Tool
Task example: Submit a PDF file
• Task completion: 70%
• Assistance needed: 40%
• Errors made: 100%
• Median task completion time: 115.60 seconds
• Play video: (PDF)
MyUnisa – Assignment Submission Tool
• All participants made errors
– Did not select appropriate file format
– Error message:
• “ERROR: The type of file does NOT match the selected file type.
(PDF!=DOC)”
• All non-expert received error at least twice
• 3 experts received error once
– Error message not comprehended
• Difference between PDF and Word not
understood
MyUnisa – Assignment Submission Tool
• Eye Tracking
– Step 1 page: File selection till “continue” button click for Step 2
• Main goal: did participant read “File Format”
– Only 3 participants with several fixations on “File Format” label and dropdown box
• Still made the error
Eye tracking results: Task 1
MyUnisa – Assignment Submission Tool
• Eye Tracking:
– Step 1: Error message
• Main goal: did participant read the error message
– All participants read error message
• Only 3 participants got it right after one error
Eye tracking results: Task 1
MyUnisa – Assignment Submission Tool
• PDF task – Recommendations:
– File Type
• Should be picked up automatically
– Error message
• Simple language to be used
• PDF!=DOC is a computer term
– FAQ and instructions should contain more information about file
format and PDF
– Help Video File to teach new users
MyUnisa – Assignment Submission Tool
• Post-test questionnaire: Participant comments:
– “MyUnisa is a very good tool in my life, saves time driving to
UNISA or Post Office to submit assignments.”
– “MyUnisa is at the cutting edge in educational pile drive.”
– “Think it would be difficult for a person who rarely uses computers.
They may have a problem understanding things like the file type.”
– “It will be very useful if users could be shown how to use this
website. I had a friend who had to show me how to work it.”
Network Management tool
• Prototype tool
• Allows network managers at Nelson Mandela Metropolitan
University to analyse and explore application performance on
network
• Novel visualisation techniques
• Visualise application delay performance
• Integrated Tertiary Software application implemented at University
• Participants: Sound knowledge in domain of network
management
• Pilot study: 6 participants
• Case study: 9 participants
Results
•
Network Overview Task
•
•
•
Usability results
•
•
•
Determine which Virtual Local Area Network has the highest
maximum total delay
Answer obtained from graph or text
100% task completion rate
100% correct answers
Eye tracking results
•
•
Graphical AOI
• Most number of fixations
• Greatest fixation percentage
Very few fixations on textual AOI
Results
•
Subnet View Task
•
•
Usability results
•
•
•
Determine the highest server delay
100% task completion rate
• Graph was filtered quick
0% correct answers
• Long time to find answers
Eye tracking results
•
•
Mean of 34.33 fixations on graph to find answer
• Difficulty of extracting information from the graph
Participants fixated on wrong areas of Y-axis
Other interesting figures &
examples
Other interesting figures & examples
• Telkom corporate website – eye tracking video
• Karnaugh maths application – usability video
Incorporate usability in your
workspace!!
Conclusions
• Learn about usability
– Websites
• Usability Professionals Association
• Human Factors International
– Books
• Prioritizing Web Usability – Nielsen, Loranger
• Institutionalization of Usability – Schaffer
– Join a Usability forum
• CUA, HFI forums
• SA UX Forum
• LinkedIn
– Take a University course
Conclusions
• Encourage usability in your project lifecycle
– User-centered design
•
Third-party usability
– Write usability guarantees into vendor contracts
– Work with vendors to ensure usability (from the
start!!!)
• Its about the user!
Conclusions
• Usability can help you!
– Websites
– Intranets
– Systems
– Products
• UNISA – usability facility available
• To discuss usability testing/research potential, contact:
– Marco Pretorius
– pretomc@unisa.ac.za / marco.pretorius@gmail.com
• Play video