The role of eye tracking in usability
evaluation of LMS in ODL context
Mr Sam Ssemugabi
Ms Jabulisiwe Mabila
(Professor Helene Gelderblom)
College of Science Engineering and Technology
University of South Africa
School of Computing
Technology has been a part of our everyday environment
1. Introduction
for generations. It empowers us and frustrates
us, it
- Eye Tracking and Usability
simplifies and complicates our life. It separates
us and
- LMS & ODL
brings us closer together. But even though we interact with
technology everyday, we easily forget that technology
products are made by people, and that someone,
somewhere should get the credit when technology works
well for us or get the blame when it doesn’t.
(Garrett James, 2003: 7)
The role of eye tracking in usability
evaluation of LMS in ODL context
1. Introduction
1. Introduction
- Eye Tracking and Usability
- LMS & ODL
2. Problem statement
2. Objective of the study
3. Methodology
3. Tasks
4. Tasks
5. Results
6. Conclusion
4. Results
5. Conclusion
The role of eye tracking in usability
evaluation of LMS in ODL context
Eye tracking
is the technique whereby an individual’s
eye movements are measured so that the
researcher knows both where a person is looking at,
at any given time and the sequence in which the
eyes are shifting from one location to another .
(Poole and Ball, 2006)
www.company.com
Eye tracking metrics
www.company.com
•
Nielson, 2006
•
Heat maps
•
Fixations
•
Saccades
•
Eye gaze duration
•
Area of interest scan path
(AOI)
The role of eye tracking in usability
evaluation of LMS in ODL context
Usability
defined as the extent to which a product
can be used by specified users
to achieve specified goals
with effectiveness, efficiency and satisfaction
in a specified context of use.
(ISO 9241-11, 1998)
www.company.com
The ODL Context
Diversity of student profile in ODL institutions
• Young and old
• Employed and unemployed
• Rural areas and urban areas
Diverse student profile - widely varying ages, experience,
backgrounds - language, location
Learning Management System
(LMS)
A LMS software application includes programs
for e-learning, administration, documentation,
tracking, and reporting, training programs and
online events (Ellis, 2009).
Usability assessment of HCI
Include:
•
Expert evaluation
•
Heuristic evaluation
•
Walkthrough
•
Observation
•
Retrospective self report
(questionnaires, think aloud)
(Holzinger, 2005)
•
Other techniques – Eye tracking
Why?
The role of eye tracking in usability
evaluation of LMS in ODL context
1. The role of eye tracking
2. Usability assessment LMS
3. ODL context – diversity of student
population
Who?
•
•
Participants
First year Unisa students
Registered for End User Computing
(EUC131T)
Methodology
Instruments
•
Tobii Eye Tracker 1750
•
Video recording
•
Questionnaire
•
Observations
Tobii 1750 eye tracker
Unisa , Pretoria
School of Computing Usability Laboratory
www.company.com
Task - Objective
)
•
Navigation
•
Reading
•
Text input
www.unisa.ac.za
www.company.com
https://my.unisa.ac.za/portal/
The Tasks
Task 1: Starting from myUnisa homepage, go to myUnisa
(use icon or drop down menu)
Task 2: Find ‘Claim myLife e-mail’ on myUnisa screen
Task 3: Set-up myLife account
(Input of student number; Surname, names, date of
birth, ID number or Passport/Foreign ID (for foreign
students); Click on ‘acknowledge guidelines’
Task 4: Join myUnisa
Task 5: Activate your myUnisa password
www.company.com
Discussion of Results
Task 1 Claim myLife e-mail account
•
•
•
Task completion – 67%
Assistance needed – 75%
Errors made – 92 %
Task 2: Join myUnisa
•
•
•
www.company.com
Task completion – 83%
Assistance needed – 42%
Errors made – 58%
Interpretation
www.company.com
Interpretation
Problem 1: Terminology
Difference between “Claim myLife e-mail
account” and “Join myUnisa” not clear
“Registration” was selected by two users which
gives a completely different screen to “join
myUnisa”
Problem 2: Process
2 separate processes, passwords mixed up
Problem3: Instructions
Transition difficulties, moving from “Claim
myLife” to “Join myUnisa”.
Participants did not knowing what to do next
Problem
4: Varying computer literacy levels
www.company.com
The Tasks
Time to complete tasks
•
Participants who used computers regularly or had
studied computing required an average of 25
minutes to complete the tasks
•
Participants without computing experience or who
had not studied computing previously required on
average 54 minutes to complete the tasks
www.company.com
Conclusion
1. Diversity in age, experience, background, and location of
ODL students necessitate rigorous usability evaluation
2. Recommendations can be made regarding the interface
design based on eye tracking data, and
3. The interpretation of the information collected through eye
tracking should be used together with other usability
evaluation methods – observation, heuristics, questionnaire
www.company.com