GSR and Blink Features for Cognitive Load Classification
Nargess Nourbakhsh
Dr. Rafael. A. Calvo, Dr. Fang Chen
School Electrical and Information Engineering
FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES
INTRODUCTION
• A system capable of monitoring its user’s
mental workload can evaluate the suitability
of its interface and interactions for user’s
current cognitive status and properly change
them when necessary.
• In this way, the optimum performance will be
obtained and many human errors will be
avoided.
• Example application domains: learning,
brain-computer interactions, driving, air traffic
control, piloting.
• One-way analysis of variance (ANOVA) on
the self-reporting scores showed a highly
significant difference between task levels
(F3,48=108.63, p<0.05).
Results
Response Time
Accummulative GSR
F3,48 = 7.22, p < 0.05
• Time between disappearing the last (fourth)
number of the task and selecting the answer
GSR Frequency Power
F3,48 = 4.07, p < 0.05
• Response time has a direct relation with the
task difficulty level.
Blink Number
F3,48 = 3.37, p < 0.05
Blink Rate
F3,48 = 3.22, p < 0.05
4000
ANOVA results of features for four task difficulty levels
3500
COGNITIVE LOAD CLASSIFICATION
3000
2500
• Subjective methods (self-report)
1500
• Behaviours (speech, gestures, movements)
1000
EXPERIMENT
• 8 Arithmetic tasks; Adding-up 4 sequentially
presented numbers (each for 3 seconds)
• 4 Task difficulty levels (in randomised order):
binary, one-digit, two-digit and three-digit
numbers
Methods
2000
• Performance based methods
• Galvanic skin response (GSR) and eye
blinks are cognitive load measures which
can be captured at low cost, with short
preparation time and minor restriction of
users’ movements.
• Classification algorithms:
• Support vector machines (SVM)
• Naïve Bayes classifiers
500
• Cross validation method:
• Leave-one-subject-out
0
1
2
3
4
Response time for each task difficulty level (milliseconds)
• Results of ANOVA test on response time of
different task levels are significant
(F3,48=62.59, p<0.05).
• These observations about subjective rating
and response time show that the designed
tasks have effectively manipulated the
cognitive load.
GSR and Blink Features
• Two GSR and two blink features were
calculated for each task:
• GSR power spectrum (frequency power)
• blink number (number of blinks in the
task)
• blink rate (number of blinks in the task
divided by task time)
• 13 Subjects (24 to 35-year-old)
1.2
1.2
1
1
0.8
0.8
COGNITIVE LOAD MEASUREMENT
0.6
Subjective Rating
0.2
2-Class
Classification
4-Class
Classification
SVM
66.4%
34.6%
Naïve Bayes
71.2%
40.4%
Classification
Algorithm
2-Class
Classification
4-Class
Classification
SVM
66.4%
37.5%
Naïve Bayes
65.4%
35.6%
2-Class
Classification
4-Class
Classification
SVM
62.5%
40.0%
Naïve Bayes
62.5%
40.0%
Classification
Algorithm
0.4
2-Class
Classification
4-Class
Classification
SVM
57.5%
31.3%
Naïve Bayes
55.0%
32.5%
0.4
0.2
0
1
2
3
1
4
2
3
4
10
Average GSR features of all subjects for the four task levels;
accumulative GSR (left), GSR frequency power (right)
8
Classification
Algorithm
Classification accuracies of blink number
0.6
0
9
Results
Classification
Algorithm
• Feature normalisation to omit subjectdependency
1.4
• Combining GSR and blink features
Classification accuracies of GSR frequency power
• Values of each feature averaged between
tasks with same difficulty levels for each
subject
• Remote eye tracker: faceLAB 4.5 of Seeing
Machines Ltd (sampling rate: 50Hz)
• Four-class
classification
and
binary
classification (considering levels 1 and 2 as
low load and levels 3 and 4 as high load)
Classification accuracies of accumulative GSR
• accumulative GSR (summation of GSR
values over task time)
• GSR device: ProComp Infiniti of Thought
Technology Ltd (sampling rate: 10 Hz)
Statistically significant results for all the
features:
Feature
Cognitive Load Measurement Methods
• Physiological methods (signals from heart,
eye, brain, muscles and skin): they are realtime, accurate and quick and do not interfere
with the main task.
•
Classification accuracies of blink rate
7
6
1.8
5
1.4
3
1.2
1.6
1.2
1
0.8
1
0.6
0
0.4
1
2
3
4
2-Class
Classification
4-Class
Classification
SVM
71.5%
53.6%
Naïve Bayes
75.0%
50.0%
0.8
0.6
0.4
0.2
0.2
0
Average Subjective Ratings of Task Difficulty Levels
Classification
Algorithm
1.4
1
2
• Feature combination improved accuracy:
1.8
1.6
4
1
2
3
4
0
1
2
3
4
Average blink features of all subjects for the four task levels;
blink number (left), blink rate (right)
Classification accuracies of blink number + GSR
frequency power
THIS RESEARCH IS SP
Publications:
N. Nourbakhsh, Y. Wang, F. Chen, R. A. Calvo. Using Galvanic Skin Response for Cognitive Load Measurement in Arithmetic and Reading Tasks, In Proc. OzCHI 2012, 26-30th
THIS RESEARCH IS SPONSORED BY
• Australian Postgraduate Award
November 2012, Melbourne, Australia, pp. 420-423, ACM Press (2012).
• Norman I. Price Scholarship, School of EIE,USYD
N. Nourbakhsh, Y. Wang, F. Chen. GSR and Blink features for Cognitive Load Classification, INTERACT 2013, Part I, LNCS 8117, pp. 159–166, Springer (2013).
• National ICT Australia (NICTA)