Toward Fully Automated PersonIndependent Detection of Mind
Wandering
Robert Bixler & Sidney D’Mello
rbixler@nd.edu
University of Notre Dame
July 10, 2013
mind wandering

indicates waning attention

occurs frequently


20-40% of the time
decreases performance


comprehension
memory
solutions

proactive

mindfulness training


tailoring learning environment


Mrazek (2013)
Kopp, Bixler, D’Mello (2014)
reactive

mind wandering detection
our goal is to detect mind wandering
related work – attention

Attention and Selection in Online Choice Tasks


Multi-mode Saliency Dynamics Model for Analyzing Gaze and
Attention


Navalpakkam et al. (2012)
Yonetani, Kawashima, and Matsuyama (2012)
distinct from mind wandering
mind wandering detection

neural activity

physiology

acoustic/prosodic

eye movements
neural activity
Experience Sampling During fMRI Reveals Default Network and Executive System
Contributions to Mind Wandering

Christoff et al. (2009)
physiology
Automated Physiological-Based Detection of Mind Wandering during Learning

Blanchard, Bixler, D’Mello (2014)
acoustic-prosodic
In the Zone: Towards Detecting Student Zoning Out Using Supervised Machine Learning

Drummond and Litman (2010)
eye movements
mindless reading
mindful reading
research questions
1. can mind wandering be detected from eye gaze data?
2. which features are most useful for detecting mind
wandering?
data collection

4 texts on research methods




self-paced page-by-page
30-40 minutes
difficulty and value
auditory probes


tobii tx300
9 per text
inserted psuedorandomly (4-12s)
type of report
end-of-page
within-page
total
yes
209
1278
1487
no
651
2839
3490
total
860
4117
4977
data analysis
1. compute fixations

OGAMA (Open Gaze and Mouse Analyzer)
(Voßkühler et al. 2008)
2. compute features
3. build supervised machine learning models
features

global

local

context
global features

eye movements






fixation duration
saccade duration
saccade length
fixation dispersion
reading depth
fixation/saccade ratio
local features

reading patterns





word length
hypernym depth
number of synonyms
frequency
fixation type





regression
first pass
single
gaze
no word
context features

positional timing




previous page times



since session start
since text start
since page start
average
previous page to average ratio
task


difficulty
value
supervised machine learning

parameters






window size (4, 8, or 12)
minimum number of fixations (5, 1/s, 2/s, or 3/s)
outlier treatment (trimmed, winsorized, none)
feature type (global, local, context, combined)
downsampling
feature selection

classifiers (20 standard from weka)

leave-several-subjects-out cross validation (66:34 split)
1. can mind wandering be detected
using eye gaze data?
best model kappas
0.3
0.25
kappa
0.2
0.15
0.1
0.05
0
End-of-page
Within-page
report type
1. can mind wandering be detected
using eye gaze data?
75
70
accuracy %
65
60
Accuracy
55
Expected
Accuracy
50
45
40
End-of-page
Within-page
1. can mind wandering be detected
using eye gaze data?
confusion matrices
end-of-page
actual
classified response
response
within-page
prior
actual
classified response
response
prior
yes no
yes .54 .46 .23
yes no
yes .61 .39 .36
no .23 .77 .77
no .42 .58 .64
2. which features are most useful for
detecting mind wandering?
kappa
average kappa values across feature
types
0.3
Global
0.2
Local
Context
0.1
0
End-of-page
Within-page
report type
Global +
Local +
Context
2. which features are most useful for
detecting mind wandering?
rank
end-of-page
within-page
1
previous value
saccade length max
2
previous difficulty
saccade length median
3
difficulty
fixation duration ratio
4
value
saccade length range
5
saccade length max
saccade length mean
6
saccade length range
saccade length skew
7
page number
fixation duration median
8
saccade length sd
fixation duration mean
9
saccade length mean
saccade duration mean
10
saccade length skew
saccade duration min
summary

mind wandering detection is possible



kappas of .28 to .17
end-of-page models performed better
global features were best

exception: context features highest ranked for end-of-page
enhanced feature set

global




pupil diameter
blink frequency
saccade angle
local


cross-line saccades
end-of-clause fixations
enhanced feature set
0.3
kappa
0.25
Original
Enhanced
0.2
0.15
0.1
End-of-page
Within-page
predictive validity
mw rate
end-of-page
predicted
actual (model)
post
transfer
knowledge learning
-.556
-.248
-.415
-.266
actual (all data)
-.239
-.207
within-page
predicted
actual (model)
actual (all data)
-.496
-.095
-.255
-.431
-.090
-.207
self-caught mind wandering
self-caught vs. probe caught
0.35
0.3
kappa
0.25
0.2
0.15
0.1
0.05
0
End-of-page
Within-page
report type
Self-Caught
what does mind wandering look like?

saccades


slower
shorter

more frequent blinks

larger pupil diameters
limitations

eye tracker cost

population validity

self-report

classification accuracy
future work

multiple modalities

different types of mind wandering

mind wandering intervention
acknowledgements





Blair Lehman
Art Graesser
Jennifer Neale
Nigel Bosch
Caitlin Mills
questions
?