Enhancing the forensic value of handwriting
using emotion prediction
Dr. Meryem Erbilek
Prof. Michael Fairhurst
Cheng Li
Outline
•
•
•
•
•
•
Introduction
Handwriting data collection
Emotion prediction approach
Experiments and results
Discussion
Conclusion and future work
Page 2
Introduction
•
Soft biometrics:
¾ Characteristics such as subject age or gender, which
provide important information about an individual without
providing a specific identification label.
•
Use of soft biometrics:
¾ Provides information about a subject.
¾ Allows elimination of some possible individuals from
consideration in an identification scenario.
¾ Allows improvement of accuracy in an identification
scenario.
Page 3
Introduction (cont.)
•
Use of soft biometrics :
¾ Predictive capabilities to higher levels of abstraction
(mental and emotional state). For example, a knowledge
of whether an individual is happy or sad, anxious or calm,
under stress or relaxed.
•
Background and motivation:
¾ Predicting human emotions have mostly been
investigated with respect to the face and voice modalities.
¾ Particularly within the handwriting biometric research
community, substantive studies of the analysis of emotion
prediction are generally difficult to find.
Page 4
Handwriting data collection
•
Task definition:
¾ Task1: Subjects were asked to copy a list of defined words (43
words, 243 characters).
¾ Task 2: Subjects were asked look at a picture and to write a
description of this in their own words. The picture was chosen to
convey a positive and “happy” message.
¾ Task 3: As for Task 2, but the picture was this time chosen to
convey a more “sad” message.
¾ Task 4: Subjects were asked to copy a specified list of words
(10 words, 50 characters) but had to do so within a time span of
a maximum of 10 seconds.
Page 5
Handwriting data collection (cont.)
•
Capture definition:
¾ All the writing samples
were captured using a
digitising tablet with a
paper overlay to
provide familiar
feedback during the
writing process.
Page 6
Handwriting data collection (cont.)
•
Emotion and score definition:
¾ “happy” and “stressed”
•
Data:
¾ 100 subjects ( 1 sample for each task)
¾ Gender balance (55 male and 45 female)
Page 7
Emotion prediction approach
•
Prediction:
¾ “happy” or “not happy”
¾ “stressed” or “not stressed”
•
Ground truth:
Page 8
Not stressed
Stressed
Not Happy
Happy
Emotion prediction approach (cont.)
•
Handwriting processing:
¾ First, 12 common handwriting features are extracted.
¾ Subsequently, extracted features are mean and
variance normalised.
•
Classification:
¾ KNN, Jrip and SVM classifiers.
¾ Leave-one-out validation methodology.
Page 9
Experimental results
•
Accuracy of “Happy” prediction
Page 10
Experimental results
•
Accuracy of “Stress” prediction
Page 11
Discussion
•
This preliminary study provides some evidence that it is
possible to identify the “higher level” state, reflecting the
emotional disposition of an individual, based on an
analysis of a fragment of that individual’s handwriting.
•
The predictive capability observed will depend to an
extent on the choice of classification infrastructure.
•
The nature of the writing task (fixed or variable) will
influence the prediction accuracy likely to be achievable.
•
The differences between different tasks are somewhat
reduced if a sufficiently powerful classifier is adopted for
the prediction processing.
Page 12
Discussion
•
Within the variable tasks there is a greater variability in
predictive accuracy.
•
The increasing information result in improving prediction
accuracy, but beyond a certain point, performance in this
respect falls back.
Page 13
Conclusion and future work
•
Provides a basic step towards developing our understanding of the
relationship between handwriting and emotional and similar higher
level states further, and has presented some results which indicate
some potential fruitful developments.
•
A particular area which perhaps needs to be improved in the first
instance is developing a better methodology for quantifying the
ground truth emotional state of the subjects.
•
Our most immediate concern, however, is to develop a more
comprehensive analysis of the data we have already collected,
building on the positive initial steps reported here, since we believe
this will support important new insights in an area where existing
work has not been extensive.
Page 14
Questions?
Dr. Meryem Erbilek
M.Erbilek@kent.ac.uk
Prof. Michael Fairhurst
M.C.Fairhurst@kent.ac.uk
Cheng Li
cl382@kent.ac.uk
Page 15