Emotion Recognition using the
GSR Signal on Android Devices
Shuangjiang Li
Outline
• Emotion Recognition
• The GSR Signal
• Preliminary Work
• Proposed Work
• Challenges
• Discussion
Emotion Recognition
• Human-Computer Interaction
•
•
•
•
•
Speech recognition
Gesture/Action recognition
Facial expression recognition
Emotion recognition
…
• Affective Computing ( Picard @MIT Media Lab around late 90s)
Emotion Recognition
• Physiological Signals
Source: http://biomedikal.in/2011/05/important-physiological-signals-in-the-body/
The GSR Signal
• Galvanic Skin Response (GSR)
• measuring the electrical conductance of the skin
• due to the response of the skin and muscle tissue to external and internal
stimuli, the conductance can vary by several microsiemens (unit of ohm).
• GSR is highly sensitive to emotions (fear, anger, startle response, etc.)
http://en.wikipedia.org/wiki/Skin_conductance
The GSR Signal
• GSR Sensor
• SHIMMER (Sensing Health with Intelligence, Modularity, Mobility and Experimental
Reusability) Platform
• The goal of SHIMMER is to provide an extremely compact extensible platform
for long-term wearable sensing in both connected and disconnected settings
using proven system building blocks.
• a highly extensible wireless sensor platform
•
•
•
•
SHIMMER firmware is based on TinyOS
Data transmit via Bluetooth
Can sense EMG, ECG, GSR, etc.
Support Matlab, LabView, Android, C#/.Net etc.
http://shimmer.sourceforge.net/
http://www.shimmer-research.com/
Adrian Burns, SHIMMER: An Extensible Platform for Physiological Signal Capture, IEEE EMBS, 2010
Preliminary Work
• Emotion recognition based on the GSR signal
• Four emotion categories: amusement, fear, relax, sadness
• Using GSR + Accelerometer signal
• Preprocessing
• Using supported accelerometer data
• Denoising using median filter
• Data rescaling and normalization
• Feature Extraction
• 6 statistical features + 10 time domain features + 4 frequency domain features
+ feature selection (SFFS)
Preliminary Work
• Recognition rate
• KNN
• 10-fold cross-validation
• Subject dependent / single subject
Proposed Work
• Emotion recognition on Android Devices
• Android GUI for reading GSR sensor data
• GSR data preprocessing
• GSR data classification
• Sequential learning
Challenge
• Emotion signal tend to very noisy.
• Emotion signal generally lacks ground truth and emotion is very
subjective.
• Recognition algorithms on Android devices should be light weight
• Dealing with sequential data
Discussion
Q&A