Ecological Factors of Emotion Recognition Using Physiological Signals
Delbert Hung*, Dr. Elaine Biddiss*ⱡ, Dr. Tom Chau*ⱡ
* Institute of Biomaterials and Biomedical Engineering, University of Toronto
ⱡ Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital
Abstract:
Research Question: Are physiological responses from emotional experiences modulated by:
1) activity level; 2) stimulus modality and 3) individual differences in reactivity to emotional
experiences?
Target Population: The primary focus of development for such technology is those with limited
or deficient communication pathways in terms of the expression of emotion (e.g. fear and
disgust in Huntington’s or Parkinson’s disease patients).
Introduction/Background: Research activity in the field of emotion recognition has increased
in recent years from the healthcare community. Machines that are ‘emotionally aware’ have
been sought for the purpose of enhancing user interaction with computers and for telemedicine
applications. Literature in emotion recognition using physiological signals has shown feasibility
at the conceptual level but is still lacking in studies that address barriers that will arise in
practical implementation. A small set of these barriers are reflected in the research questions
posed above.
Methods: To investigate these “ecological” factors, we designed a factorial layout experiment
with 2 mobility states (sitting/walking) x 3 stimuli modality (visual/auditory/personal imagery)
where we will conduct non-invasive surface recording of physiological signals (e.g.
electrocardiogram, skin conductance and respiration) from thirty participants. Affective stimuli
will be presented from a computer using items from the International Affective Pictures System
(IAPS), the International Affective Digitized Sounds (IADS) and personal imagery cues.
Emotional responses to the stimuli will be assessed by self-report using a modified SelfAssessment Manikin. Individual differences in emotional experiences will be measured using
two psychometric tests: the Affect Intensity Measure and Behaviour Activity/Inhibition Systems
Scales.
Features extracted from the physiological signals will be compared in a factorial MANOVA.
Emotional experience data from self-reports will be used as labels for supervised machine
learning algorithms that will try to classify emotional states based on physiological signal inputs.
Multiple machine learning algorithms will be tested and compared to select for one that achieves
the highest sensitivity, specificity and overall classification accuracy.
Results and Conclusion: The project is still under the data collection stage and there are no
definitive results or conclusions that can be drawn