Ενσωμάτωση συναισθημάτων σε
παιχνίδια σοβαρού σκοπού
Εβδομάδα 6
Εισαγωγή στη Συναισθηματική νευροεπιστήμη
Summary
• The need for ubiquitous contemporary
controllers in smart homes
• Beyond the interfaces, applications and
controllers. The Controller Application
Communication (CAC) framework
• Facilitating developers in adopting new
techniques independently of hardware device, OS
and programming language.
• Introducing proposed architecture in lab
environment
• Moving on in smart home environment
…σύγχρονα χειριστήρια…
• Χειριστήρια που έκαναν την εμφάνισή τους
πρόσφατα:
– inertial sensors (Wii remote controller)
– pressure sensors (Wii balance board)
– camera systems (Kinect sensor)
– biosensors (Neurosky’s Mindwave - one channel
EEG)
– multisensory devices (smart phones, smart
watches)
…σύγχρονες συσκευές αλληλεπίδρασης…
•
•
•
•
Computers
smartTV
tablet
Smart phones
Η ανάγκη για διάχυτη πληροφορία χειριστηρίων
σε έξυπνα σπίτια
• Παρέμβαση και πρόληψη (e.g. Wii suite of controllers has been used for
enabling elderly people to increase their physical activity levels by
engaging in exergames)
• Διακριτική παρακολούθηση υγείας (e.g. Kinect has been utilized assess
probability of elderly fall)
• Εφαρμογές διαισθητικής αλληλεπίδρασης (χειρονομίες)
Πλαίσιο επικοινωνίας συσκευών και εφαρμογών
(CAC) framework
• Ενδιάμεση εφαρμογή για την διασύνδεση
χειριστηρίων και εφαρμογών.
• Συνεχής ροή πληροφορίας από τα χειριστήρια
στις εφαρμογές (streaming) (e.g. skeleton,
RGB image, center of pressure) to the server.
• Ομαδοποίηση πελατών λογισμικού.
Πλαίσιο επικοινωνίας συσκευών και
εφαρμογών (CAC) framework
• Πρώτόκολλο επικοινωνίας: Web Sockets (fullduplex communication channels, designed to
be implemented in web browsers and web
servers
• Μορφή μηνυμάτων: JSON
• Συμβατά χειριστήρια: Microsoft Kinect, Wii
remote control, Wii balance board, Mindwave,
Emotiv, Android Sensors
Αρχιτεκτονική πλαισίου επικοινωνίας συσκευών
και εφαρμογών
Konstantinidis, E.I., Antoniou, P.E., Bamparopoulos, G., Bamidis, P.D.: A lightweight framework for transparent cross
platform communication of controller data in ambient assisted living environments. (2013), (under review at
Information Sciences)
Technology and Active Aging - Well being
Technology intervenes non-pharmacologically
Serious games of physical and cognitive training are used
against cognitive decline
Elderly people improve their health and well being
crucial for prolonged independent living
 How can we tailor “silvergames” environment and conditions to suit
“grown up kids” capabilities and needs?
 Which are the biological and behavioral signals we can combine with
gaming?
 How can emotion, a function which remains stable or even improves
across adult lifespan, be employed?
 How can the fusion of the collected information modify the game
environment so as to motivate the user to interact with the game on
a daily basis?
Prevention of Cognitive Decline
 Cognitive decline is responsible for much of the Quality of Life deterioration in
seniors
 Non-pharmacological arsenal:
– Physical exercise
– Cognitive exercise
 Proper training  neural plasticity changes  slow down or invert cognitive
decline (Smith et al., 2009).
Serious Games
 Motion capturing is an integral part of the game design
 employment of sensors within a gaming environment
projected on mobiles devices (phones, tablets) and SmartTVs
 body movement
 gesture and facial recognition
Serious Games for Elderly Healthcare
 Current State Of The Art: A cohort of studies on serious games for
seniors has only been recently introduced.
 Objective: Identify and meet senior users’ needs within the
gaming environment
 Seniors have to be motivated so as to retain their engagement
to the game on a frequent basis.
 Approach: This is a game design issue that may be answered via
Affective Computing
 develop computer interfaces so as to explore the underlying
relation between the user’s emotions and human health, both
mental and physical.
Literature Guidelines for “Silvergames”
address usability issues related to physical and mental (cognitive
and/or emotional) changes in elderly users.
the game control should be easy in adaptation
provide a pleasant experience by being responsive (findings
based on video observations)
the game control should be accurate
provide a realistic experience that will fit the user’s
expectations (findings based on interviews)
the game control should employ appropriate gestures, visual and
auditory feedback
the game control should employ clear user interface, ability to
adjust the difficulty
Beyond Guidelines
?
Affection across Adult Lifespan
Positivity effect=put more priority
on positive information than on
negative information.
Cognitive decline, potential health
problems, physical limitations.
Different modalities
Brain Activity via EEG
Facial characteristics via cameras
Voice via microphones
Heart Rate via ECG
Posture, gestures via fisheye camera
Proposed Methodology
DSS schematic overview
Re-adjustment of gaming
environment
4
Long term
analysis
5
Gaming interaction
2
Sensor
analyzers
1
Central
processing
Sensor –
Level
processing
3
Short term
analysis –
Context analysis
Current
Profile
Sensor Data
Fusion
Billis et al., “A Decision-Support Framework for promoting Independent Living and Ageing Well”, J Biomed. Health Inf. (under minor
revision)