Technology to support
psychosocial self-management
Kurt L. Johnson, Ph.D.
Henry Kautz, Ph.D.
Technology to address
• Limitations in executive function
• Behavioral dyscontrol
• Limitations in social interaction
Potential advantages of ATC
•
•
•
•
Less intrusive than teachers or coaches
Context aware
Demand aware
Record data and deliver interventions
using same system
• User control of interventions
• Potentially more acceptable to recipient
Executive Function
• Prospective memory assistance
– Shared calendars
– Integrated task lists
– Remote prompting
• Time management
– Alerts/alarms
Technology
• Wearable multi-sensor board (UW/Intel)
– Acceleration (movement)
– Sound
– Wireless data transfer
Acceleration Data
6
Recognized Class
Interpreted Data
Time in seconds
7
Speech Detection
• From the audio signal, we want to
extract features that can be used to
determine
– When the someone is talking
– Whether it is the user or someone else
– Turn-taking
– Rate of conversation
• For privacy, the features should not
allow the audio to be reconstructed
Speech Production
vocal tract
filter
The source-filter Model
Privacy-preserving speech detection
works by detecting the fundamental
formants
Segmenting Speech Regions
Technology
• RFID Bracelet (Intel Research)
– Detect touching tagged objects
– Hand acceleration
– Wireless data transfer
Technology
• Video
– Subject watching computer monitor:
• Eye tracking
• Expression (challenging)
– Open area, controlled lighting:
•
•
•
•
Location
Proximity to others
Body pose
Head pose (challenging)
Behavioral Dyscontrol
• Notice personal factors preceding behavioral
dyscontrol?
– Autonomic
– Physical behavior (rocking, shifting position)
• Notice social environmental factors
– Proximity of voices
– Ambient noise
– Activity of others
• Context sensitive AI intervention via cell
phone/PDA
Promotion or modifying social
interaction
• Notice proximity of people
• Social networking – understanding
availability of known communication
partners
• Monitoring initiation and maintenance of
communication
• Too short? Too long?
• Context sensitive AI intervention via cell
phone/PDA
Related Research
• Assisted technology for persons with
cognitive disabilities is a small but growing
field
• The most advanced work to date on assistive
technology for autism is by Rosalind Picard
(MIT)
–
–
–
–
ESP: Emotional Social Intelligence Prosthesis
MindReader
Affect recognition tester
SelfCam
MindReader, Tester, SelfCam