Assisted Cognition
Henry Kautz,
Oren Etzioni, Dieter Fox,
Gaetano Borriello, Larry Arnstein
University of Washington
Department of Computer Science & Engineering
1
An Epidemic of Alzheimer’s
Disease
2
3
4
Statistics for United States
4.6 million people with Alzheimer’s
16 million people by 2050
Today costs $100 billion @ year for care
Additional $61 billion in lost productivity
from family members
$ ½ Trillion total cost by 2050!
5
Lost Competencies
Short-term memory
Ability to carry out complex tasks (driving,
paying bills, cooking, house-hold tasks)
Ability to orient self in time and space
Memory of events
Dressing, bathing, cooking, eating
Memory of concepts
Self-initiative
Recognize friends, relatives
6
Cognition in Context
Can often compensate for physical
disabilities by change in environment
Wheelchairs
Cognitive competence also depends on
environment
Physical
Social
7
Social Context
8
Social Context
dressing
personal grooming
gardening
cooking
exercise
self-medicating
shopping
9
Problem
Caregiver burnout
½ of all family caregivers suffer depression
“The 36 Hour Day”
Far too few professional caregivers to
provide constant 1-on-1 help in
institutional settings
Already a nationwide shortage of good
staff
10
Assisted Cognition Systems
Learn to interpret human behavior from lowlevel sensory data
General commonsense knowledge
Patterns of behavior idiosyncratic to the particular
user
External data sources
Actively offer prompts and other forms of help
as needed
Alert human caregivers when necessary
11
Architecture
13
Applications
The Activity Compass
The Adaptive Prompter
14
The Activity Compass
Goal: help person safely and independently
move about the community (including use
of public transit)
User carries GPS/wireless equipped PDA
AC system tracks user’s position, predicts where
user is going based on past experience
System offers help when
Inferred plan is likely to fail otherwise (e.g. miss bus)
User is likely to be lost or disoriented (wandering, on
wrong bus)
User explicitly consults PDA
15
Gathering Data
16
green – GPS readings (10 sec), yellow – location estimation
(probability distribution)
17
Creating the User Model
Training Data:
20,000 GPS readings
3 weeks of occasional use
Reduce noise using Kalman filter
Hand labeled by mode of transportation
Walking, In Car, On Bus, Riding Bike, Inside
Predicting current mode
Input: current location/time/velocity
Decision tree learning: 98.9% accuracy (10 FCV)
Predicting next mode transition(s)
Input: current mode/location/time/velocity
Decision tree learning: 98.8% accuracy (10 FCV)
18
Crisis Prediction (I)
When might user need help?
“Cutting it too close”
Associated with each leaf of decision tree is a
spatio-temporal window
Compute expected position of next transition
within that window
If position is at or near upper temporal
boundary, increased probability that expected
transition will fail to hold (e.g., user will miss
the bus!)
19
Crisis Prediction (II)
When might user need help?
External changes in the world
Some kinds of transitions (e.g. board bus) are
enabled by external forces (the bus!)
Real-time Seattle transit information available
online
Use information to more finely label training data
Crisis = prediction that is inconsistent with external
information
20
Crisis Prediction (III)
When might user need help?
Novel events
After-the-fact discovery that predicted
behavior did not occur
Ask user to confirm actions are intended
Explicit error models
Dangerous locations
Wandering trajectory
21
Intervention Strategies
Must balance
disutility of crisis
cost of annoying user
probability of crisis
do not want to over-rely on negative
reinforcement
Qualitative preference language
“Never let me miss a bus late at night”
22
User Interface
23
The Adaptive Prompter
Goal: help a person carry out a multistep task
Smart home tracks residents and objects
Hierarchical recognition model
Simple behavior (sleeping, walking)
Simple actions (get into bed)
Meaningful patterns of actions – plans
Model of possible failures and interventions
24
Example
1.
2.
3.
4.
5.
Joe enters bathroom at 9:00 am.
He turns on water, and picks up toothbrush.
Nothing happens for 15 seconds. AC system
recognizes “tooth brushing” activity has stalled.
Prompts Joe to pick up toothpaste. Joe does so
and completes task.
Joe leaves bathroom with water still running. AC
system gently encourages Joe to go back and
turn it off.
25
Towards a Behavior
Description Language
Requirements
probabilities on fluents and events
continuous (or finely discretized) time
probability distributions on temporal relations
between events
hierarchical events
plans – intended complex events
defective plans
system interventions
utilities of defects and interventions
26
Approach
Develop scenarios for AP in consultation
with experts on Alzheimer’s care
Prototype specification language
Semantics via translation into Dynamic
Bayesian Networks
Interventions: Dynamic Decision Networks
A terrific KR challenge!
See work by Martha Pollack, Hung Bui,
Geib & Goldman, Daphne Koller
27
Data Source: Elite Care
Oakfield Estates
28
People
University of Washington Computer Science &
Engineering
UW Medical Center
Alzheimer’s Disease Research Center (ADRC)
UW Institution on Aging
Intel Research
People & Practices – User studies of future technology needs
Intel Research Seattle – Ubiquitous computing
Elite Care
Oakfield Estates assisted living
http://assistcog.cs.washington.edu/
29
Advertisement
UbiCog 2002 – Workshop on Ubiquitous
Computing for Cognitive Aids
September 29, 2002
Gothenberg, Sweden
Part of UBICOMP-2002, the major
ubiquitous computing conference
Slots for speakers still available, email
Henry Kautz <kautz@cs.washington.edu>
30