Web-Based Cognitive
Health Coaching for Elders
in a Home Environment
Holly Jimison, PhD
Misha Pavel, PhD
ORCATECH
Oregon Center for Aging & Technology
What Can Be Done about Cognitive Decline?
• Good news
– Neural plasticity at all ages
– New drugs to delay cognitive decline
• Potentially good news
– Monitoring computer interactions may detect
decline earlier
– Development of cognitive exercises to delay
decline and possibly remediate
ORCATECH
2
Project Objectives
• Develop & test computer games for continuous cognitive
monitoring of elders in a home environment
– Sufficiently enjoyable to play frequently
– Embedded assessment algorithms
– Dynamic user models
• Develop & test a cognitive health coaching system that
provides opportunities for cognitive skill building
– Assess cognitive deficits
– Provide tailored cognitive exercises
– Interactive cognitive health coaching
ORCATECH
3
Problems with Current Cognitive Assessment
Methods
• Assessments done only after concern on the
part of the patient or caregiver – usually late
• Repeated assessments infrequent – yearly at
best
• Large day-to-day variability in cognitive
performance with onset of dementia
• Unwanted variability in assessments due to
education, language, motivation, etc.
ORCATECH
4
Traditional Cognitive Tests
•
•
•
•
•
•
•
ORCATECH
Verbal Fluency
Word List Learning
Constructional Praxis
Trail Making Test
Symbol Digit Modalities Test
Letter-Number Sequencing
Finger Tap Test
5
Adaptive Cognitive Computer Games
• 30 users (all cognitively
healthy)
– Average age 81.5 yrs
(standard dev= 6.04 yrs)
– 13% male, 87% female
Adaptive Cognitive Computer Games
Spry Learning / OHSU (NIST grant)
• 9 Interactive games
developed with Spry Learning
as part of NIST grant
• Being evaluated in local
elders’ residences
• Neuropsychological data was
collected at baseline, 3 month,
6 month, and at a planned 1
year period
ORCATECH
6
Elders’ Use of Computers
• Growth in computer use by elders rose 47%
since 2000 – fastest growing group
• 70% use email
• 59% Web browsing
• 35% play computer games (similar to general
population rate of 39%)
ORCATECH
7
In-Home Monitoring of Computer Interactions
• Frequent data  trend detection
– Individual as their own control
– Less influenced by
• Educational background
• Language
• Cultural differences
• Dynamic algorithms to model user performance
• Inexpensive  screening, coaching
ORCATECH
8
Game Data Analysis and Interpretation
• Objective: Develop metrics that reflect cognitive capability
and function of the players
For Each Game
• Ideal Player:
Develop framework capturing the information processing
resources and capabilities required by an ideal player
• Cognitive Models and Limits:
Describe ways that a human player may differ from an ideal
one
• Specific Metrics:
Characterize the required cognitive processes by a small set
of parameters
ORCATECH
9
Letter Lotto – Word Game
Word games:
• Word length
• Word complexity
• Speed of word generation
• Total number of words
Compare to:
• Verbal Fluency
ORCATECH
10
Letter Generation Rate as a Game Metric
Total Time to Guess Words (Each Game)
vs Number Letters Guessed Correctly (User 1020)
250
200
Time (Sec)
150
100
50
Regression Equation:
y= 2.31189 x + 13.2406
r = 0.866714
0
ORCATECH
0
10
20
30
40
50
60
70
11
# Letters Guessed Correctly
80
90
100
Letter Time vs. Neuropsychological Scores
Neuropsych Average Total Fluency vs Letter Generation Time
85
Fluency vs Letter Generation
y = -4.44616 x + 75.3999
Average Total Fluency (Letter + Animal)
80
r = -0.343154
75
70
65
60
55
50
0
ORCATECH
0.5
1
1.5
2
2.5
3
3.5
4
12
Letter Generation
Time (Sec/Letter)
4.5
5
Frequency Generation of Words by Length
Average Frequency of Words Correct by Length
– Try harder because it’s
the goal
– Have to use all letters
– Hence not necessarily
most difficult
0.9
0.8
Average Frequency
(Words Guessed/Total Possible)
• Shorter words are
guessed more frequently
than longer ones
• Exception is 7 letter
word
1
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
1
2
3
4
5
6
Word Length
ORCATECH
13
7
8
9
10
Neuropsychological Scores vs. Proportion of Words
Neuropsych
AverageScore
Total vs
Fluency
vs Game
Difficulty
Neuropsych
Proportion
of Words
85
Average Total Fluency (Letter + Animal)
80
75
70
65
Fluency vs Game Difficulty
60
y = 0.522433 x + 40.2374
r = 0.403347 (with outliers)
Outliers
55
y = 0.687054 x + 31.3083
r = 0.659844 (w/o outliers)
50
30
ORCATECH
35
40
45
50
55
60
65
14of
Game
Difficulty
Proportion
Generated Words
70
75
On the Flip Side – Memory Game
Memory games:
• Short-term memory
• Working memory
• Spatial memory
• Abstract reasoning
ORCATECH
15
Deterministic Computational Model of Working Memory
• Characterize Memory Capacity as a Function of:
– Intervening number of events
– Intervening time
– Memory load
• Simple Memory Model: Discrete Buffer
Time, Events
B
A
C
D
A
E
B
C
F
B
G
H
D
G
E
I
B
A
C
D
D
E
B
B
F
F
G
H
D
D
D
B
A
C
C
D
E
E
B
E
F
G
H
H
H
B
A
B
C
D
D
E
D
E
F
G
F
F
D
E
F
E
B
ORCATECH
C
D
16
E
Probability of Correct
Results
Subject 1020, N = 8687
1
0.5
0
0
5
10
15
Probability of Correct
Intervening Number of Events
1
0.5
ORCATECH
0
0
5
10
15
Intervening
17 Time [sec]
20
25
0.5
0
2
4
6
8
10
12
14
16
Probability of Correct
1
0.8
0.6
0.4
2
4
6
8
10
12
14
16
18
20
22
0.8
0.6
0.4
0.2
0
5
10
15
1
0.5
0
0
5
10
15
Intervening Time [sec]
Intervening Time [sec]
ORCATECH
Subject 1024, N = 892
1
Intervening Number of Events
Intervening Number of Events
0.2
Probability of Correct
Subject 1021, N = 387
1
Probability of Correct
Probability of Correct
Examples of Other Fits
18
20
25
Weekly Tracking of Working Memory Buffer Size
Subject 1021, N = 120
2.5
Memory
Size
BufferSize
RelativeBuffer
2
1.5
1
0.5
0
10
ORCATECH
20
30
40
50
60
Time19[Days]
70
80
90
Estimation of Working Memory Capacity
3.5
Size
Buffer
Square
BuffereRoot
Sizeof[#of
Items]
3
2.5
2
1.5
1
0.5
0
0
ORCATECH
5
10
15
Subjects
20
20
25
Assessment of Neuropsychological Test Scor
200
180
TRAILS-B Scores
160
140
120
100
80
60
40
20
0
60
ORCATECH
70
80
90
100
Trail-Making Test B
R = 0.60
Digit Span
R = 0.36
110
120
130
140
Predicted Scores from Flipside
21
150
160
21 Tally (2-D Black Jack)
00
8
18
0
ORCATECH
19
22
Dual Task
games:
• Divided
attention
• Working
memory
• Spatial
memory
Evaluation Metrics
high
Double
Bust
Cost
Single
Bust
Accept
low
Single
Win
Multiple
Win
Bust-Win
low
ORCATECH
Quality
23
high
Response Time vs Quality of Move
Value of subject move / value of Ideal move
Subject ID:1020
HIGH-QUALITY
LOW-QUALITY
α=3
FAST
in quality
function
SLOW
Response Time per move (sec)
ORCATECH
24
Reference Players
• Ideal Player
- uniform card distribution
- look a single move ahead
- choose ‘loc*’ where
Valuec(loc*)= min(Valuec(loci)), i = 1 … # empty loc
Valueq(loc*)= max(Valueq(loci)), i = 1 … # empty loc
• Chance Player
- place card randomly
- all locations are equally likely
ORCATECH
25
Prob Searching Occ Loc
Modeling Subject Reaction Time
a= 9.9886
b= 38.4971
To= 1.284 s
Te= 0.683 s
Avg RT per move
Number of Occupied Locations
Subject ID 1025
“Serial Processing
of locations”
Number of Occupied Locations
ORCATECH
RTmin= 2.9397 s
26
ORCATECH
27
Modeling Subject Reaction Time
ORCATECH
28
Modeling Subject Reaction Time
Subject ID
a
b
T_occ
T_emp
T_min
1020
10.7166
13.3114
1.2583
0.4043
3.3855
1021
14.1551
1.1685
1.8104
0.2804
1.2226
1022
11.8667
3.534
0.9403
0.9704
2.9144
Took more21.4271
time
1024
10.8912
looking at…
0.9
1.2769
0
2.053
37.9464
1.2048
1.6135
0.9237
1025
9.9886
38.4971
1.2840
0.6830
2.9397
1026
18.0063
1.348
0.8088
0.2773
1.1533
0.7120
0.0282
1.6898
1.1771
0
2.5044
0.5736
0.8603
2.2749
1023
1027
1028
1029
Model
shows these
18.0757
0.9
subjects
don’t12.3714
spend
13.3458
time searching
13.7456
1.0617
empty locations
1030
10.9933
19.787
1.2249
9.9950
7.3894
1032
11.5338
4.6529
1.2456
0.1127
1.176
1033
8.7695
12.867
0.5714
0.0385
2.137
1034
12.8849
21.9481
0.6782
9.9996
2.9198
ORCATECH
29
Cost and Benefits of Divided Attention
Forced Bust-Win
Unforced Bust
Forced Bust
ORCATECH
30
Pyramid Builder – Color / Shape Matching
Shape / color games:
• Divided attention
• Speed / accuracy
• Visual search
• Planning
• Working memory
ORCATECH
31
Response Time vs. Move Number
File: tst3.mouse
8
• Response time (move
duration) Increases
with the Complexity of
the Game
– Average move duration
increases with move
number
– Variance increases with
move number
Response Time [Sec]
7
6
5
4
3
2
1
0
0
20
40
60
80
Move Number
ORCATECH
32
100
120
140
Example: Response Time Distribution
(Data from a Single Game)
File: tst2.mouse
0.16
• Fast responses to
known or easy
decision
• Medium requiring
some search and
decision
0.14
0.12
Relative Frequnecy
Multimodal distribution
suggests
0.1
0.08
0.06
0.04
0.02
0
• Long – pondering and
planning of execution
ORCATECH
0
0.5
1
1.5
2
Response Time[Sec]
33
2.5
3
3.5
Resulting Search Efficiency
tst3.move
4
3.8
slope = 24 ms/item
Move Times [sec]
3.6
3.4
Correlation
with
Trail-making B
3.2
3
2.8
R = 0.49
2.6
2.4
2.2
2
0
5
10
15
20
Number of Distractors
ORCATECH
34
25
30
Scavenger Hunt – Trail Making Test
Rapid movement
games:
• Motor speed
• Visual search
• Set switching
• Speed of processing
• Divided attention
Compare to:
• Finger Tap Test
• Trail Making Test
ORCATECH
35
FreeCell – Solitaire Planning Game
Strategy games:
• Planning
• Visual search
User Performance Metrics
t
Dt   f  d  d 1 ,
• Difficulty
 1
dt = number of moves to solution at time t
• User Performance
– Reduction in difficulty
t  dt  dt 1
• Outcome of game
• Time to completion
ORCATECH
37
FreeCell Performance Curve
S ubject nb1.als
90
80
70
subject
Difficulty
Difficulty
60
50
40
solver
30
20
10
0
Actual
Expected
0
10
20
30
40
50
Events
60
70
80
Number of moves
ORCATECH
38
90
100
Performance Scores for Normals vs MCI
Average
(Individual
Ave(Perf))
St Dev
(Individual
Ave(Perf))
Average
(Individual
SD(Perf))
Normals
0.58
0.12
0.38
Mild
Cognitive
Impairment
0.27
0.72
0.55
ORCATECH
39
Benefits of Cognitive Computer Games
Improvements over standard techniques:
–
–
–
–
–
Daily monitoring (not yearly, after-the-fact)
Individual as their own control
Adaptive measurement – more sensitive measure
Understanding of variability, trend detection
Unobtrusive, more representative performance
– Potential for remediation
ORCATECH
40
Cognitive Health Coaching Project Objectives
Remote Cognitive Health Coaching to the Home
• Provide tailored cognitive exercises & advice with tailored
feedback
• General coaching protocols based on principles of health
behavior change
– Using health behavior change techniques (motivational
interviewing)
– Tailored action plans
– Automated feedback
– Facilitate care manager and caregiver communications
• Extend the outreach of health coaches
– Tailored messaging / automated dynamic protocols
• Incorporate patient and family members into the care
team
ORCATECH
41
Overview of Methods
• Needs Assessment
– Interviews & focus groups with patients, family caregivers,
home health workers, clinicians, nurse care managers,
payers
– Interview researchers/experts with an interest in deploying
and evaluating a health intervention in a home environment
•
•
•
•
Clinical / Behavioral Protocol Development (Cognitive)
Design Specifications
System Development
Implementation / Usability Testing / Stakeholder
Feedback
• Test in Homes of Elders
ORCATECH
42
Elder Focus Group Findings
• “Brain before body”
• Most had multiple chronic diseases, but were not as
interested in working on the “medical” aspects of health
• Highest priorities were quality-of-life and independence
• Advice for others
– Considered depression and socialization to be the big
problems for their age group
– Many elders recognized a downward spiral in older
people who became increasingly embarrassed about
their abilities and appearance, so that they tended to
avoid leaving their room or socializing.
ORCATECH
43
Expert Interviews
• Researchers
• Government
– Legislators, policy makers
• Industry
– Health record developers, interoperability specialists, disease
management system vendors
• Elder Care Professionals
– Geriatricians, home health nurses, senior activity coordinators,
health coaches
• Cognitive Health Intervention Experts (continuing)
ORCATECH
44
Cognitive Health Coaching Design
Elder-Specific HRA
HomeCare
Nurse /
Coach
Health Coaching System
Shared Care Plan
Medication List
(PHR for Chronic Care)
ORCATECH
Diagnoses
Lab Values
Cognitive Protocols
(Spry Games)
Monitoring Modules
Sensors
Allergies
Elder
45
Family
Support
Welcome Ellen! Your Cognitive Exercise Plan
Ellen – Last week you did a great job with the number of FreeCell and Sudoku
exercises played. This week, in addition let’s try adding some verbal exercises,
such as the Spelling Bee game featured below.
Click here for more information on the importance of verbal exercise.
Reply to Coach
Previous Messages
Update My Information
View My Action Plan
Ellen’s Cognitive Exercise Progress
Detail on Progress
Return to PHMS
Goal
2/day
2/day
2/day
2/day
2/day
2/day
2/day
2/day
FreeCell
Hunt
LLotto
Sp Bee
FlipSide
Pyr Bldr
21Tally
Sudoku
Mon
***
**
*
**
*
**
***
Tue
**
**
Wed
****
*
**
**
**
**
Th
*
*
***
Fri
**
*
*
******
*
*
**
**
***
****
Sat
Log Off
Cognitive Coaching
for Better Living
ORCATECH
Sun
46
Review of Methods
• Needs Assessment
– Interview researchers/experts with an interest in deploying and
evaluating a health intervention in a home environment
– Interviews & focus groups with patients, family caregivers, home
health workers, clinicians, nurse care managers, payers
• Clinical / Behavioral Protocol Development (Cognitive)
• Design Specifications
• Implementation / Usability Testing / Stakeholder
Feedback
• Test in Homes of Elders
ORCATECH
47
Components of Cognitive Health Coaching
• Adaptive Computer Games - Cognitive Exercise
–
–
–
–
NIST grant with Spry Learning
Adaptive cognitive games
Embedded cognitive assessment algorithms
Suite of 9 games measuring various aspects of cognition
• Health Advice Related to Cognition
–
–
–
–
–
–
Physical exercise
Social Interactions
Nutrition
Sleep
Novelty exercises
Mood management (depression advice, humor)
ORCATECH
48
Sample Message – Novelty Exercises
Non-routine actions and thoughts encourage the growth of new
connections in the brain. This week let’s work on novelty exercises
that can exercise your brain. Here are some examples to choose
from:
□ Turn your desk clock or an illustrated calendar upside down.
□ Go to a new park, or a new store.
□ Try brushing your teeth with your non-dominant hand.
□ Distinguish coins using only your sense of touch.
□ Turn the volume off on television and invent the dialog based on
what you see the characters doing.
□ Find a magazine you normally wouldn't choose, and read it for a
while.
ORCATECH
49
Sample Message – Novelty Exercises
◙ Turn your desk clock & calendar upside down.
When you look at a familiar image right side up, your left brain quickly
labels it and diverts your attention to other things.
When the picture is upside down, the quick-labeling strategy doesn't
work and your right-brain kicks in, trying to interpret the shapes,
color and relationships of a puzzling picture.
This week, place at least one clock and your calendar upside
down. Practice using them this way and see how it feels. Do
you get better over time?
Next week we’ll try something new.
ORCATECH
50
User Model
1. Elder Assessment
• Health Risk Appraisal
• Health Goals
• Motivation
• Barriers
• Readiness-to-Change
• Contact Preferences
•Timing / Media
• Social Support Contacts
• Data Sharing Preferences
• Authentication (pswd?)
Web forms – spread timing
Elder User Model
• Goal (Assume
Cognitive Health)
• Motivations
• Barriers
• Change State
• Contact Prefs
• Social Support
• Display Prefs
Coach User Model
Predefined
2. Family / Friend
Assessment
• Contact Preferences
•Timing / Media
• Level of commitment
• Authentication (pswd?)
Fam/Fr User Model
• Contact Info/Prefs
• Data Permission
• Authentication (pwd)
Web form
ORCATECH
51
3. Create / Update
Tailored Action Plan
Daily goals & activities
Monitor
continuous
4. Tailored Feedback &
Recommendations
Based on Spry Games
Based on meeting other goals
Sleep behaviors
Diet
Exercise …
5. Generic News and
Articles of Interest
Standard Web content
Architecture for Tailored Messaging
Shared Care Plan
Elder Assessment
Elder DB
Monitoring
User Model
Raw data
User State
Sensors
Computer Interactions
Self Report
State Dependent
Active Methods
(Coaching Protocols)
Update Action Plan
Tailored Content Delivery
• reminders
• encouragement
• tailored feedback/education
• prompt for assessment
Content Database
Preconfigured Messages
Tutorials
General Content
Definitions
Summary Reports
Prioritized Alerts
Recommended Messaging
Elder’s Technology Interface
Health CoachTechnology
(computer, cell phone, exercise watch)
(computer, cell phone)
ORCATECH
52
Welcome Ellen! Your Cognitive Exercise Plan
Ellen – Last week you did a great job with the number of FreeCell and Sudoku
exercises played. This week, in addition let’s try adding some verbal exercises,
such as the Spelling Bee game featured below.
Click here for more information on the importance of verbal exercise.
Reply to Coach
Previous Messages
Update My Information
View My Action Plan
Ellen’s Cognitive Exercise Progress
Detail on Progress
Return to PHMS
Goal
2/day
2/day
2/day
2/day
2/day
2/day
2/day
2/day
FreeCell
Hunt
LLotto
Sp Bee
FlipSide
Pyr Bldr
21Tally
Sudoku
Mon
***
**
*
**
*
**
***
Tue
**
**
Wed
****
*
**
**
**
**
Th
*
*
***
Fri
**
*
*
******
*
*
**
**
***
****
Sat
Log Off
Cognitive Coaching
for Better Living
ORCATECH
Sun
53
Review of Methods
• Needs Assessment
– Interview researchers/experts with an interest in deploying
and evaluating a health intervention in a home environment
– Interviews & focus groups with patients, family caregivers,
home health workers, clinicians, nurse care managers,
payers
• Clinical / Behavioral Protocol Development (Cognitive)
• Design Specifications
• Implementation / Usability Testing / Stakeholder
Feedback
• Test in Homes of Elders
ORCATECH
54
Summary
• Scalable approach to delivering health care to the home
–
–
–
–
Extends outreach of lower cost professionals
Can involve family members
Timely and continuous
Home health management is key to caring for chronic conditions
• Cognitive monitoring and potential remediation
– Most important aspect of health
– Keeps people independent – (better quality of life & lower costs)
• Cognitive Health Coaching System
– Serves as a framework for testing the best delivery of cognitive
interventions
ORCATECH
55
Acknowledgements
• ORCATECH – NIA funding
• Intel funding for initial FreeCell Cognitive Monitoring
• NIST Funding for Spry Learning Cognitive Computer
Games
• Health Coaching funding to OHSU from Intel as part of
the Behavioral Intervention & Assessment Commons
More info: jimisonh@ohsu.edu
ORCATECH
56
Representative FreeCell Performance Data
MCI Subject and Healthy Subject
Over Time
Subject with Mild Cognitive Impairment
Cognitively Health Subject
MCI Subject FreeCell Efficiency Over Time
Healthy Elder FreeCell Efficiency Over Time
2.5
2.5
2
2
1.5
1.5
1
1
0.5
0.5
0
2/23/04
-0.5
3/14/04
4/3/04
4/23/04
5/13/04
6/2/04
6/22/04
0
2/3/04
-0.5
-1
-1
-1.5
-1.5
ORCATECH
57
2/23/04
3/14/04
4/3/04
4/23/04
5/13/04
6/2/04
Median Login Inter-keystroke Intervals
Milliseconds
Subject with
mild cognitive
impairment
Two cognitively
healthy elders
Date
ORCATECH
58
Framework for Data and Inferences
• Data
– Movements – trajectory and speed
– Selection of objects – ID and response time
– Placement of objects – position and speed
Raw Data
• Interpretation
–
–
–
–
Characterization of movements
Inference of tactics
Inference of strategy
Inference of abilities
Tactics
Strategies
State
Estimation
ORCATECH
59
Duration of Move
↑ with Complexity of Game
File: tst3.mouse
8
– Average move duration
increases with move
number
– Variance increases with
move number
Response Time [Sec]
7
6
5
4
3
2
1
0
0
20
40
60
80
Move Number
ORCATECH
60
100
120
140
Mouse Move: Trajectory and Dwell Time
– Multimodal Trajectory of a
mouse movement on a single
move.
– The size of the disks indicates
the dwell time at a that location
Move number 53
650
600
550
500
450
400
350
300
250
200
ORCATECH
61
0
100
200
300
400
500
Background on Shared Care Plan
• Personal Health Record specifically designed for people with
multiple chronic conditions
• Marc Pierson, MD – Whatcom County, Washington
• Pursuing Perfection Project funded by Robert Wood Johnson
Foundation
• Further refinement of the application was made possible
through grants from the Agency for Healthcare Research and
Quality (AHRQ), the Foundation for eHealth Initiative (HRSA
OAT), and PeaceHealth
• Commercial version called PHMS from the Congral Company
– Nic Ivancic
ORCATECH
62
Project Team Members
•
•
•
•
•
•
•
Holly Jimison – PI
Misha Pavel – Co-PI
Nicole Larimer – Senior Research Associate
Michael Shapiro – Graduate Research Assistant
Bobby Hagerty – Health Educator / OHSU Brain Institute
Nic Ivancic – Programmer, Congral Company
Lori Nichols - Consultant
– Director, Whatcom Health Information Network
– Shared Care Plan Personal Health Record
• Marc Pierson – Consultant
• Spry Learning Company
ORCATECH
63
Correlation Amongst Standard Test (Animal and Let
•
Average Animals vs Average Letter Fluency (C,F,L)
35
Average Animal vs Average Letter
•
y = 0.36845 x + 7.78601
30
2
R =0.354482 (with outliers)
Outliers
y = 0.378927 x + 7.89503
25
2
Letter Fluency
R =0.464943 (w/o outliers)
•
•
20
15
10
5
0
0
5
10
15
20
25
30
35
Animal Fluency
ORCATECH
64
What is the inherent variability
within animal and letter fluency?
Within users averaged over
baseline, 3 month, and 6 month
test period
R^2~=.46
Used sum of animal and letter
score as total verbal fluency
Newly Developed Cognitive Computer Games
Spry Learning / OHSU (NIST grant)
ORCATECH
65