ASHA 2013
1
VALIDATING PATIENT OUTCOMES
USING AN IPAD-BASED SOFTWARE
PLATFORM FOR LANGUAGE &
COGNITIVE REHABILITATION
Swathi Kiran, Carrie Des Roches, Isabel Balachandran,
*Stephanie Keffer, Elsa Ascenso, *Anna Kasdan
Speech and Hearing Sciences, Boston University
Department of Neurology, Massachusetts General Hospital
Funding from Wallace H. Coulter Foundation:
BU-Coulter Translational Partnership Program
ASHA 2013
Disclosure-Swathi Kiran
Has significant financial Interest
• Chief Scientist for Constant Therapy
• Ownership stock in Constant Therapy
2
ASHA 2013
3
Disclosure
Other Authors: Carrie Des Roches, Isabel Balachandran,
Elsa Ascenso
- Nothing to disclose
Significant contributors
- Stephanie Keffer, Anna Kasdan
- Nothing to disclose
ASHA 2013
4
Introduction
• About 795,000 Americans each year suffer a new or
recurrent stroke (NIDCD.gov). Also, about 1.7 million
individuals suffer from traumatic brain injury each year
(CDC.gov).
• Individuals with language and cognitive deficits following
brain damage likely require long-term rehabilitation.
• Consequently, it is a huge practical problem to provide the
continued communication therapy that these individuals
require.
ASHA 2013
5
Using technology to improve treatment delivery
• Recent studies have examined the efficacy of
rehabilitation techniques, such as videoconferencing, for
individuals with hearing, stuttering and motor speech
issues
• Other studies have provided aphasia therapy over the
internet to individual patients
• More recently, there have several computerized braintraining software designed for normal adults.
ASHA 2013
6
What is the evidence behind using technology to
deliver treatment?
• CogMed
• A software targeted at improving working memory abilities in
individuals with brain injury (Johansson & Tornmalm M, 2012;
Lundqvist et al. 2012). These studies found improvements in
working memory skills on the trained CogMed software as well as
on other working memory tasks and functional settings.
•
.
Posit Science
• Barnes et al (2009) examined the effectiveness of the software
Posit Science in improving auditory processing speed in individuals
with mild cognitive impairment (MCI).
• Although differences between the experimental and control group
were not statistically significant, verbal learning and memory
measures were higher in the experimental group than the control
group.
ASHA 2013
7
What is the evidence behind using technology
to deliver treatment?
• Lumosity
• Finn and McDonald (2011) used Lumosity software to target
attention, processing speed, visual memory in experimental and
waitlisted controls. Results showed experimental participants
improved on the training exercises more than the controls.
• There are other software programs- that function more
like AAC- devices.
• Therefore, there an increased awareness and momentum
for applying computer technology in the rehabilitation of
aphasia
ASHA 2013
8
Rationale
• Additionally, there is increased patient demand to transition
from traditional but outdated flashcard based therapy in
order to keep up with the evolution of technology.
• Nonetheless, the burden of evidence for technology-based
treatment applications is no different than traditional
treatment approach for rehabilitation after brain damage
• Q1. Can we provide a technologically based rehabilitation
program that meets the same benchmarks for clinical
efficacy?
• Q2. How do we individualize treatment for patients with
brain damage as no two patients are alike?
ASHA 2013
9
Study
Question: Does a structured therapy program that includes
homework practice delivered through an IPAD result in
significant gains in overall communication?
Goal: Compare patients who receive a structured IPAD
delivered therapy program that is practiced up to 7 days a
week with patients who receive standard one-on-one
individualized therapy that is provided 1 or 2 days per week
by a therapist.
ASHA 2013
10
Study
• Because of the flexibility that ipads provide to patients and
the accessibility to free/paid apps that provide variable
levels of exercises, it is important to standardize the
nature and form of treatment that is provided to patients
using ipads.
• Since patients have access to ipads at home, it provides a
unique opportunity to examine the extent of compliance
when patients are provided with a homework regimen
11
ASHA 2013
Participants
Experiment (N = 40)
Control (N= 9)
Ave Age
63 years (SD = 11)
68 years (SD = 10)
Ave Months post Onset
54 months (SD = 47)
98 months (SD = 132)
WAB – AQ (western
aphasia battery)
68.5 (SD = 26.52)
68.1 (SD = 31.35)
CLQT Composite
Severity (cognitive
linguistic quick test)
64% (SD = 25)
54% (SD = 28)
ASHA 2013
Demographic data
12
ASHA 2013
13
ASHA 2013
High Cognitive/Low
Language
High Cognitive/High
Language
Low Cognitive/Low
Language
Low Cognitive/High
Language
14
ASHA 2013
15
Experimental Design
Pre-Tx
Asses
sment
EXPERIMENTAL PATIENTS (N = 40)
Post -Tx
Assessment
Weeks
Pre-Tx
Asses
sment
CONTROL PATIENTS (N = 10)
Post -Tx
Assessment
ASHA 2013
16
ASHA 2013
17
ASHA 2013
Sample therapies/assessments for language and
cognitive processing
19
ASHA 2013
Structure of the tasks- Language
Language
Naming
Rhyme
judgment
Reading
Writing
Semantic processing
Sentence Planning
Word copy
Word identification
Category
identification
Active Sentence Completion
Word copy completion
Syllable identification
Category
matching
Feature matching
Passive Sentence
Completion
Word spelling
Phoneme
identification
Category matching
Reading passages
Long reading
comprehension
Sound-letter matching
Feature matching
Letter to sound matching
Picture naming
Word spelling
completion
Picture spelling
Picture spelling
completion
Voice Mail task
20
ASHA 2013
Structure of the tasks- Cognitive
Cognitive
Memory
Visuospatial
Picture matching
Word matching
Attention
Auditory
Spoken word
matching
Symbol
scanning
Problem
solving
Visuospatial
Symbol
cancellation
Analytical
reasoning
Flanker
Arithmetic
addition
Clock reading
Voice mail task
Map reading
Word sorting
Picture
sorting
Map
reading
multiplicatio
n
subtraction
division
Quantitative
reasoning
Time
estimation
Word math
problem
Executive
Skills
Instruction
Sequencing
ASHA 2013
21
ASHA
222013
Individualized therapy assignment based on initial
performance
23
ASHA 2013
# 29 during week 6 homework
Reading Passages (Level 3)
# 25 during week 4 homework
Word Identification
Word Copy (10 items)
Picture Ordering Tasks (5 items)
Map Tasks (10 items, Level 2)
Category Matching
Sound Identification (10 items)
Syllable Identification (10 items)
Picture Spelling (Level 3)
Clock Tasks (10 items)
# 44 for all 10 weeks
Week1
Picture
Spelling
(Level 2)
Reading
Passage
(Level 2)
Week2
Week3
Week4
Week5
Week6
Addition
(Level 3)
Picture
Spelling
(Level 3)
Addition
(Level 4)
Picture
Spelling
(Level 3)
Addition
(Level 4)
Picture
Spelling
(Level 3)
Addition
(Level 4)
Picture
Spelling
(Level 4)
Addition
(Level 5)
Picture
Spelling
(Level 5)
Word
Ordering
(Level 2)
Subtraction
(Level 3, 5
items)
Reading
Passage
(Level 3)
Word
Ordering
(Level 3)
Subtraction Subtraction
(Level 3) (Level 3)
Reading Reading
Passage Passage
(Level 3) (Level 3)
Word
Word
Ordering Ordering
(Level 3) (Level 3)
Map Tasks
(Level 2)
Subtraction Subtraction
(Level 4)
(Level 5)
Syllable
Syllable
Identification Identification
Multiplication
Multiplicatio (Level 2, 5
n (10 items) items)
Word
Ordering
Word
Map Tasks Map Tasks Task (Level Ordering Task
(Level 3) (Level 3)
4)
(Level 5)
Week 7 Week 8
Syllable
Identificati Syllable
on
Identification
Multiplicati
on (Level Multiplication
2)
(Level 3)
Picture
Picture
Ordering
Ordering Tasks (Level
Tasks
2)
Division
Week 9
Week 10
Syllable
Multiplication
Identification (Level 4)
Picture Ordering
Multiplication Tasks (Level 4, 10
(Level 3)
items)
Picture
Ordering
Tasks (Level
3)
Division (Level 3)
Division
(Level 2)
Clock Math (Level
2)
Word Matching
(10 items)
ASHA 2013
Based on WAB, CLQT
High Cognitive/Low
Language
High Cognitive/High
Language
Low Cognitive/Low
Language
Low Cognitive/High
Language
24
25
ASHA 2013
Low Language
profile- Low
cognitive profile
High Language
profile- low
cognitive profile
Low language
profile- high
cognitive profile
High language
profile- high
cognitive profile
86 year old male
75 year old male
77 year old male
56 year old male
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
category matching
feature matching
picture naming
rhyming
sound
identification
word identification
sound to letter
matching
word copy
picture matching
symbol
cancellation
•
•
•
•
picture spelling
naming picture
clock reading
instruction
sequencing
picture ordering
sound matching
symbol matching
voicemail
•
•
•
•
•
•
•
•
•
category
identification
category matching
feature matching
letter to sound
matching
reading passage
sound
identification
sound to letter
matching
word copy
word spelling
word ordering
•
•
•
•
•
•
•
•
•
•
category matching
feature matching
letter to sound
matching
sound to letter
matching
map reading
picture spelling
reading passage
rhyming
Sound
identification
syllable
identification
word spelling
word problems
ASHA 2013
26
ASHA 2013
27
Carrie/
• Please insert snapshots of the patient dashboard- that
shows start therapy, we will now do…
ASHA 2013
28
ASHA 2013
29
Methods
• During the weekly clinic sessions, the clinician would
decide to continue the participant on the same task or to
modify the treatment plan based on his/her performance.
• If the participant achieved 95% or higher accuracy two
times in succession,
• The clinician would either progress the next level of difficulty (e.g.,
Addition Level 1 to Addition Level 2)
• Would progress to a different task (e.g., assign category
identification after category matching).
• If participants performed at low accuracies or no change
over several sessions, that therapy task was replaced
with another task from the task list.
ASHA 2013
30
ASHA 2013
31
Overview of data analysis
1. Total therapy duration in weeks
2. Compliance- weekly log in times
3. Individual patient level analysis
1. Analyze by week
2. Analyze by task
3. Analyze by item
4. Overall patient performance over time
5. Patient performance over time relative to population
mean
6. Group level analysis: Analysis of tasks by patients
7. Group level analysis: Analysis of task by items, co-factors
8. Changes on standardized measures
32
ASHA 2013
1. Total therapy duration in weeks
The average therapy period for controls was 12.964 weeks
Average therapy period for experimental patients was 12.567 weeks
Weeks of iPad therapy: experimental vs. control
18
Therapy Time (weeks)
16
14
12
10
8
Controls (n=7)
6
Experimental (n=36)
4
2
0
1
3
5
7
9
11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
Patient
ASHA
332013
2. Compliance- Rates of log in to therapy
Experimental patients
Control patients
6
5
4
3
2
1
0
1/14/13
2/14/13 3/14/13
4/14/13
5/14/13
6/14/13
2
Ideal Control
871
904
697
698
80
882
699
52
1305
Ideal Experimental
37
813
1227
955
23
831
45
25
846
29
58
47
903
527
44
913
32
24
30
683
849
28
556
81
827
335
42
59
1174
1049
1339
551
978
27
1091
334
1079
31
46
212
86
879
ASHA 2013
8
Legend:
Control patients
Experimental patients
34
Patient usage by week
Average therapy time per week
12
10
7
6
4
1
0
ASHA 2013
35
3. Individual patient level analysis: By week
ASHA 2013
36
3. Individual patient level analysis: By task
ASHA 2013
3. Individual patient level analysis: By
items
37
ASHA 2013
38
4. Historical individual performance
Addition LV3
Addition LV4
1
Addition LV5
Clock Math LV2
0.9
Division LV1
Division LV2
0.8
Division LV3
0.7
Division LV4
Map Reading LV2
0.6
Map Reading LV3
0.5
Multiplication LV1
Multiplication LV2
0.4
Multiplication LV3
Multiplication LV4
0.3
Multiplication LV5
0.2
Picture Ordering LV1
Picture Ordering LV2
0.1
Picture Ordering LV3
Patient #44
15982
15415
14431
13853
13111
11369
Picture Spelling LV2
10403
9800
9144
8131
8378
7948
7692
7357
Picture Ordering LV5
7135
6332
Picture Ordering LV4
5758
3192
3728
3827
3966
4236
4783
4906
5345
0
Picture Spelling LV3
Picture Spelling LV4
ASHA 2013
39
4. Historical individual performance
Addition LV4
Clock Math LV2
Clock Math LV3
Division LV1
Division LV2
Long Reading Comprehension LV3
Map Reading LV1
Map Reading LV2
Map Reading LV3
Multiplication LV1
Multiplication LV2
Multiplication LV3
Multiplication LV4
Naming Picture LV1
Picture Matching LV2
Picture Matching LV3
Picture Matching LV4
Picture Ordering LV1
Picture Ordering LV2
Picture Ordering LV3
Picture Ordering LV4
Picture Ordering LV5
Picture Spelling LV4
Picture Spelling LV5
Reading Passage LV3
Rhyming LV1
Subtraction LV2
Subtraction LV3
Subtraction LV4
Word Problem LV1
Word Problem LV2
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Patient #913
ASHA 2013
40
5. Patient performance over time relative to
population mean
ASHA 2013
41
ASHA 2013
42
ASHA 2013
43
ASHA 2013
44
6. Group level analysis: Analysis of tasks by
patients- Individual Analysis Quantifying Change
1.2
14
Rhyming
12
1
10
y = 0.005x + 0.7786
R² = 0.0467
0.8
8
Accuracy
Latency
0.6
Linear (Accuracy)
6
0.4
4
y = -0.2369x + 8.6689
R² = 0.4769
0.2
2
0
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
Linear (Latency)
Individual analysis: Quantifying Change
• With tasks that had an R2 value of above 0.25, the
average of the first two sessions was subtracted from the
average of the last two sessions
• Then each of those values was determined to be a “good”
or a “bad” change
• A “good” change in accuracy was any value above 0
• A “good” change in latency was any value below 0
ASHA 2013
47
Individual subject analysis by task
Low Language
profile- Low
cognitive profile
Low language profile- high cognitive profile
High Language
profile- low
cognitive profile
High language profilehigh cognitive profile
ASHA 2013
48
Group analysis of task improvement
Average % Significant (Overall)
% of 19.63% considered to be "Good"
100.00%
90.00%
80.00%
Percent of patients with significant R2 values by task for
accuracy and latency
70.00%
60.00%
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%
Percent of Significant R2
Percent of Significant considered "Good"
19.63%
79.62%
Group analysis of task improvement
Change in GOOD Significant R2
Average
Smallest Change
Largest Change
Percent of significant R2 value
100.00%
90.00%
Accuracy (%)
14.45%
0.48%
74.86%
ACCURACY
80.00%
70.00%
60.00%
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%
Percent of Significant R2
Percent of Significant considered "Good"
Group analysis of task improvement
Change in GOOD Significant
R2
Average
Largest Change
Smallest Change
Percent of significant R2 values
100.00%
Latency (seconds)
0.144538506
0.74861
0.004765
LATENCY
90.00%
80.00%
70.00%
60.00%
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%
Percent of Significant R2
Percent of Significant considered "Good"
52
ASHA 2013
7. Group level analysis- co-factors
Task
• Mixed regression models
for each treatment task
• For e.g., for word
identification
Estimates for Accuracy
Estimate
SE
Label
Conditional improvement
Effect of WABAQ on
improvement
Effect of Composite Severity on
improvement
DF t value Pr > |t|
0.01961
0.0091
294
2.15
0.0325
-0.00066
0.0001
294
-4.88
<.0001
0.01820
0.0137
294
1.32
0.1874
Overall effect of
treatment is
significant
Accuracy
Estimate
Category Identification -0.00633
Category Matching
-0.00019
Feature Matching
0.02356
Letter to sound
matching Level 2
0.01018
Rhyming
0.01521
Word spelling
completion LV 1
0.01456
Word Identification
0.01961
Addition Level 1
0.122
Addition LV2
-0.05748
Addition LV3
-0.1141
Addition LV4
-0.1361
Addition LV5
0.02881
Clock Reading Level 1 0.000816
Clock Math Lv 2
-0.03675
Clock Math LV3
0.01032
Division LV1
0.05175
Division LV4
0.48
Instruction sequencing 0.03433
Map Reading LV1
0.2592
Map Reading LV2
-0.1268
p value
0.5397
0.9641
0.0034
Latency
Estimate
0.1135
-0.05835
-0.1878
P value
0.569
0.2064
0.0109
0.294
0.0032
-0.5329
-0.0223
0.0002
0.6982
0.0187
0.0325
0.3455
0.0307
0.1456
0.1438
0.1826
0.9405
0.9272
0.9718
0.6368
0.9587
0.6964
0.0014
0.1432
-0.09176
0.1173
-5.9979
2.8665
1.6905
-3.1877
0.1198
-0.4324
33.3335
0.3882
0.0183
0.0847
0.0093
0.1655
0.0719
0.8927
0.0016
0.0458
-0.7804
0.7296
618.69
0.3172
4.6205
0.0971
0.09436
0.9073
16.2842<.0001
ASHA 2013
53
7. Can treatment outcome be predicted for
each patient?
Solution for Random Effects
PatientId
23
24
903
913
955
978
1049
1079
1091
1339
Estimate
-0.07216
-0.5612
0.4057
0.3272
0.08395
1.0518
-0.9511
-0.02336
-0.3460
0.4617
Std Err Pred
0.2081
0.1757
0.2544
0.3258
0.2471
0.2442
0.1953
0.2463
0.2881
0.2044
DF
589
589
589
589
589
589
589
589
589
589
t Value
-0.35
-3.19
1.59
1.00
0.34
4.31
-4.87
-0.09
-1.20
2.26
Pr > |t|
0.7289
0.0015
0.1113
0.3157
0.7342
<.0001
<.0001
0.9245
0.2303
0.0243
This patient show
significantly less
improvement than
the average
improvement
This patient show
significantly more
improvement than
the average
improvement
54
ASHA 2013
8. Mean Changes on standardized test
performance
14
12
Changes in standardized testing scores: experiemtnal vs.
control
Average Change in Score (Post-Pre)
10
Average Control Change
(n=9)
Average TX Change (n=40)
8
6
4
2
0
-2
-4
WAB
CLQT
BNT
55
ASHA 2013
8. Change on standardized tests
Subtest
Experimental Group
(N= 40)
Control Group (N = 9)
WAB-LQ
2.13% (t = -2.05, p <.05)
1.42% (t = 1.07, ns)
WAB-CQ
2.60% (t = -2.05, p <.05)
1.32% (t = 1.03, ns)
WAB-AQ
4.14% ( t = 3.11, p <.01)
0.65% (t = .76, ns)
CLQT-composite severity
9.12% (t = 3.28, p < .01)
4.44 % (t = .76, ns)
CLQT-Attention
11.3 % (t = -.4.55, p
<.0001)
7.6% (t = -1.19, ns)
CLQT-Memory
1.55% (t = -0.84, ns)
1.140 (t = 0.30, ns)
CLQT-Executive Function
6.41% (t = 3.15, p < .01)
1.66 (t = 0.52, ns)
CLQT- Language
1.42% (t = 1.15, ns)
1.65 (t = 1.26, ns)
CLQT- Visuospatial skills
7.81 (t = 3.43, p < .001)
2.96 (t = 0.61, ns)
ASHA 2013
56
Summary of results
• Experimental and control patients completed 11 weeks of
•
•
•
•
•
treatment
Patient compliance was high-ranged from 1 time/week to 11
times/week
Patients who logged in more often showed more changes on
tasks assigned
Individual patient analysis by items, task, weekly- gives
insight into how patients perform
Changes in accuracy and latency are across tasks
Changes seen on standardized tests for experimental
patients, less for control patients
ASHA 2013
THANK YOU !
Questions?
57
ASHA 2013
58
References
• Barnes D, Yaffe K, Belfor N, Jagust W, DeCarli C, Reed B, Kramer J.
Computer-Based Cognitive Training for Mild Cognitive Impairment: Results
from a Pilot Randomized, Controlled Trial. Alzheimer Disease and Associated
Disorders 2009; 23(3), 205-210.
• Finn M, McDonald S. Computerised cognitive training for older persons with
mild cognitive impairment: A pilot study using a randomised controlled trial
design. Brain Impairment 2011; 12(3), 187–199.
• Johansson B, Tornmalm M. Working memory training for patients with
acquired brain injury: Effects in daily life. Scandanavian Journal of
Occupational Therapy 2012; 19(2), 176-83.
doi:10.3109/11038128.2011.603352.
• Lundqvist A, Grundström K, Samuelsson K, Rönnberg J Computerized
training of working memory in a group of patients suffering from acquired
brain injury. Brain Injury 2010; 24, 1173-1183.
doi:10.3109/02699052.2010.498007