Memory Screening: Testing
Protocols for Detection and
Monitoring
J. Wesson Ashford, M.D., Ph.D.
Clinical Professor (affiliated),
Department of Psychiatry and Behavioral Sciences
Senior Research Scientist,
Stanford / VA Alzheimer’s Disease and Aging Clinical Research Center
Stanford University and VA Palo Alto Health Care System
Chair, Alzheimer’s Foundation of America, National Memory
Screening Advisory Board
February 26, 2016
Slides at: www.medafile.com (Dr. Ashford’s lectures)
Financial Disclosure
• Dr. Ashford has been developing memory tests for
assessment of neuropsychiatric illnesses since 1979.
• Dr. Ashford has never made any money from developing
these tests, though he has spent a small amount of funds
maintaining websites to make some of these tests available
for free: www.medafile.com .
• Dr. Ashford’s son, Curtis Ashford, has a company, MemTrax,
LLC, which owns the trademark, MemTrax.
• MemTrax, LLC is working to commercialize a Continuous
Recognition Test: www.memtrax.com
• The USA Internal Revenue Service considers Dr. Ashford’s
activities with respect to MemTrax to be a “hobby”
3
AFA Resources and Support
• AFA’s National Toll-free Helpline
– (phone, email, Skype, chat) – 866-232-8484
• Dementia Care Professionals of America- training and
certification
• Care Connection monthly teleconference
• National Memory Screening Program
• Educational training, resources, and conferences
• AFA Care Quarterly magazine
• AFA unites 2,400+ member organizations nationwide that
provide direct resources and care
www.alzfdn.org
866-232-8484
Dementia is Under-diagnosed
• Early dementia is easily missed
–
–
–
–
especially Alzheimer’s disease (AD)
the problem is subtle – MCI – mild cognitive impairment
family members avoid the problem and compensate for the patient
physicians tend to miss the initial signs and symptoms
• about 90% of AD cases are missed early
• Many AD patients are never recognized for their disease
– Estimates are that 25% to 50% of cases remain undiagnosed
– Diagnoses are still missed at moderate and severe levels
• No definitive laboratory test for diagnosing AD exists
– Efforts to develop biomarkers – none currently accepted
– Early recognition by brain scan is confusing (may rule out AD)
Reasons to Diagnose Alzheimer’s Disease Early
Social / Financial
• Undiagnosed patients face avoidable problems:
– social, financial problems
• Early education of caregivers of how to handle patient
• Advance planning while patient is competent
– will, proxy, power of attorney, advance directives
• Reduce family stress and misunderstanding (blame, denial)
• Promote safety (driving, compliance, cooking, etc.)
• Patient’s and family’s right to know, including genetic risks
• Opportunity to reduce caregiver burden
• Promote advocacy for research and treatment development
Reasons to Diagnose Alzheimer’s Disease Early
Medical
• Early diagnosis and appropriate intervention may lessen
disease burden and early treatment may improve overall
course substantially
– Neurophysiological pathways in patients with AD are still
viable and are a target for treatment
• Specific treatments now available (after dementia diagnosis)
– anti-cholinesterases, memantine)
– Improve cognition
– Improve function (ADLs)
– Slow dementing process, earlier treatment is better
– Decrease development of behavior problems
– Delay nursing home placement, possibly over 20 months
– Delay nursing home placement longer if started earlier
Introducing the time-index model
of the course of Alzheimer’s disease
Estimate MMSE as a function of time
(calculated from the CERAD data set)
MMSE score
30
25
20
15
10
5
0
The best model to fit the progression,
both mathematically and biologically,
is the Gompertz survival curve
(99.7% fit to mean changes over time):
S(t) = exp(Ro/alpha *(1- exp (alpha * t)))
-10
-8
-6
-4
-2
0
2
Time-Index Scale
4
6
8
10
Estimated years into illness
AAMI / MCI/ early AD -- DEMENTIA
Ashford et al., 1995
Total Item Information Function for the MMSE
30
Information
25
20
15
10
5
0
-4 -3 -2 -1
0
1
2
3
4
5
6
7
8
Alzheimer's Severity Horographic Function (time-index year units)
9 10
AD all (easiest to hardest at p=.5)
Mini-Mental State Exam items
PROBABILITY CORRECT
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
-4 -3 -2 -1
0
1
2
3
4
5
6
7
8
DISABILITY ("time-index" year units)
9 10
PENCIL
APPL-REP
WATC
LOCATION
PENY-REP
TABL-REP
CLOS-IS
RIT-HAND
CITY
FOLD-HLF
SENTENCE
COUNTY
NO-IFS
FLOOR
SEASON
YEAR
PUT-LAP
MONTH
ADDRESS
DRAW-PNT
DAY
SPEL_ALL
DATE
APPL-MEM
PENY-MEM
TABL-MEM
Problems with the MMSE
• Mini-Mental State Exam
– Folstein et al., 1975 (antique)
• Considerable noise
• Several items do not provide adequate information
• Poor range for measuring change
– Large standard error of measurement
• Poor power for assessing medication benefit
• Inadequate screening tool
• Better, shorter tests are available
• Now, copyright is being enforced (not free!!)
Launched by the Alzheimer’s Foundation of America (AFA) in 2003, National
Memory Screening enables thousands of local sites
nationwide to provide free, confidential memory screenings to individuals
concerned about memory loss with the objective of
early detection and intervention.
•
•
Since the inception of the program, over 2.5 million people
have been screened.
Memory screenings are taking place in all 50 states and Puerto Rico.
Screenings take place year round and during National Memory
Screening Month (which is November in conjunction with Alzheimer’s
Awareness Month)
Memory screenings:
 Free and confidential
 Take approximately 10 minutes.
 Consist of a series of questions to gauge
memory, language and thinking skills.
 Results are not a diagnosis, but a memory
screening can suggest if someone should
see a physician for a full evaluation.
To search for a screening site in your area visit:
www.nationalmemoryscreening.org
To register to become a memory screening site visit:
www.afascreenings.org/register
AFA provides sites with all of the necessary
materials to carry out the screenings- free of charge!
What is the reason you came today?
(Check all that apply)
(Voluntary Participant Survey – NMSD 2015)
I have noticed changes in my memory over the last year
I feel I have more problems with my memory than most other
people my age
My family or friends have encouraged me to get screened
42%
I have relatives with Alzheimer’s disease or dementia
20%
11%
10%
I have been a caregiver for someone with AD or dementia
8%
I have been diagnosed with memory loss
2%
I feel regular memory screening is important
40%
Other
13%
Have you ever had a memory
test before?
(Voluntary Participant Survey – NMSD 2015)
No I have not had a memory test before
At a NMSD in the past
At a health fair
78%
3%
2.5%
In my doctor’s office
7%
On a computer
1%
Other
6%
CLINICAL TOOLS FOR
COGNITIVE SCREENING
116 listed by NIA, including:
•
•
•
•
•
•
•
•
•
•
MINI-MENTAL STATE EXAM
MIS (Memory Impairment Screen)
Mini-Cog
GP-Cog
Brief Alzheimer Screen (BAS)
7-Minute Screen
MOCA (Montreal Cognitive Assessment)
SLUMS (St. Louis Mental Status test)
GLOBAL CLINICAL SCALE
Continuous Recognition Test
https://www.nia.nih.gov/research/cognitive-instrument
National Memory Screening Day
NMSD - 2010
• 2334 sites participated
• e.g., Alzheimer’s agencies, pharmacies, hospitals,
community centers
• estimated 60,000 individuals were screened
• 48 sites agreed to provide data to the AFA
• Information was provided on 4,396 individuals
• A mean of 55 +/- 34 participants were seen at
each site (range 5–159)
Bayley et al. 2015
Demographic Characteristics
of Participants- NMSD - 2010
Characteristic n (%)
Sex
Male 1,257 (28.8)
Female 3,109 (71.2)
Age
<35:
40 (0.9)
35–44: 57 (1.3)
55–64: 604 (13.8)
65–74: 1,330 (30.5)
75–84: 1,524 (34.9)
>85:
617 (14.1)
Black 320 (7.4)
Other 221 (5.1)
> High School
356 (8.3)
Bachelor’s degree
865 (20.2)
Postbachelor’s
1,192 (27.9)
45–54: 190 (4.4)
Race
White 3,757 (87.4)
Hispanic 188 (4.3)
Education
Elementary
197 (4.5)
High school
1,666 (39.0)
Tests Administered
(number, percent)
GP-Cog - General Practitioner assessment of
Cognition
1369
45%
Mini-Cog
686
23%
MIS - Memory Impairment Screen
249
8%
Mini-Mental State Examination
452
15%
71
2%
201
7%
9
0%
Kokmen Short Test of Mental Status
MoCA - Montreal Cognitive Assessment
SLUMS - Saint Louis University Mental Status
Examination
Time to Administer Available Short
Screening Tests
• MMSE
10 -- 15 min
• Too long, poor screener
• 7-Minute Screen
7 – 10 min
• Too complex
• Brief Alzheimer Screen (BAS)
2 – 4 min
• High sensitivity, specificity, only one form
• Mini-cog
3 – 5 min
• Unclear sensitivity,specificity
• Memory Impairment Screen
4 min
• Need shorter, easier to perform test
• (a suitably accurate test that takes less than
2 minutes is not available)
BRIEF ALZHEIMER SCREEN
(Normal vs Mild AD, MMS>19)
20
True Positive Rate (%) (Sensitivity)
100
27
90
26
25
80
14
13
12
11
10
70
9
60
8
animals 1 m
AUC = 0.868
animals 30 s
AUC = 0.828
MMSE
AUC = 0.965
20
Date+3 Rec
AUC = 0.875
10
BAS
AUC = 0.983
50
40
97
30
6
0
0
10
20
30
40
50
60
70
80
False Positive Rate (%) (1-Specificity)
JW Ashford, MD PhD, 2003
90 100
Brief Alzheimer Screen (BAS) ROC for Univ. Kentucky ADRC Clinic
Cases
Schmitt et al., 2006
Factors for Deciding whether
a Screening Test is Cost-Effective
1) Benefit of a true positive screen
2) Benefit of a true negative screen
3) Cost of a false positive screen (? Harm)
4) Cost of a false negative screen (? Harm)
5) Incidence of the disease (in population –
increasing)
6) Test sensitivity (in population)
7) Test specificity (in population)
8) Test cost (include cost of test administrator)
$W = Cost–Worthiness Calculation
$W > ($B x I x Se) – ($C x (1-I) x (1-Sp)) - $T
•
BENEFIT
– $B = benefit of a true positive diagnosis
• Earlier diagnosis may mean proportionally greater savings
• Estimate: (100 years – age ) x $1000
• Save up to $50,000 (e.g., nursing home cost for 1 year)
– (after treatment cost deduction at age 50, none at age 100)
– (cost-savings may vary according to your locale)
•
•
•
•
•
– True negative = real peace of mind (no money)
COST
– $C = cost of a false positive diagnosis
• $500 for further evaluation
– (time, stress of suspecting dementia)
– False negative = false peace of mind (no price)
I = incidence (new occurrences each year, by age)
Se = sensitivity of test = True positive / I
Sp = specificity of test = True negative / (1-I) = (1-False positive/(1-I)
$T = cost of test, time to take (Subject, Tester)
Kraemer, Evaluating Medical Tests, Sage, 1992
Cost Justified for
Dementia Screen
Cost-Worthy Test Evaluation
Benefit = $50,000 - 0; False Pos = $500
600
550
500
450
400
350
300
250
200
150
100
50
0
-50
-100
Se, Sp
.8, .8
.9, .9
.95, .95
1,1
50 55 60 65 70 75 80 85 90 95
AGE
Need to Develop Better Screening
and Early Assessment Tools
•
•
•
•
Genetic vulnerability testing (trait risk)
Vulnerability factors (education, occupation, head injury)
Early recognition (10 warning signs), ADLs
Positive diagnostic tests
– CSF – tau levels elevated, amyloid levels low
– Brain scan – PET – amyloid, tau ligands
• Screening tools (6th vital sign in elderly)
• Mild Cognitive Impairment assessments
• Detecting early change over time
– predicting progression, measuring rate
Cognitive Impairment Assessment
• Detect Alzheimer’s disease (universal screening test)
• Also detect: fronto-temporal dementia, mild traumatic brain injury,
intoxication, safety to operate machinery & drive, mild cognitive
impairment, other types of dementia, chemobrain, etc.)
• Be on multiple platforms:
– Computer for rapid, objective assessment
– World-Wide Web – for testing anywhere
– Doctor’s offices
– Smart-phone, tablets (HTML-5)
– KIOSK administration – drug stores, shopping malls
• Be very brief (1 to 2 minutes), but have high sensitivity, specificity
• Have multiple test forms so it can be repeated often
• Detect change over time (monthly, weekly, daily, hourly testing) (infinite
number of comparable forms with high test-retest consistency)
Audience Screening using a
Continuous Recognition Test (CRT)
• CRT is widely used academically for memory study
• Presentation of complex pictures (that are easily
remembered normally) are useful for detecting
memory difficulties
• Testing memory using a pictures approach can be
standardized for population use
• Picture memory is less affected by education
• Picture memory can be tested by computer
• Audiences can be shown slide presentations
The relationship between discriminability (d′) and age
on the audience-based continuous recognition test of
memory for 868 individuals with all information
4
3
d'
2
1
0
40
-1
60
Age (years)
80
100
Ashford et al., 2011
The relationship between discriminability
performance (d′) and age in 868 individuals on
the continuous recognition test of memory
d' ± 1 SEM
4
3
2
1
29
68
135
40-49
50-59
60-69
239
359
38
80-89
90-99
0
70-79
Age (years)
The relationship between discriminability
performance (d′) and age in 868 individuals on
the continuous recognition test of memory.
d' ± 1 SD
4
3
2
1
29
68
135
239
359
38
0
40-49
50-59
60-69
70-79
Age (years)
80-89
90-99
MemTrax - Memory Test
(on-line version)
•
•
•
•
•
•
•
•
•
Test to screen for memory function and change
Available on any platform
Determines percent correct and recognition time
Measures episodic memory, inhibition
Test takes 1 to 2 minutes
Test can be repeated often (even hourly)
High test-retest consistency
Minimal learning over repeated tests
Change over time sensitively detected
MemTrax Test Development
• Based on the continuous recognition task paradigm
Categories
Mountain Fountain
Art
House Birdhouse
1
3
s
3
s 3
s
Pic1_CArt1_Repeat0
Pic2_CMountain1_Repeat
Pic3_C0 Fountain1_Repeat
Pic4_C0 Art1_Repeat1
Pic5_CHouse1_Repeat0
Pic6_C3
s 3
s
Birdhouse1_Repea
t0
3
s
3
s 3
s
2
3
4
5
Pic26_CArt1_Repeat2
Pic27_CMountain1_Repeat2
Pic49_CMountain5_Repeat1
Pic50_CArt5_Repeat1
3
s
3
s
MemTrax Test Data
• Data collection period: September 2011 - August 2013
• Collected in collaboration with HAPPYneuron, Inc. (Lyon, France)
• 4,800 repeat administratons (over 24 months, up to 24 times)
• Analysis approved by Stanford IRB
• Total number of tests taken: 30,435
• Tests excluded: 12,153
– Repeat administrations: 4,800 (over 24 months, up to 24 times)
– Age below 21: 4,999
(no IRB approval for under 21 analyses)
– Age above 100: 1,573 (unrealistic number)
– Performance outliers: 780 (invalid scores - less than 70% correct
• Total valid, initial tests: 18,282
– Females: 12,445
– Males: 5,837
• Mean age: 50.28 years, SD: 15.23, Range 21-99 years
Data Analysis
(25 initial presentations, 25 repeats)
(males, females separated)
-
Number of subjects by age
Total number responses (similar to beta)
Number of correct responses (similar to d’)
Cumulative number of correct responses
Percent correct versus age
Recognition time versus age
Recognition time versus percent correct
Number wrong and +2STDs versus age
Recognition time and +2STDs versus age
Number Wrong, m vs f, mean + 2SD
16
15
14
13
12
11
y = 0.0004x2 - 0.0183x + 2.5847
y = 0.001x2 - 0.0577x + 7.9844
R² = 0.6824
R² = 0.4372
y = 0.0008x2 - 0.0536x + 3.4393
y = 0.0017x2 - 0.1309x + 9.5718
R² = 0.6321
10
R² = 0.4285
9
8
7
6
5
4
3
2
1
0
20
30
40
50
60
AGE (years)
70
80
90
Recognition Time, m vs f, mean + 2SD
1600
y = 0.0679x2 - 4.403x + 932.75
Recognition Time (msecs)
1500
R² = 0.925
1400
y = 0.0514x2 - 3.2768x + 927.19
1300
R² = 0.8199
1200
1100
1000
900
800
20
30
40
50
60
AGE (years)
70
80
90
Positives versus Negatives
www.memtrax.com
•
•
•
•
•
•
•
•
•
•
Very Fast (less than 2 minutes)
Easy (explanation can be understood)
Innovative (based on extensive memory studies)
Tests complex encoding (unlike any other available test)
Fun (subjects very willing to retake tests frequently)
Reliable (test works on many platforms)
Inexpensive (after development, cost is negligible)
Infinite number of repeatable tests (test-retest reliability)
Validity - face, construct (measures memory encoding)
Normative data (collected in large quantities)
•
•
•
•
•
•
Does not test remote memory (but does have cultural sensitivity)
Human monitoring optional (can be taken without supervision)
Requires access to internet enabled device (at present)
Computers cannot yet make diagnosis (may assist)
Discriminant/concurrent validity lacking (need to establish)
Distraction could influence test results
Needs for Better Memory Testing
Clinicians/Professionals/Researchers
• Testing & Monitoring
– Individuals
– Family members
– Friends
– Care givers
• Researchers
– Pre-post intervention trials
– Cognitive studies
– Adaptable for fMRI studies
Future Directions for Testing
• Precise assessment of cognition from normal to
moderate dementia
• Item Response Theory and Computerized
Adaptive Testing (on-line update of analyses)
• Expansion to measure numerous functions
• Analysis of tests for risk/benefit (cost-worthiness)
• Expansion of applications – driving, various types
of cognitive impairment, hospital monitoring
• Use for recruitment of subjects for AD research
– MemTrax being used by www.brainhealthregistry.org
New Applications for
Memory Testing
• Medicare Annual Wellness Visit cognitive assessment
• 6th vital sign for health-provider visits
• Pharmacist assures client is able to understand directions
for prescribed medication
• Evaluate potential for learning in subject beginning
rehabilitation therapy (stroke, etc.)
• Assessment of anesthesia recovery
• Contact sport side-line assessment of concussion
• Determine if individual is safe to operate heavy
machinery, drive