Scientific Summary
UC Davis / SENAS (Spanish and English
Neuropsychological Assessment Scales)
Friday Harbor Psychometrics 2012
Acknowledgements
• Funded in part by Grant R13AG030995-01A1
from the National Institute on Aging
• The views expressed in written conference
materials or publications and by speakers and
moderators do not necessarily reflect the official
policies of the Department of Health and Human
Services; nor does mention by trade names,
commercial practices, or organizations imply
endorsement by the U.S. Government.
Friday Harbor Psychometrics Workshop 2011
Overview
•
•
•
•
•
•
•
•
SENAS Overview
Demographic variables and longitudinal change
Ethnicity, clinical diagnosis, and cognitive change
Age, education, and relationship to brain
Decomposing demographic and brain effects
Intracranial volume and cognition
Factorial invariance
Final thoughts
Friday Harbor Psychometrics 2012
SENAS Overview
Friday Harbor Psychometrics 2012
Purpose of SENAS Project (circa 1992)
• Create matched English and Spanish language
neuropsychological tests for ages 60+
 New scales based upon neuropsychological model of
cognitive functioning
 Scales psychometrically matched
• Within English and Spanish language versions
• Between English and Spanish language versions
 Distribution of item difficulty appropriate to elderly
population
Friday Harbor Psychometrics 2012
Approach
• Concept of item and test bias central to project
• Development and validation based on modern
psychometric methods
 Item response theory (IRT)
 Latent variable modeling
• Empirically based
 Data based approach to making decisions about item
selection and scale construction
 Empirical evaluation of reliability and validity
Friday Harbor Psychometrics 2012
Guiding Principles
Initial Scale Development
• Scales targeted to assess neuropsychologically relevant
cognitive domains
• Verbal and non-verbal measures
• Non-timed
• New scales, not translations of existing scales
• Examiner administered
• Item generation
• New items
• Broad range of difficulty
Friday Harbor Psychometrics 2012
SENAS Scales and Abilities Measured
Ability
Verbal Measure
Non-Verbal Measure
Conceptual Thinking
Verbal Conceptual Thinking
Non-Verbal Conceptual
Thinking
Semantic Memory
Object Naming
Picture Association
Attention Span
Verbal Attention Span
Visual Attention Span
Episodic Memory
Word List Learning
Spatial Configuration Learning
Spatial Abilities
Pattern Recognition
Spatial Localization
Verbal Abilities
Verbal Comprehension
Verbal Expression
Executive Function
Category Fluency
Letter/Phoneme Fluency
Working Memory
Friday Harbor Psychometrics 2012
Working Memory
Model for Item Selection
Friday Harbor Psychometrics 2012
Test Information for 3MS and SENAS
Object Naming
Friday Harbor Psychometrics 2012
Mungas et al., 2004
Scientific Applications
Friday Harbor Psychometrics 2012
Summary Conclusions / Challenge for
Conference
• Demographic variables have robust effects on
baseline test scores
 Especially ethnicity and education
• Demographic variables have minimal effects on
longitudinal change
 Brain and disease variables account for change
Friday Harbor Psychometrics 2012
Community
Caucasian
English
N ≈ 235
Caucasian
N ≈ 700
Cognition
N≈
350
African American
N ≈ 425
Hispanic English
N ≈ 450
Clinical
African American
N ≈ 140
Hispanic
N ≈ 130
Longitudinal
N ≈ 175
Hispanic Spanish
N ≈ 900
Hispanic
Friday Harbor Psychometrics 2012
MRI
Clinic
Morphometric
Life
Experience
Measures
Ethnicity, education, and cognitive change
Friday Harbor Psychometrics 2012
Purpose of this study
• Understand how demographic variables relate to cognitive
change
• Demographic variables of interest
 Race/Ethnicity
 Education
 Language of test administration
Friday Harbor Psychometrics 2012
Independent Variables
• Model 1




Verbal Memory Form
Previous Evaluation
Spanish Administration
Prev Eval by Spanish
Interaction
• Model 3
 Age & Education
 Gender
Friday Harbor Psychometrics 2012
• Model 2
 Race/Ethnicity
• Model 4
 Recruitment Source
 Clinical Diagnosis
 APOE Genotype
Language, Ethnicity, Education Effects
Episodic Memory
Baseline
Effect
Estimate
Intercept
.62
Change
SE
Estimate
SE
.08
Time
-.04
.02
Previous
Eval
-.06
.04
-.04
.04
.33
.12
Language
(Spanish)
-.30
.13
Span x
PrevEval
Hispanic
-.29
.11
.04
.03
African
American
-.06
.08
.01
.02
Education
.03
.01
-.00
.00
Friday Harbor Psychometrics 2012
Language, Ethnicity, Education Effects
Executive Function
Baseline
Effect
Estimate
SE
Intercept
.32
.07
Change
Estimate
SE
Time
-.03
.02
Previous
Eval
.01
.03
-.01
.03
.08
.07
Language
(Spanish)
-.08
.11
Span x
PrevEval
Hispanic
-.32
.09
.02
.03
African
American
-.30
.07
.04
.02
Education
.05
.01
-.00
.00
Friday Harbor Psychometrics 2012
Education:
0.1
0.2
0.3
0.4
9 Years
12 Years
15 Years
0.0
Expected Executive Function Score (standardized)
0.5
Executive Function Trajectories by Education
0
1
2
Time (years)
Friday Harbor Psychometrics 2012
3
4
Average Ethnic Group Performance by
Model – Episodic Memory
Friday Harbor Psychometrics 2012
Ethnicity, clinical diagnosis, and cognitive
change
Friday Harbor Psychometrics 2012
Episodic Memory by Diagnosis Change
Episodic Memory
Group:
1.0
0.5
0.0
−0.5
−1.0
−2.0
−1.5
Expected Test Score (standardized)
MCI−MCI (n=68)
MCI−Demented (n=77)
Demented (n=72)
Normal (n=209)
Normal−MCI (n=45)
MCI−Normal (n=21)
0
1
2
3
4
5
Time (years)
Friday Harbor Psychometrics 2012
Mungas et al., 2010
Episodic Memory by Diagnosis Change
African American
Episodic Memory − African American
Group:
1.0
0.5
0.0
−0.5
−1.0
−2.0
−1.5
Expected Test Score (standardized)
MCI−Stable (n=31)
MCI−Demented (n=25)
Normal (n=69)
Normal−MCI (n=12)
0
1
2
3
Time (years)
Friday Harbor Psychometrics 2012
4
5
Episodic Memory by Diagnosis Change
Hispanic
Episodic Memory − Hispanic
Group:
1.0
0.5
0.0
−0.5
−1.0
−2.0
−1.5
Expected Test Score (standardized)
MCI−Stable (n=19)
MCI−Demented (n=24)
Normal (n=65)
Normal−MCI (n=16)
0
1
2
3
Time (years)
Friday Harbor Psychometrics 2012
4
5
Episodic Memory by Diagnosis Change
Caucasian
Episodic Memory − Caucasian
Group:
1.0
0.5
0.0
−0.5
−1.0
−2.0
−1.5
Expected Test Score (standardized)
MCI−Stable (n=39)
MCI−Demented (n=100)
Normal (n=75)
Normal−MCI (n=17)
0
1
2
3
Time (years)
Friday Harbor Psychometrics 2012
4
5
Age, education, and relationship to brain
Friday Harbor Psychometrics 2012
MRI Effect Sizes and Age and Education
Adjustment - Episodic Memory
Variance Explained
(R-squared)
Age Adjusted
No Adjustment
Education Adjustment
0.40
0.35
0.30
0.25
0.20
0.15
0.10
0.05
0.00
Combined
(n=315)
Caucasian
(n=134)
African
American
(n=79)
Hispanic
(n=102)
Sample
Friday Harbor Psychometrics 2012
Mungas et al., 2009
Executive Function, MRI, Age & Education
African Americans
-.24
Age
-.55
BM
.00
Education
.36
-.03
.32
-.26
.39
Executive Function
Friday Harbor Psychometrics 2012
Mungas et al., 2009
Executive Function, MRI, Age & Education
Hispanics
.01
Age
-.60
BM
.00
Education
.30
-.17
.46
-.40
.44
Executive Function
Friday Harbor Psychometrics 2012
Mungas et al., 2009
Executive Function, MRI, Age & Education
Caucasians
-.07
Age
-.53
BM
.00
Education
.29
-.10
.29
-.36
.28
Executive Function
Friday Harbor Psychometrics 2012
Mungas et al., 2009
Age and Education Influences on
MRI - Cognition Relationships
• It is possible to separate disease effects from
extraneous influences
• MRI effects on cognition can obscured by
demographic effects on test performance,
especially when
• Demographic relationship with test score is
larger than relationship with disease
• Substantial heterogeneity of demographic
variable in population of interest
Friday Harbor Psychometrics 2012
Decomposing demographic and brain
effects
Friday Harbor Psychometrics 2012
Reed et al., 2010
Friday Harbor Psychometrics 2012
Data
• 288 subjects. 158 normal, 92 MCI, 38 demented
•
96 African Americans, 74 Hispanics (32 tested in English, 42 tested in
Spanish), and 118 Caucasians.
• Education M = 12.7 yrs (range 0-25)
• Age M = 74.7 yrs (range 60-93)
• Mean evaluations = 3.5; 74% had 3 or more evaluations. N of
evaluations truncated at 5.
Friday Harbor Psychometrics 2012
Reed et al., 2010
• Mem-D ~ 20% of episodic memory variance
• Mem-B ~ 20% of episodic memory variance
• Mem-R ~ 50% of episodic memory variance
Friday Harbor Psychometrics 2012
Reed et al., 2010
Relationships of memory components with global
cognitive function (CDR sum of boxes)
DV
CDR Sum
Memory Standardized
Component Coefficient
p
Mem-D
-0.09
ns
Mem-B
Mem-R
-0.43
-0.44
0.001
0.001
Friday Harbor Psychometrics 2012
Reed et al., 2010
Relationships of memory components with clinical
progression (conversion to MCI or dementia)
Memory
Component
Mem-D
Mem-B
Mem-R
Friday Harbor Psychometrics 2012
Relative Risk Ratio
(confidence interval)
1.58 (0.92 - 2.71)
0.19 (0.11 - 0.33)
0.27 (0.18 - 0.40)
Reed et al., 2010
Relationships of memory components with longitudinal
change in cognition (executive function)
Memory
Component
Mem-D
Random
Effect
baseline
Parameter
0.364
Standard p
Error
0.043
0.001
Mem-B
baseline
0.180
0.031
0.001
Mem-R
baseline
0.329
0.034
0.001
Mem-D
Mem-B
change
change
-0.011
0.050
0.010
0.010
ns
0.001
Mem-R
change
0.047
0.011
0.001
Friday Harbor Psychometrics 2012
Reed et al., 2010
Effects of Mem-D?
• Substantial variance in Episodic Memory was uniquely
related to demographic variables (ethnicity and education
especially)
 Greater amount for other cognitive domains
• Mem-D was minimally related to clinical outcomes
 Especially longitudinal decline and conversion
• Challenge in clinical neuropsychological assessment is to
separate demographic from brain influences on test scores
Friday Harbor Psychometrics 2012
Reed et al., 2010
Intracranial volume and cognition
Friday Harbor Psychometrics 2012
Farias et al., 2012
Model of joint effects of intracranial
volume and brain structure
Friday Harbor Psychometrics 2012
Farias et al., 2012
Semantic Memory by ICV
Friday Harbor Psychometrics 2012
Farias et al., 2012
Cognition and ICV
• ICV related to Semantic Memory and Executive
Function
• ICV not related to Episodic Memory
• Early development might have impact on cortical
development
Friday Harbor Psychometrics 2012
Farias et al., 2012
Factorial invariance
Friday Harbor Psychometrics 2012
Sample
• Community dwelling recruited using cognitive
screening protocol and clinic referrals
 Whites - n=678
 Blacks - n=352
 Hispanics, English Speaking n=434
 Hispanics, Spanish Speaking n=877
• Broad range of cognitive function
 Normal to demented
Friday Harbor Psychometrics 2012
Mungas et al., 2011
Best Factor Structure
Object Naming
Spatial
Picture Association
Verbal Abstraction
Pattern Recognition
Verbal
Verbal Expression
Verbal Comprehension
Word List Learning 1
Friday Harbor Psychometrics 2012
NonVerbal Reasoning
Visual Attention
Attention
Memory
Word List Learning 2
Spatial Config Learning
Spatial Localization
Verbal Attention
Working Memory
Category Fluency
Fluency
Phonemic Fluency
Mungas et al., 2011
Invariance of Dimensional Structure
Across Ethnic and Language Groups
• Same number of dimensions
• Invariant Factor Loadings
 Observed test scores have same relationship to
latent dimensions
• Some differences in Intercepts for tests
 Spanish speaker has to have greater latent
attention ability to achieve a given Verbal
Attention score
Friday Harbor Psychometrics 2012
Mungas et al., 2011
Verbal Attention by Attention/Working
Memory
Caucasians
Hispanics: English-speakers
African Americans
Hispanics: Spanish-speakers
3
VRATT
2
1
0
-1
-2
-3
-2
-1
0
1
2
ATTN/WM
Friday Harbor Psychometrics 2012
Mungas et al., 2011
3
Final Thoughts
Friday Harbor Psychometrics 2012
Longitudinal Trajectories
Friday Harbor Psychometrics 2012
Longitudinal Trajectories
Friday Harbor Psychometrics 2012
Conclusions
• Cross-sectional results can be misleading
 Confound lifelong ability and disease effects
• Longitudinal assessment important to separate
these effects
 Ideally beginning in mid adulthood
 In absence of ideal – evidence that
demographic effects on cognitive decline are
small and distal
• Don’t assume a low score is impairment
Friday Harbor Psychometrics 2012
References
• Farias, S. T., D. Mungas, et al. (2012). "Maximal brain size remains an
important predictor of cognition in old age, independent of current
brain pathology." Neurobiol Aging 33(8): 1758-1768.
• Mungas, D., B. R. Reed, et al. (2004). "Spanish and English
Neuropsychological Assessment Scales (SENAS): Further
development and psychometric characteristics." Psychological
Assessment 16(4): 347-359.
• Mungas, D., B. R. Reed, et al. (2005). "Spanish and English
Neuropsychological Assessment Scales: relationship to demographics,
language, cognition, and independent function." Neuropsychology
19(4): 466-475.
Friday Harbor Psychometrics 2012
References
• Mungas, D., B. R. Reed, et al. (2009). "Age and education effects on
relationships of cognitive test scores with brain structure in
demographically diverse older persons." Psychology and Aging 24(1):
116-128.
• Mungas, D., L. Beckett, et al. (2010). "Heterogeneity of cognitive
trajectories in diverse older persons." Psychol Aging 25(3): 606-619.
• Mungas, D., K. F. Widaman, et al. (2011). "Measurement invariance
of neuropsychological tests in diverse older persons."
Neuropsychology 25(2): 260-269.
• Reed, B. R., D. Mungas, et al. (2010). "Measuring cognitive reserve
based on the decomposition of episodic memory variance." Brain 133:
2196-2209.
Friday Harbor Psychometrics 2012
Ethnic Differences and DIF
• SENAS Picture Association and Verbal Learning
• English Administration - N = 1113
 396 Hispanics, 480 Whites, 237 Blacks
• Spanish Administration - N = 801
• Ability adjusted for all-source DIF
 Education, Ethnicity, Age
• MIMIC approach
Friday Harbor Psychometrics 2012
Picture Association
Effects of DIF Adjustment
Friday Harbor Psychometrics 2012
Word List Learning
Effects of DIF Adjustment
Friday Harbor Psychometrics 2012
Effect of DIF Adjustment on Individual
Scores (95% confidence interval)
Picture
Association
Word List
Learning
White
±.29 s.d.
±.10 s.d.
Black
±.41 s.d.
±.02 s.d.
Hispanic - English
±.23 s.d.
±.00 s.d.
Hispanic - Spanish
±.36 s.d.
±.06 s.d.
Friday Harbor Psychometrics 2012
SENAS and DIF
• DIF accounted for a very small amount of ethnic
differences in Object Naming, Picture Association,
Word List Learning
• DIF effects differ by domain
 Greater for measures of semantic memory
 Smaller for episodic memory
• DIF effects can potentially bias interpretation of
individuals scores
 Especially in borderzone areas
Friday Harbor Psychometrics 2012
Cross-sectional ethnic differences can be
explained by confounding variables
Friday Harbor Psychometrics 2012
Semantic Memory
Mean Ethnic Group Differences
Friday Harbor Psychometrics 2012