Scientific Summary UC Davis / SENAS (Spanish and English Neuropsychological Assessment Scales) Friday Harbor Psychometrics 2012 Acknowledgements • Funded in part by Grant R13AG030995-01A1 from the National Institute on Aging • The views expressed in written conference materials or publications and by speakers and moderators do not necessarily reflect the official policies of the Department of Health and Human Services; nor does mention by trade names, commercial practices, or organizations imply endorsement by the U.S. Government. Friday Harbor Psychometrics Workshop 2011 Overview • • • • • • • • SENAS Overview Demographic variables and longitudinal change Ethnicity, clinical diagnosis, and cognitive change Age, education, and relationship to brain Decomposing demographic and brain effects Intracranial volume and cognition Factorial invariance Final thoughts Friday Harbor Psychometrics 2012 SENAS Overview Friday Harbor Psychometrics 2012 Purpose of SENAS Project (circa 1992) • Create matched English and Spanish language neuropsychological tests for ages 60+ New scales based upon neuropsychological model of cognitive functioning Scales psychometrically matched • Within English and Spanish language versions • Between English and Spanish language versions Distribution of item difficulty appropriate to elderly population Friday Harbor Psychometrics 2012 Approach • Concept of item and test bias central to project • Development and validation based on modern psychometric methods Item response theory (IRT) Latent variable modeling • Empirically based Data based approach to making decisions about item selection and scale construction Empirical evaluation of reliability and validity Friday Harbor Psychometrics 2012 Guiding Principles Initial Scale Development • Scales targeted to assess neuropsychologically relevant cognitive domains • Verbal and non-verbal measures • Non-timed • New scales, not translations of existing scales • Examiner administered • Item generation • New items • Broad range of difficulty Friday Harbor Psychometrics 2012 SENAS Scales and Abilities Measured Ability Verbal Measure Non-Verbal Measure Conceptual Thinking Verbal Conceptual Thinking Non-Verbal Conceptual Thinking Semantic Memory Object Naming Picture Association Attention Span Verbal Attention Span Visual Attention Span Episodic Memory Word List Learning Spatial Configuration Learning Spatial Abilities Pattern Recognition Spatial Localization Verbal Abilities Verbal Comprehension Verbal Expression Executive Function Category Fluency Letter/Phoneme Fluency Working Memory Friday Harbor Psychometrics 2012 Working Memory Model for Item Selection Friday Harbor Psychometrics 2012 Test Information for 3MS and SENAS Object Naming Friday Harbor Psychometrics 2012 Mungas et al., 2004 Scientific Applications Friday Harbor Psychometrics 2012 Summary Conclusions / Challenge for Conference • Demographic variables have robust effects on baseline test scores Especially ethnicity and education • Demographic variables have minimal effects on longitudinal change Brain and disease variables account for change Friday Harbor Psychometrics 2012 Community Caucasian English N ≈ 235 Caucasian N ≈ 700 Cognition N≈ 350 African American N ≈ 425 Hispanic English N ≈ 450 Clinical African American N ≈ 140 Hispanic N ≈ 130 Longitudinal N ≈ 175 Hispanic Spanish N ≈ 900 Hispanic Friday Harbor Psychometrics 2012 MRI Clinic Morphometric Life Experience Measures Ethnicity, education, and cognitive change Friday Harbor Psychometrics 2012 Purpose of this study • Understand how demographic variables relate to cognitive change • Demographic variables of interest Race/Ethnicity Education Language of test administration Friday Harbor Psychometrics 2012 Independent Variables • Model 1 Verbal Memory Form Previous Evaluation Spanish Administration Prev Eval by Spanish Interaction • Model 3 Age & Education Gender Friday Harbor Psychometrics 2012 • Model 2 Race/Ethnicity • Model 4 Recruitment Source Clinical Diagnosis APOE Genotype Language, Ethnicity, Education Effects Episodic Memory Baseline Effect Estimate Intercept .62 Change SE Estimate SE .08 Time -.04 .02 Previous Eval -.06 .04 -.04 .04 .33 .12 Language (Spanish) -.30 .13 Span x PrevEval Hispanic -.29 .11 .04 .03 African American -.06 .08 .01 .02 Education .03 .01 -.00 .00 Friday Harbor Psychometrics 2012 Language, Ethnicity, Education Effects Executive Function Baseline Effect Estimate SE Intercept .32 .07 Change Estimate SE Time -.03 .02 Previous Eval .01 .03 -.01 .03 .08 .07 Language (Spanish) -.08 .11 Span x PrevEval Hispanic -.32 .09 .02 .03 African American -.30 .07 .04 .02 Education .05 .01 -.00 .00 Friday Harbor Psychometrics 2012 Education: 0.1 0.2 0.3 0.4 9 Years 12 Years 15 Years 0.0 Expected Executive Function Score (standardized) 0.5 Executive Function Trajectories by Education 0 1 2 Time (years) Friday Harbor Psychometrics 2012 3 4 Average Ethnic Group Performance by Model – Episodic Memory Friday Harbor Psychometrics 2012 Ethnicity, clinical diagnosis, and cognitive change Friday Harbor Psychometrics 2012 Episodic Memory by Diagnosis Change Episodic Memory Group: 1.0 0.5 0.0 −0.5 −1.0 −2.0 −1.5 Expected Test Score (standardized) MCI−MCI (n=68) MCI−Demented (n=77) Demented (n=72) Normal (n=209) Normal−MCI (n=45) MCI−Normal (n=21) 0 1 2 3 4 5 Time (years) Friday Harbor Psychometrics 2012 Mungas et al., 2010 Episodic Memory by Diagnosis Change African American Episodic Memory − African American Group: 1.0 0.5 0.0 −0.5 −1.0 −2.0 −1.5 Expected Test Score (standardized) MCI−Stable (n=31) MCI−Demented (n=25) Normal (n=69) Normal−MCI (n=12) 0 1 2 3 Time (years) Friday Harbor Psychometrics 2012 4 5 Episodic Memory by Diagnosis Change Hispanic Episodic Memory − Hispanic Group: 1.0 0.5 0.0 −0.5 −1.0 −2.0 −1.5 Expected Test Score (standardized) MCI−Stable (n=19) MCI−Demented (n=24) Normal (n=65) Normal−MCI (n=16) 0 1 2 3 Time (years) Friday Harbor Psychometrics 2012 4 5 Episodic Memory by Diagnosis Change Caucasian Episodic Memory − Caucasian Group: 1.0 0.5 0.0 −0.5 −1.0 −2.0 −1.5 Expected Test Score (standardized) MCI−Stable (n=39) MCI−Demented (n=100) Normal (n=75) Normal−MCI (n=17) 0 1 2 3 Time (years) Friday Harbor Psychometrics 2012 4 5 Age, education, and relationship to brain Friday Harbor Psychometrics 2012 MRI Effect Sizes and Age and Education Adjustment - Episodic Memory Variance Explained (R-squared) Age Adjusted No Adjustment Education Adjustment 0.40 0.35 0.30 0.25 0.20 0.15 0.10 0.05 0.00 Combined (n=315) Caucasian (n=134) African American (n=79) Hispanic (n=102) Sample Friday Harbor Psychometrics 2012 Mungas et al., 2009 Executive Function, MRI, Age & Education African Americans -.24 Age -.55 BM .00 Education .36 -.03 .32 -.26 .39 Executive Function Friday Harbor Psychometrics 2012 Mungas et al., 2009 Executive Function, MRI, Age & Education Hispanics .01 Age -.60 BM .00 Education .30 -.17 .46 -.40 .44 Executive Function Friday Harbor Psychometrics 2012 Mungas et al., 2009 Executive Function, MRI, Age & Education Caucasians -.07 Age -.53 BM .00 Education .29 -.10 .29 -.36 .28 Executive Function Friday Harbor Psychometrics 2012 Mungas et al., 2009 Age and Education Influences on MRI - Cognition Relationships • It is possible to separate disease effects from extraneous influences • MRI effects on cognition can obscured by demographic effects on test performance, especially when • Demographic relationship with test score is larger than relationship with disease • Substantial heterogeneity of demographic variable in population of interest Friday Harbor Psychometrics 2012 Decomposing demographic and brain effects Friday Harbor Psychometrics 2012 Reed et al., 2010 Friday Harbor Psychometrics 2012 Data • 288 subjects. 158 normal, 92 MCI, 38 demented • 96 African Americans, 74 Hispanics (32 tested in English, 42 tested in Spanish), and 118 Caucasians. • Education M = 12.7 yrs (range 0-25) • Age M = 74.7 yrs (range 60-93) • Mean evaluations = 3.5; 74% had 3 or more evaluations. N of evaluations truncated at 5. Friday Harbor Psychometrics 2012 Reed et al., 2010 • Mem-D ~ 20% of episodic memory variance • Mem-B ~ 20% of episodic memory variance • Mem-R ~ 50% of episodic memory variance Friday Harbor Psychometrics 2012 Reed et al., 2010 Relationships of memory components with global cognitive function (CDR sum of boxes) DV CDR Sum Memory Standardized Component Coefficient p Mem-D -0.09 ns Mem-B Mem-R -0.43 -0.44 0.001 0.001 Friday Harbor Psychometrics 2012 Reed et al., 2010 Relationships of memory components with clinical progression (conversion to MCI or dementia) Memory Component Mem-D Mem-B Mem-R Friday Harbor Psychometrics 2012 Relative Risk Ratio (confidence interval) 1.58 (0.92 - 2.71) 0.19 (0.11 - 0.33) 0.27 (0.18 - 0.40) Reed et al., 2010 Relationships of memory components with longitudinal change in cognition (executive function) Memory Component Mem-D Random Effect baseline Parameter 0.364 Standard p Error 0.043 0.001 Mem-B baseline 0.180 0.031 0.001 Mem-R baseline 0.329 0.034 0.001 Mem-D Mem-B change change -0.011 0.050 0.010 0.010 ns 0.001 Mem-R change 0.047 0.011 0.001 Friday Harbor Psychometrics 2012 Reed et al., 2010 Effects of Mem-D? • Substantial variance in Episodic Memory was uniquely related to demographic variables (ethnicity and education especially) Greater amount for other cognitive domains • Mem-D was minimally related to clinical outcomes Especially longitudinal decline and conversion • Challenge in clinical neuropsychological assessment is to separate demographic from brain influences on test scores Friday Harbor Psychometrics 2012 Reed et al., 2010 Intracranial volume and cognition Friday Harbor Psychometrics 2012 Farias et al., 2012 Model of joint effects of intracranial volume and brain structure Friday Harbor Psychometrics 2012 Farias et al., 2012 Semantic Memory by ICV Friday Harbor Psychometrics 2012 Farias et al., 2012 Cognition and ICV • ICV related to Semantic Memory and Executive Function • ICV not related to Episodic Memory • Early development might have impact on cortical development Friday Harbor Psychometrics 2012 Farias et al., 2012 Factorial invariance Friday Harbor Psychometrics 2012 Sample • Community dwelling recruited using cognitive screening protocol and clinic referrals Whites - n=678 Blacks - n=352 Hispanics, English Speaking n=434 Hispanics, Spanish Speaking n=877 • Broad range of cognitive function Normal to demented Friday Harbor Psychometrics 2012 Mungas et al., 2011 Best Factor Structure Object Naming Spatial Picture Association Verbal Abstraction Pattern Recognition Verbal Verbal Expression Verbal Comprehension Word List Learning 1 Friday Harbor Psychometrics 2012 NonVerbal Reasoning Visual Attention Attention Memory Word List Learning 2 Spatial Config Learning Spatial Localization Verbal Attention Working Memory Category Fluency Fluency Phonemic Fluency Mungas et al., 2011 Invariance of Dimensional Structure Across Ethnic and Language Groups • Same number of dimensions • Invariant Factor Loadings Observed test scores have same relationship to latent dimensions • Some differences in Intercepts for tests Spanish speaker has to have greater latent attention ability to achieve a given Verbal Attention score Friday Harbor Psychometrics 2012 Mungas et al., 2011 Verbal Attention by Attention/Working Memory Caucasians Hispanics: English-speakers African Americans Hispanics: Spanish-speakers 3 VRATT 2 1 0 -1 -2 -3 -2 -1 0 1 2 ATTN/WM Friday Harbor Psychometrics 2012 Mungas et al., 2011 3 Final Thoughts Friday Harbor Psychometrics 2012 Longitudinal Trajectories Friday Harbor Psychometrics 2012 Longitudinal Trajectories Friday Harbor Psychometrics 2012 Conclusions • Cross-sectional results can be misleading Confound lifelong ability and disease effects • Longitudinal assessment important to separate these effects Ideally beginning in mid adulthood In absence of ideal – evidence that demographic effects on cognitive decline are small and distal • Don’t assume a low score is impairment Friday Harbor Psychometrics 2012 References • Farias, S. T., D. Mungas, et al. (2012). "Maximal brain size remains an important predictor of cognition in old age, independent of current brain pathology." Neurobiol Aging 33(8): 1758-1768. • Mungas, D., B. R. Reed, et al. (2004). "Spanish and English Neuropsychological Assessment Scales (SENAS): Further development and psychometric characteristics." Psychological Assessment 16(4): 347-359. • Mungas, D., B. R. Reed, et al. (2005). "Spanish and English Neuropsychological Assessment Scales: relationship to demographics, language, cognition, and independent function." Neuropsychology 19(4): 466-475. Friday Harbor Psychometrics 2012 References • Mungas, D., B. R. Reed, et al. (2009). "Age and education effects on relationships of cognitive test scores with brain structure in demographically diverse older persons." Psychology and Aging 24(1): 116-128. • Mungas, D., L. Beckett, et al. (2010). "Heterogeneity of cognitive trajectories in diverse older persons." Psychol Aging 25(3): 606-619. • Mungas, D., K. F. Widaman, et al. (2011). "Measurement invariance of neuropsychological tests in diverse older persons." Neuropsychology 25(2): 260-269. • Reed, B. R., D. Mungas, et al. (2010). "Measuring cognitive reserve based on the decomposition of episodic memory variance." Brain 133: 2196-2209. Friday Harbor Psychometrics 2012 Ethnic Differences and DIF • SENAS Picture Association and Verbal Learning • English Administration - N = 1113 396 Hispanics, 480 Whites, 237 Blacks • Spanish Administration - N = 801 • Ability adjusted for all-source DIF Education, Ethnicity, Age • MIMIC approach Friday Harbor Psychometrics 2012 Picture Association Effects of DIF Adjustment Friday Harbor Psychometrics 2012 Word List Learning Effects of DIF Adjustment Friday Harbor Psychometrics 2012 Effect of DIF Adjustment on Individual Scores (95% confidence interval) Picture Association Word List Learning White ±.29 s.d. ±.10 s.d. Black ±.41 s.d. ±.02 s.d. Hispanic - English ±.23 s.d. ±.00 s.d. Hispanic - Spanish ±.36 s.d. ±.06 s.d. Friday Harbor Psychometrics 2012 SENAS and DIF • DIF accounted for a very small amount of ethnic differences in Object Naming, Picture Association, Word List Learning • DIF effects differ by domain Greater for measures of semantic memory Smaller for episodic memory • DIF effects can potentially bias interpretation of individuals scores Especially in borderzone areas Friday Harbor Psychometrics 2012 Cross-sectional ethnic differences can be explained by confounding variables Friday Harbor Psychometrics 2012 Semantic Memory Mean Ethnic Group Differences Friday Harbor Psychometrics 2012