Supplementary information S1 | Reliability, clinical validity and norms for theory of mind measures
Test
False Belief
Tasks1,2
Reliability
Clinical validity
Norms
Population
Both children7 and adolescents8 with autism
Extensive research
Developed for
Internal consistency
α = 0.60–0.82.3
are typically impaired on false belief tasks.
base. Numerous
children, variants
Adults with autism might not differ from
types of false belief now available for
Test-retest
Mean intervals of:
controls on standard false belief tasks, but are
tasks are available.
adults
26 days, r = 0.66 to 0.773
often impaired on more advanced false belief
2–3 weeks, POs = 0.54–1.0.4
tasks.7,9
People with bvFTD are typically impaired on
Across different FB tasks
POs of 0.64–0.75,5 and 0.77–
at least some false belief tasks.10–14
0.95.6
Studies involving a range of disorders
Extensive research
Age ≥13 years
TASIT Social
Test-retest
Mean interval of 7 days,
characterized by social cognitive dysfunction
base. Adult and
Inference
Tests15
rs = 0.88 (Part 2)16 and 0.83
have typically identified deficits in detecting
adolescent norms
(Part 3).16
sarcasm, but no difficulty in understanding
available.15
sincere social interchanges. These disorders
Alternative forms
Mean interval of 15 weeks,
include autism,17 bvFTD14,18 (non-significant
16
rs = 0.62 (Part 2) and 0.78
trend in one study19), TBI,20–23 and
(Part 3).16
schizophrenia.24,25
Although there are exceptions,34 people with
Extensive research
Children and adults
Strange Stories Internal consistency
7,26
27
Test
α = 0.74–0.75.
autism are typically impaired on the ToM
base. Norms
stories.7,26,28–31,35,36 The test is also sensitive to available for
Inter-rater
7
27
ICCs of 0.81, 0.90 and
social communication disorder.32 When
children aged 5–
0.99.28 Percentage agreements control stories are administered, people with
12 years.39
of 95–97%29, 98%30 and
autism do not usually exhibit deficits.
99%.31 ks of 0.91,32 0.78,33
People with bvFTD exhibit no impairment on
0.7227 and 0.76.31
the ToM stories37 or control stories.38
Test-retest
Mean interval of 12 months,
r = 0.64.27
People with autism are typically impaired in
Extensive research
Separate child and
Faux-Pas
Internal consistency
Test40,41
α = 0.91 for ToM and 0.42 for their understanding of faux pas.29,41,44
base. Norms
adult versions
control stories.42 Split half
The Faux-Pas Test has excellent sensitivity
available for
available
r = 0.95 for ToM stories and
and specificity to bvFTD,13 and can
children aged 7–
0.43 for control stories.42
differentiate between even high-functioning
11 years.41
45
people
with
bvFTD
and
controls.
Inter-rater
Estimates of 0.76,43 as well as People with schizophrenia are impaired on the
rs of 0.9813 and 0.95,44 and
Faux-Pas Test.43,46
95% agreement.29
Test-retest. Mean interval of
3 months, r = 0.83.43
Most studies show that people with autism are Extensive research
Separate child and
Reading the
Internal consistency
α = 0.61,48 0.60–0.63,49
impaired on the original and revised
base. Norms are
adult versions
Mind in the
0.37,50 0.64,42 0.53–0.77,51
RMET,8,34,35,36,47,57,58 although exceptions
reported for
available
Eyes Test
(RMET)35,47*
0.7052 and 0.58–0.63.53 Split
have been noted.29
children and
half r = 0.6342 –0.71.54
Most studies also show that people with
adolescents.59
bvFTD, even high-functioning people, do not
Test-retest
Mean intervals of:
perform as well as controls.13,584
7 days, ICC = 0.70,51
2 weeks, ICC = 0.65,55
30 days, ICC = 0.83,48
1 year, ICC = 0.6356–0.74.50
*The data reported here refer to the adult version; reliability data are also available for the children’s version. 60,61 α refers to Cronbach’s alpha,
mean interval refers to average interval between testing occasions, PO refers to proportion of observed agreement, ICC refers to intraclass
correlation coefficient, k refers to kappa. Abbreviations: bvFTD, behavioural-variant frontotemporal dementia.
1.
2.
Wimmer, H. & Perner, J. Beliefs about beliefs - Representation and constraining
function of wrong beliefs in young childrens understanding of deception. Cognition
13, 103-128 (1983).
Perner, J. & Wimmer, H. John thinks that Mary thinks that - Attribution of 2nd-order
beliefs by 5-year-old to 10-year-old children. J. Exp. Child Psychol. 39, 437-471
(1985).
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
Hughes, C. et al. Good test-retest reliability for standard and advanced false-belief
tasks across a wide range of abilities. J. Child Psychol. Psychiatry 41, 483-490
(2000).
Mayes, L.C., Klin, A., Tercyak, K.P., Cicchetti, D.V. & Cohen, D.J. Test-retest
reliability for false-belief tasks. J. Child Psychol. Psychiatry 37, 313-319 (1996).
Charman, T. & Campbell, A. Reliability of theory of mind task performance by
individuals with a learning disability: a research note. J. Child Psychol. Psychiatry 38,
725-730 (1997).
Grant, C.M., Grayson, A. & Boucher, J. Using tests of false belief with children with
autism: how valid and reliable are they? Autism 5, 135-145 (2001).
White, S., Hill, E., Happe, F. & Frith, U. Revisiting the Strange Stories: revealing
mentalizing impairments in autism. Child Dev. 80, 1097-1117 (2009).
Mazza, M. et al. Affective and cognitive empathy in adolescents with autism
spectrum disorder. Front. Hum. Neurosci. 8, 791 (2014).
Hill, E.L., Sally, D. & Frith, U. Does mentalising ability influence cooperative
decision-making in a social dilemma? Introspective evidence from a study of adults
with autism spectrum disorder. J. Conscious. Stud. 11, 144-161 (2004).
Freedman, M., Binns, M.A., Black, S.E., Murphy, C. & Stuss, D.T. Theory of mind
and recognition of facial emotion in dementia: challenge to current concepts.
Alzheimer Dis. Assoc. Disord. 27, 56-61 (2013).
Le Bouc, R. et al. My belief or yours? Differential theory of mind deficits in
frontotemporal dementia and Alzheimer's disease. Brain 135, 3026-3038 (2012).
Fernandez-Duque, D., Baird, J.A. & Black, S.E. False-belief understanding in
frontotemporal dementia and Alzheimer's disease. J. Clin. Exp. Neuropsychol. 31,
489-497 (2009).
Gregory, C. et al. Theory of mind in patients with frontal variant frontotemporal
dementia and Alzheimer's disease: Theoretical and practical implications. Brain 125,
752-764 (2002).
Shany-Ur, T. et al. Comprehension of insincere communication in neurodegenerative
disease: lies, sarcasm, and theory of mind. Cortex 48, 1329-1341 (2012).
Flanagan, S., McDonald, S. & Rollins, J. The Awareness of Social Inference Test Revised (TASIT-R) manual. (Pearson Clinical and Talent Assessment, Sydney, NSW
2011).
McDonald, S. et al. Reliability and validity of The Awareness of Social Inference
Test (TASIT): A clinical test of social perception. Disabil. Rehabil. 28, 1529-1542
(2006).
Mathersul, D., McDonald, S. & Rushby, J.A. Understanding advanced theory of mind
and empathy in high-functioning adults with autism spectrum disorder. J. Clin. Exp.
Neuropsychol. 35, 655-668 (2013).
Kipps, C.M., Nestor, P.J., Acosta-Cabronero, J., Arnold, R. & Hodges, J.R.
Understanding social dysfunction in the behavioural variant of frontotemporal
dementia: the role of emotion and sarcasm processing. Brain 132, 592-603 (2009).
Rankin, K.P. et al. Detecting sarcasm from paralinguistic cues: anatomic and
cognitive correlates in neurodegenerative disease. Neuroimage 47, 2005-2015 (2009).
McDonald, S., Flanagan, S., Rollins, J. & Kinch, J. TASIT: A new clinical tool for
assessing social perception after traumatic brain injury. J. Head Trauma Rehabil. 18,
219-238 (2003).
McDonald, S., Flanagan, S., Martin, I. & Saunders, C. The ecological validity of
TASIT: A test of social perception. Neuropsychol. Rehabil. 14, 285-302 (2004).
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
McDonald, S. & Flanagan, S. Social perception deficits after traumatic brain injury:
interaction between emotion recognition, mentalizing ability, and social
communication. Neuropsychology 18, 572-579 (2004).
McDonald, S. et al. Assessing social cognition and pragmatic language in adolescents
with traumatic brain injuries. J. Int. Neuropsychol. Soc. 19, 528-538 (2013).
Kern, R.S. et al. Theory of mind deficits for processing counterfactual information in
persons with chronic schizophrenia. Psychol. Med. 39, 645-654 (2009).
Sparks, A., McDonald, S., Lino, B., O'Donnell, M. & Green, M.J. Social cognition,
empathy and functional outcome in schizophrenia. Schizophr. Res. 122, 172-178
(2010).
Happe, F.G. An advanced test of theory of mind: understanding of story characters'
thoughts and feelings by able autistic, mentally handicapped, and normal children and
adults. J. Autism Dev. Disord. 24, 129-154 (1994).
McKown, C., Allen, A.M., Russo-Ponsaran, N.M. & Johnson, J.K. Direct assessment
of children's social-emotional comprehension. Psychol. Assess. 25, 1154-1166 (2013).
Rogers, K., Dziobek, I., Hassenstab, J., Wolf, O.T. & Convit, A. Who cares?
Revisiting empathy in Asperger syndrome. J. Autism Dev. Disord. 37, 709-715
(2007).
Spek, A.A., Scholte, E.M. & Van Berckelaer-Onnes, I.A. Theory of mind in adults
with HFA and Asperger syndrome. J. Autism Dev. Disord. 40, 280-289 (2010).
Jolliffe, T. & Baron-Cohen, S. The Strange Stories Test: a replication with highfunctioning adults with autism or Asperger syndrome. J. Autism Dev. Disord. 29, 395406 (1999).
Kaland, N. et al. The Strange Stories test--a replication study of children and
adolescents with Asperger syndrome. Eur. Child Adolesc. Psychiatry 14, 73-82
(2005).
Freed, J. et al. Performance of children with social communication disorder on the
Happe Strange Stories: Physical and mental state responses and relationship to
language ability. J. Commun. Disord. 55, 1-14 (2015).
McKown, C., Gumbiner, L.M., Russo, N.M. & Lipton, M. Social-emotional learning
skill, self-regulation, and social competence in typically developing and clinicreferred children. J. Clin. Child Adolesc. Psychol. 38, 858-871 (2009).
Schuwerk, T., Vuori, M. & Sodian, B. Implicit and explicit Theory of Mind reasoning
in autism spectrum disorders: The impact of experience. Autism 19, 459-468 (2015).
Baron-Cohen, S., Jolliffe, T., Mortimore, C. & Robertson, M. Another advanced test
of theory of mind: evidence from very high functioning adults with autism or asperger
syndrome. J. Child Psychol. Psychiatry 38, 813-822 (1997).
Lahera, G. et al. Movie for the Assessment of Social Cognition (MASC): Spanish
validation. J. Autism Dev. Disord. 44, 1886-1896 (2014).
Lough, S. et al. Social reasoning, emotion and empathy in frontotemporal dementia.
Neuropsychologia 44, 950-958 (2006).
Snowden, J.S. et al. Social cognition in frontotemporal dementia and Huntington's
disease. Neuropsychologia 41, 688-701 (2003).
O'Hare, A.E., Bremner, L., Nash, M., Happe, F. & Pettigrew, L.M. A clinical
assessment tool for advanced theory of mind performance in 5 to 12 year olds. J.
Autism Dev. Disord. 39, 916-928 (2009).
Stone, V.E., Baron-Cohen, S. & Knight, R.T. Frontal lobe contributions to theory of
mind. J. Cogn. Neurosci. 10, 640-56 (1998).
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
Baron-Cohen, S., O'Riordan, M., Stone, V., Jones, R. & Plaisted, K. Recognition of
faux pas by normally developing children and children with Asperger syndrome or
high-functioning autism. J. Autism Dev. Disord. 29, 407-418 (1999).
Soderstrand, P. & Almkvist, O. Psychometric data on the Eyes Test, the Faux Pas
Test, and the Dewey Social Stories Test in a population-based Swedish adult sample.
Nordic Psychol. 64, 30-43 (2012).
Zhu, C.Y. et al. Impairments of social cues recognition and social functioning in
Chinese people with schizophrenia. Psychiatry Clinical Neurosci. 61, 149-158 (2007).
Zalla, T., Sav, A.M., Stopin, A., Ahade, S. & Leboyer, M. Faux pas detection and
intentional action in Asperger Syndrome. A replication on a French sample. J. Autism
Dev. Disord. 39, 373-382 (2009).
Torralva, T., Roca, M., Gleichgerrcht, E., Bekinschtein, T. & Manes, F. A
neuropsychological battery to detect specific executive and social cognitive
impairments in early frontotemporal dementia. Brain 132, 1299-1309 (2009).
Konstantakopoulos, G. et al. Self-rated and performance-based empathy in
schizophrenia: the impact of cognitive deficits. Soc. Neurosci. 9, 590-600 (2014).
Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y. & Plumb, I. The "Reading the
Mind in the Eyes" Test revised version: a study with normal adults, and adults with
Asperger syndrome or high-functioning autism. J. Child Psychol. Psychiatry 42, 241251 (2001).
Vellante, M. et al. The "Reading the Mind in the Eyes" test: Systematic review of
psychometric properties and a validation study in Italy. Cogn. Neuropsychiatry 18,
326-354 (2013).
Voracek, M. & Dressler, S.G. Lack of correlation between digit ratio (2D : 4D) and
Baron-Cohen's "Reading the Mind in the Eyes" test, empathy, systemising, and
autism-spectrum quotients in a general population sample. Pers. Individ. Diff. 41,
1481-1491 (2006).
Khorashad, B.S. et al. The "Reading the Mind in the Eyes" Test: Investigation of
Psychometric Properties and Test-Retest Reliability of the Persian Version. J. Autism
Dev. Disord. 2651-2666 (2015).
Prevost, M. et al. The Reading the Mind in the Eyes test: validation of a French
version and exploration of cultural variations in a multi-ethnic city. Cogn.
Neuropsychiatry 19, 189-204 (2014).
Dehning, S. et al. Comparative cross-sectional study of empathy among first year and
final year medical students in Jimma University, Ethiopia: Steady state of the heart
and opening of the eyes. BMC Med. Educ. 12, 34 (2012).
Harkness, K.L., Jacobson, J.A., Duong, D. & Sabbagh, M.A. Mental state decoding in
past major depression: Effect of sad versus happy mood induction. Cogn. Emot. 24,
497-513 (2010).
Wakabayashi, A. & Katsumata, A. The Motion Picture Mind-Reading Test:
Measuring individual differences of social cognitive ability in a young adult
population in Japan. J. Individ. Differ. 32, 55-64 (2011).
Yildirim, E.A. et al. Investigation of the reliability of the "reading the mind in the
eyes test" in a Turkish population. Turk Psikiyatri Derg. 22, 177-186 (2011).
Fernandez-Abascal, E.G., Cabello, R., Fernandez-Berrocal, P. & Baron-Cohen, S.
Test-retest reliability of the 'Reading the Mind in the Eyes' test: a one-year follow-up
study. Mol. Autism 4, 33 (2013).
Vogindroukas, I., Chelas, E.N. & Petridis, N.E. Reading the Mind in the Eyes Test
(Children's Version): a comparison study between children with typical development,
58.
59.
60.
61.
children with high-functioning autism and typically developed adults. Folia Phoniatr.
Logop. 66, 18-24 (2014).
Keri, S. Social influence on associative learning: double dissociation in highfunctioning autism, early-stage behavioural variant frontotemporal dementia and
Alzheimer's disease. Cortex 54, 200-209 (2014).
Tonks, J., Williams, W.H., Frampton, I., Yates, P. & Slater, A. Assessing emotion
recognition in 9-15-years olds: preliminary analysis of abilities in reading emotion
from faces, voices and eyes. Brain Inj. 21, 623-629 (2007).
Hallerback, M.U., Lugnegard, T., Hjarthag, F. & Gillberg, C. The Reading the Mind
in the Eyes Test: test-retest reliability of a Swedish version. Cogn. Neuropsychiatry
14, 127-143 (2009).
Muller, C.M. & Gmunder, L. An Evaluation of the "Reading the Mind in the EyesTest" with seventh to ninth graders. J. Ment. Health Res. Intellect. Disabil. 7, 34-44
(2014).