Part 5 Test Adaptations


1. Conceptualizations
2. Example
What is the central issue in
translations/adaptations?

Producing instruments that measure
target constructs adequately in
target cultures
A Note on Terminology

Translation
• Conventional term, still often used

Adaptation
• Has become generic term for modern
translation practices
• Based on increased sensitivity for nonlinguistic factors in translations, such as
cultural norms of address, relevance of
thorough knowledge of target culture
Main Applications of
Translations/Adaptations

Comparative Studies
• Comparison of construct or mean scores
across cultures
• High demands on comparability of
scores


Maximizing comparability
Monocultural studies in target culture
• Main issue is ensuring validity in new
context
• Few demands on comparability scores
Translations in Historical Perspective

Stage 1:
• Close translations were standard practice
• Techniques were developed (e.g.,
translation back translation)

Important societal developments:
• Globalization and migration (multi-ethnic
societies)

Stage 2:
• Increasing appreciation that close
translations have problems, e.g., Grade
12 = Form 6 = ……?
• Need for adaptations, localizations
• Need for standardization of adaptation
procedures
What is a Good Translation/
Adaptation?

Dependent on perspective
• Linguistic perspective
• Psychological perspective

Mapping problem:
• Translating/adapting can be seen as finding an
optimal mapping of text in two languages

What is a good mapping?
• A good mapping shows equivalence of the
original and translation
Example


What is the American equivalent of
the Dutch item “Hoe heet de koningin
van Nederland?” (Suppose that item
is part of a test of crystallized
intelligence)
Literal/close translation: What is the
name of the queen of the
Netherlands?”
• Problem: Item more difficult for American
children than for Dutch children

Adaptation: “What is the name of the
president of the USA?”
• Problem: Queen and president are not
equally known in their respective
countries
What Does “Equivalent” Mean?

Eusebius Hieronymus (St. Jerome, famous
bible translator from Greek and Hebrew to
Latin; ±347—419/420):
• 2 types of translations: “words” and
“meanings” (he favored the latter)

Here two types of equivalence relevant:
linguistic
}
psychological
mapping/equivalence
Linguistic Equivalence



(Broader than similarity of
words)
Linguistic equivalence refers to
similarity of linguistic features of
a text.
Examples of relevant linguistic
features are:
• Lexical similarity
• Grammatical accuracy
• In general: emphasis on formal-textual
characteristics (cf. automatic
translations)
Psychological Equivalence


Psychological equivalence refers
to similarity of (psychological)
meaning and scores
Similarity in a broad sense:
• Textual, e.g.,


Connotation of words, implied context of
text
Comprehensibility
• Metrical:

Score comparability
Relationship between Two
Perspectives
Three possible relations between linguistic and
psychological features, depending on the
overlap:
c. none
a.
complete
b. partial
psych.
linguistic
Translatable
Poorly translatable
Essentially
non-translatable
Translatability
A psychological test/item is

Well translatable if linguistic and
psychological features yield the same
translation

Poorly translatable if linguistic
and psychological features do not
entirely converge (e.g., translation of
slang: meaning is translatable, but
conciseness is lost)

Non-translatable if there is a
complete or nearly complete
nonoverlap (e.g., Jabberwocky)
Jabberwocky (Lewis Carroll, 1871)
'Twas brillig, and the slithy toves
Did gyre and gimble in the wabe;
All mimsy were the borogoves,
And the mome raths outgrabe.
"Beware the Jabberwock, my son!
The jaws that bite, the claws that
catch!
Beware the Jubjub bird, and shun
The frumious Bandersnatch!"
Illustration by John Tenniel
Framework for
Translations/Adaptations


Need for a theoretical—
methodological framework that links
all stages of a project
Bias and equivalence as key
concepts
Steps in Designing Cross-Cultural Tests
(Hambleton & Patsula, 1999)
1. Ensure that construct equivalence exists in the language and cultural groups
of interest.
2. Decide whether to adapt an existing test or develop a new test.
3. Select well-qualified translators.
4. Translate and adapt the test.
5. Review the adapted version of the test and make necessary revisions.
6. Conduct a small tryout of the adapted version of the test.
7. Carry out a more ambitious field-test.
8. Choose a statistical design for connecting scores on the source and target
language versions of the test.
9. If cross-cultural comparisons are of interest, ensure equivalence of the
language versions of the test.
10. Perform validation research, as appropriate.
11. Document the process and prepare a manual for the users of the adapted
tests.
12. Train users.
13. Monitor experiences with the adapted test, and make appropriate revisions.
Overview of Common
Procedures to Examine
Accuracy of Translations/
Adaptations


Procedures as opportunities to
strengthen the quality of a
translation/adaptation project
Two taxonomies presented here:
• common: (back) translations vs.
committee approach
• use of existing/new material
Theoretical and Methodological
Background

Crucial concept in translations is
equivalence:
• Linguistic

Mapping of linguistic meaning (word
meaning, sentence meaning)
• Psychological


Mapping of psychological meaning (serves
the same psychological function in all
languages?)
A good translation combines these
considerations
Options

Adoption (Close “literal” translation)
• Advantage: maintains metric equivalence
• Disadvantage: adequacy (too) readily
assumed, should be demonstrated

Adaptation (changing contents of one
or more items so as to increase
cultural appropriateness)
• Advantage: more flexible, more tailored to the
context
• Disadvantage: fewer statistical techniques
available to compare scores across cultures

Assembly (composing a new
instrument)
• Advantage: very flexible
• Disadvantage: almost no comparability
A Sample of Possible Procedures
(after Harkness, 2003)
Translation stage
• Translation back translation
• Committee approach (forward
translations)
• Mixed approaches (e.g., independent
forwards)
Pretesting stage
(qualitative)
•Think alouds, focus groups
•Feedback from mono- and bilinguals
•Comprehension and readability checks
Pretesting or actual
administration
(quantitative)
• Equivalence and bias analyses
(DIF, structural equivalence)
Strength and Weakness of
Translations Back Translation
Main strengths
• Well accepted quality
check; standard
procedures well known
in scientific community
(incl. researchers, grant
institutions and journal
boards)
• No knowledge of target
language required
Main weaknesses
• Capitalizes on linguistic,
cultural, and itemwriting skills of (usually)
a single person
• Can produce stilted
language
• Readability and
comprehensibility in
target language may be
problematic
What is the Best Option?


One type is not intrinsically better or
worse than another
Main question is
NOT

What is globally the best choice?
BUT

What is the best choice in a specific
case?
Four Important Perspectives
(Harkness & Van de Vijver, in preparation):
Construct
equivalence
Cultural
equivalence
• Similarity of
construct in
source and
target culture
• Norms about
interaction
(modes of
address)
• “Cultural fact
sheet”
Linguistic
equivalence
• Translation
accuracy:
Retention of
denotation
and
connotation
Measurement
equivalence
• Retention of
psychometric
features
(response
styles)
• Similarity of
factors
measured by
a test and
comparability
of scores
Adaptation Perspectives
Construct
Culture
Language
Measurement
Integration
Indicator


A good translation/adaptation
combines equivalence
perspectives
What is a good translation/
adaptation?
•A translation or adaptation is
good when it combines high
levels of construct, cultural,
linguistic, and measurement
equivalence.
Is There a Best Way to Translate
an Instrument?

Simple items often straightforward to
translate
• Close translations will do well, various
kinds of equivalence jointly maximized

More complex items often require
choices about which equivalence will
be maximized:
• Maximizing comparability or cultural
appropriateness ?


Different perspectives on
equivalence often, but not
always compatible
Example: cross-cultural differences
in modes of address
• Maximizing linguistic equivalence may
challenge cultural appropriateness (e.g.,
requests may be too direct)
• Maximizing cultural appropriateness
may challenge statistical equivalence
(e.g., rephrasing may threaten
comparability of scores)
Taxonomy of Adaptations
Need for adaptation can be
• Construct-driven
• Culture-driven (communication
style)
• Language-driven
• Measurement-driven
(familiarity/recognizability)
Most examples come from
Traveling with Cognitive Tests:
Testing the Validity of a KABC-II Adaptation in India
Maike Malda
Fons J. R. van de Vijver
Krishnamachari Srinivasan
Catherine Transler
Prathima Sukumar
Accepted for publication in Assessment.
Kaufman Assessment Battery
for Children (second edition)
Subtests:







Atlantis
Number Recall
Rover
Triangles
Word Order
Pattern Reasoning
Story Completion  replaced by
adaptation of
WISC(/-R/-III) Picture Arrangement
1. Example Construct-Driven


Problems with the behaviors or
attitudes associated with the
construct or with communication
norms pertaining to these behaviors
or attitudes
Usage of somatic and psychological
symptoms in depression inventories

Differential norms in allowance to express
psychological symptoms across cultures


Patel, Abas, Broadhead, Todd, & Reeler (2001)
• In Zimbabwe, multiple somatic complaints such as
headaches and fatigue are the most common
presentations of depression. On inquiry, however,
most patients freely admit to cognitive and
emotional symptoms. Many somatic symptoms,
especially those related to the heart and the head,
are cultural metaphors for fear or grief. Most
depressed individuals attribute their symptoms to
“thinking too much” (kufungisisa), to a
supernatural cause, and to social stressors. Our
data confirm the view that although depression in
developing countries often presents with somatic
symptoms, most patients do not attribute their
symptoms to a somatic illness and cannot be said
to have “pure” somatisation. This means that it is
vital to understand the culture specific terminology
used by patients and to assess mood in those with
multiple somatic complaints.
Consequence
• Common western measures of depression will
under-diagnose depression in Shona speakers.
2. Example Culture-Driven
Example: ‘Burglar’ (Picture Arrangement; adapted for use
in low-SES children in Bangalore, India )


Problems:
1. Unclear whether the burglar was getting in
or getting out;
2. Man not recognized as burglar;
3. Window was not recognized (vertically
moving windows are uncommon in India)
33 a
Malda, Van de Vijver, Srinivasan, Transler (2008): Adapting a Western Cognitive Test for
Non-Western Context: The KABC-II in Bangalore, India
3. Example of Language-Driven
Adaptation


Example: Do you often feel distressed?
Translation to Dutch:
• “Distressed” does not have an equivalent word in Dutch
• Possible solutions



Composite of different emotions in Dutch; ask for
frequency of composite (“how often do you feel X and Y?”).
Problem: composite may not be recognizable
Choose a single emotion that is as close as possible;
problem: change of item content if no close match can be
found
Describe the emotion in the item (e.g., vignette); problem:
may require a similar description in English original
• Need to check adequacy of chosen solution in statistical
analysis
• Combination of judgmental and statistical evidence
crucial in instruments that are more difficult to
translate/adapt

Language and test content:
• Adaptation of words in subtest Atlantis:
Kannada nonsense words
(e.g., English ‘Dablee’  Kannada
‘Ribu’)
 Important: number of syllables
• Adaptation of digits in subtest Number
Recall
 based on number of syllables (1 in
English version; first 2 and then 3 in
Kannada version)

4. Example of Measurement-Driven
Adaptation (Unfamiliarity)


Kaufman ABC used in Bangalore
(Kannada-speaking children)
Adaptation of words in subtest Word Order
based on:


Unfamiliarity and ambiguity of objects and words
Number of syllables
Original version 
Kannada version 
36
Malda, Van de Vijver, Srinivasan, Transler (in review): Adapting a Western Cognitive Test for
a Non-Western Context: The KABC-II in Bangalore, India
Original version 
Kannada version 
Problem: word for star in Kannada is too long,
English word “star” is well known but too short
(monosyllabic)
Original version 
Kannada version 
Problems:
(1) Key was often called ice cream;
(2) English word “key” was often used, which is
too short (monosyllabic)
Original version 
Kannada version 
Problem: original drawing was not easily
recognized as house, distinguishing features
added
Example: ‘Painting’
Problem: mirror was not recognized
Rover

Test content:



Additional instructions in subtest Rover
One additional instruction in subtest Pattern
Reasoning
Slight change of subtest composition and item order
in
subtest Triangles
Sample item Original version 
Sample item Indian version 
Problem: original sample item was too difficult; this
item has been added as actual test item
Background Reading
(1)
(2)