Cross Cultural Research
Dr. K. A. Korb
University of Jos
15 May 2009
OUTLINE
• Issues in Cross-Cultural Research
– Validity
– Participant
– Instrument
• Types of cross-cultural Research
Dr K. A. Korb
University of Jos
Validity
• Validity: The degree of accuracy with which a
conclusion is drawn
– Researchers have a burden to demonstrate that
their research finding have validity
– The validity of a study’s finding is directly related
to the validity of the research methods
• Interpretive validity: Valid communication
between the research and target group
Dr K. A. Korb
University of Jos
Validity
• Ecological validity: Extent that findings
generalize beyond the research procedure to
the natural context
• Theoretical (construct) validity: Establishing valid
operational measures for the concept being studied
– This related to:
• Independent variables
• Dependent variables
Dr K. A. Korb
University of Jos
Research Example
Theory:
Research Study:
Theory:
Dr K. A. Korb
University of Jos
IV:
Regular
Reading
DV:
Reading
Achievement
Read book
every day with
an adult for
10 min.
Scores on
reading section
of classroom
exams
IV:
Regular
Reading
DV:
Reading
Achievement
Validity
• Equivalence: For a cross-cultural comparison to be
meaning, two condition must be met:
– Equivalence in the conceptual meaning of the theoretical constructs
between cultures
– Equivalence in the relevance of the empirical method between
cultures
• If any aspect of a research study is not equivalent across
cultures then the comparison not valid
Dr K. A. Korb
University of Jos
Validity
• Types of Equivalence
– Theoretical:Equivalent in meaning of theoretical framework tested
and specific hypotheses tested
– Linguistic: Words used in the instruments are similar across languages
– Measurements: Instruments are equally valid and reliable across
cultures
– Sampling: Samples are representatives of their culture and equivalent
on noncultural demographic characteristics
– Procedure: Procedures used to collect data across cultures are similar
Dr K. A. Korb
University of Jos
Participants
• Cultures to be studied
– Differ on independent variables of interest
– Check to confirm that the cultures do differ on the
independent variable
• Subgroups within the cultures
– Should be as similar as possible on non-cultural relevant
demographic characteristics
• Individuals within the subgoups
Dr K. A. Korb
University of Jos
Instruments
• Perspective 1: Use similar instruments for
each culture
– Assumption: The instrument is not biased against
a particular culture
• Conclusion: Differences between groups reflect
construct differences in each culture
– Advantage: Comparison is straight forward
– Disadvantage: The instrument might not be
culturally meaningful for all cultural groups
Dr K. A. Korb
University of Jos
Instruments
• Perspective 2: Use different instruments for each
culture
– Assumption: The same instrument cannot be equivalent
across culture because psychological constructs are
context-bound
• Conclusion: Difficult interpretation on comparisons across cultures
– Advantages: Measures behaviors that are more
appropriate for each culture
– Disadvantage
• Uncertain whether the instruments are measuring conceptually
similar constructs
• Comparison between cultures is difficult
Dr K. A. Korb
University of Jos
Instruments
• Linguistics Equivalence: Can be established through
back-translation of an instrument
– Step 1, Forward Translation: Translate from English into
Language
– Step 2, Back Translation: An independent person
translates the translation back into English.
• The instrument is considered equivalent in both
languages if the back-translation is equivalent to the
original
Dr K. A. Korb
University of Jos
Instruments
• Cultural Bias: The instruments does not measure
equivalent concepts in both cultures
– Theoretical/construct bias: The definition of the concept differs
between cultures
– Item bias: Poor item translation
– Procedural bias: Differences in testing procedures across cultures
• Testing conditions
• Family with testing procedures
• Response styles
Dr K. A. Korb
University of Jos
Instruments
• Response Style: Systematic tendency to respond in a
certain way to items on a questionnaire
–
–
–
–
Acquiescence bias: Agree with statements as presented
Central tendency bias: Avoid extreme categories
Social Desirability: Tendency to portray oneself in favorable light
Reference Group Effect
• Participant self-report by comparing themselves to others
• Individuals with different cultural will have different standards of
comparison
Dr K. A. Korb
University of Jos
Reference Group Effect
• Standard item: I have respect for the
authority figures with whom I interact.
• Cultural comparison item: Compared to most
Japanese I know, I think I have respect for the
authority figures with whom I interact.
Dr K. A. Korb
University of Jos
Reference Group Effect
Dr K. A. Korb
University of Jos
Control of External Variables
• To reduce alternative explanations of
differences between cultural groups
– Cultural populations should be selected a priori
(before the experiment) based on ethnographic
descriptions
– Measure the dependent variable by two or more
measures
– Eliminate effect of irrelevant variables though
statistical analysis (ANCOVA)
Dr K. A. Korb
University of Jos
Research Designs
• Causal- Comparative: Compare groups of people on
an independent variable to determine the effects of
an independent variable that cannot be manipulated
• Correlational: Compare one group of people on
multiple variables to determine the relationship
between variables.
• Experimental: Compare treatment and control
groups on a dependent variables. The experimenter
must implement a treatment.
Dr K. A. Korb
University of Jos
Cross-Cultural Research
• Cross- Cultural Comparisons: Studies comparing different
cultures on the same psychological variable
– Determine how individuals within cultures differ on psychological
variables of interest
• Cultural Studies: Studies based on a theoretical framework
that predicts and explains differences between cultures
– Correlation studies: Studies that determine whether the relationships
between psychological variables are similar across cultures
• Linkage studies: Studies that establish links between contents
of culture and psychological variables of interest
– Unpacking studies
– Experiments
Dr K. A. Korb
University of Jos
Unpaking Studies
• Instead of culture as independent variable, a Context
Variable is hypothesized to influence the dependent
variable
– Context variable: Specific variable that explains cultural
differences that then influences the dependent variable
• Both the context variable and the dependent
variable are measured
– Degree to which the context variable influence the
dependent variable is statically tested
Dr K. A. Korb
University of Jos
Cross-Cultural Experiments
• Priming Experiment: Experimentally manipulate
mindsets related to culture to determine whether
behavior differs as function of a primed mindsets
• Example (Trafimow,Triandis,& Goto, 1991)
– Treatment 1: Private Mindset
• Please think of what makes you different from your family and friends.
– Treatment 2: Collective Mindset
• Please think of what you have in common with your family and friends.
What do they expect from you to do?
– Participants were then asked to write a paragraph that described
themselves
Dr K. A. Korb
University of Jos
Frequency of Individually-Oriented and GroupOriented Responses for American and Chinese
(Trafimow et al., 1991)
Dr K. A. Korb
University of Jos
Conclusion
• Only compare populations that have
theoretical reasons for differing on a
dependent variable
• Replace culture with specific independent
variables that might influence the dependent
variable
– What aspects of the cultures cause differences in
the dependent variable?
Dr K. A. Korb
University of Jos
Revision
• What validity consideration need to be taken
into account in cross-cultural research
studies?
• What are some key issues to consider when
developing instruments in cross-cultural
research studies?
• Describe research designs that are useful in
cross- cultural research studies
Dr K. A. Korb
University of Jos