Comprehensive Exams/Dr. Mary Beth Oliver
Nan Yu/March 14, 2008
Questions:
This question asks you to discuss what you consider to be the most important
methodological approaches and data analytic strategies that may be useful to you in
studying persuasion in the context of message framing and social distance. (If this
topic is too different from your dissertation research interests, you can shift the
focus/topic of investigation for purposes of this question.)
In framing answer to this question, first provide one or two hypotheses that you
would like to explore within this area of research.
Please note that your IV(s) and DV(s) should be readily identifiable from the
hypothesis(ses) that you provide.
From there, identify what you would consider to be an appropriate design for
examining your hypothesis/hypotheses. Your discussion should include how you
would measure/manipulate your variables and prepare your data for analysis,
various considerations about experimental/survey procedures that you would need
to entertain depending on the research design, etc. Your discussion should not only
provide an explanation of what you perceive to be the strengths of the methodology
and measurement that you choose, but also a consideration of the limitations or
potential problems that you may encounter and the ways that you may address these
limitations.
Finally, discuss what you believe would be the most accurate and fruitful ways that
you could analyze your data to answer your questions of interest. Your response
here should reflect a broad understanding of various statistical approaches, their
applicability to different types of data, their assumptions, etc.
The following discussion will emphasize on three major sections: 1) a brief
introduction of the issue context and theoretical framework for my hypotheses; 2) the
method outlines that are designed to test the proposed hypotheses; and 3) the potential
data-analysis strategies. Given that no data has been collected for the proposed studies,
1
the discussion will primarily focus on the principles and guidelines.
Introduction and Hypotheses (a brief version)
Issue selection
Influenza (the flu) and flu shots are chosen as the topics of
interest for the proposed studies. Influenza refers to “contagious respiratory illness caused
by influenza viruses, (Influenza, n.d., p1). The consequences of influenza could be severe
and can even lead to death. Each year, more than 200,000 people in the U.S. are
hospitalized because of flu complications and 36,000 people will die from flu (Influenza,
n.d., p1). Getting a flu shot (a flu vaccine) before the flu season comes is suggested by
medical professionals for years and is believed as one of the most efficient ways to
prevent influenza. In sum, this particular issue is chosen because the significance of the
health hazard which also can be prevented by a relatively mature method. The primary
goal of the study is to investigate the combined persuasive effects of social distance and
message framing on the issues of influenza and flu shots. In addition, I am also interested
in examining the effect of social distance in various cultural backgrounds.
One of the potential challenges of choosing influenza and flu shots as the topics of
interest is that they lack of novelty due to long-term media exposure and people’s prior
experiences with them, both of which can bias the post-treatment measures. However,
I’ve conducted a short and small survey among both American (N=23) and Chinese
(N=17) students at a northeastern university. The results showed that 65 percent of
Americans and 80 percent of Chinese participants have never taken the flu shots. On the
other hand, participants from both countries universally held a variety of
misunderstanding of both the disease and the vaccine (e.g. flu shots can cause flu, flu
shots are useless, influenza is actually good for enhancing your immune system. etc). Due
to the small sample size, I cannot really generalize these findings to a greater population,
2
but at least they could imply that the public awareness of influenza and flu shots are not
as high as what people expected and people are holding inaccurate understanding of the
issue.
Theoretical framework for H1
To take a flu shot to prevent influenza is a
prevention-type health behavior. According to message framing theory, gain-framed
appeals should be more effective in persuading people to take measures to prevent
disease (Rothman, Bartels, Wlaschin, & Salovey, 2006). However, Construal Level
Theory predicts an interaction effect between psychological distance and the effect of
message framing. More specifically, potential gains are associated with high-level
construals and become more salient when the psychological distance increases. Potential
losses are associated with low-level construals and become more salient when the
psychological distance decreases. As a result, if a health hazard is psychological proximal
to an individual, it could activate thoughts or actions to reduce the hazard. If a health
benefit is psychological distant to an individual, it could activate thoughts or actions to
seek for the benefit (Liberman, Trope, & Stephan, 2007; Chandran & Menon, 2004).
Social distance is one dimension of psychological distance. The rest includes temporal
distance, spatial distance, and hypotheticality (Liberman, Trope, & Stephan, 2007, p.353).
From the perspective of Construal Level Theory, I expect that the presentation of
social distance in the message will interact with the gain – or loss frames and will have an
impact on people’s perceived risk or affect people’s perceptions of influenza and their
judgments or behavior intent to prevent influenza. In sum:
H1a: A loss-framed PSAl with an emphasis on socially proximal objects
will lead to increased persuasive effectiveness of the ad, which includes:
i)
higher level of perceived severity of influenza
3
ii)
higher level of perceived fear
iii)
higher intention to take actions to prevent influenza
H1b: A gain-framed PSA with an emphasis on socially distant objects will lead to
higher intention to take actions to prevent influenza.
In H1a and H1b, independent variables (IVs) are 1) frame valence (loss v. gain)
and 2) social distance (self v. other) and both are manipulated in the message. Dependent
variables (DVs) encompass variables about attitudinal, affective and behavioral responses.
Figure 1 illustrates the hypothesized interaction effect of social distance and message
frame.
Effectiveness of the health message
Social distance
High
Self
Other
Low
Frames
Gain
Loss
Figure 1. Hypothesized interaction effect of social distance and
message framing on the effectiveness of a health message
Theoretical framework for H2
When linking Construal Level Theory to
cultural psychology, the effects of the social distance would be different in individualist
or collectivist cultures. Previous literature in cultural psychology suggests that people
4
from individualist cultures are more independent from one another, whereas people from
collectivist cultures are more group-bind and more likely to hold interdependent views
(Markus & Kitayama, 1991; Shavitt, Lalwani, Zhang, & Torelli, 2006). In other words,
people with independent views should be more sensitive to the distinction between self
and other, whereas people with interdependent views should be less responsive to this
difference.
From this perspective, I expect that in the collectivist cultures like China, Korea
or India, the persuasive effects of social distance will not be as significant as it would be
in the individualist cultures like the U.S. or some European countries. Therefore, I
propose that when the valence of the frame is held constant and neutral:
H2a: For subjects from individualist cultures, a PSA with an emphasis on socially
proximal objects will lead to higher level of perceived severity of influenza and higher
intention to perform related preventive behaviors than the one with an emphasis on
socially distant objects.
H2b: The effect detected in H2a will not be significant among subjects from
collectivist cultures.
In H2a and H2b the IVs are 1) Cultural orientation (Chinese v. Americans) and 2)
Social distance (self v. other). The DVs include variables about attitudinal and behavioral
responses. Figure 2 illustrates the hypothesized interaction between social distance and
cultural orientation.
5
Effectiveness of the health message
Cultural Orientation
High
Individualists
Collectivists
Low
Social distance
Self
Other
Figure 2. Hypothesized interaction effect of cultural orientation and
social distance on the effectiveness of a health message
the effectiveness of a health message
Method
Two studies will be conducted for the purpose of investigating the two different
sets of hypotheses. Study 1 will be used to examine H1a and H1b. Study 2 will be
applied to test H2a and H2b. The discussion in this section will contain the design
strategies for both studies.
Participants
Study 1
Participants will be recruited from undergraduate classes at a
northeastern university. They will receive a small portion of extra credit for participating
in the study. The age, gender, race, and questions related socioeconomic status (SES) will
be measured in order to understand the features of the participants. In addition, the prior
experience with influenza and flu shots will be also measured in order to control their
possible moderating effects when necessary.
6
Study 2
The participants in study 2 are expected to have different cultural
orientations: individualistic versus collectivistic. Therefore, participants should include
people from both cultures. Americans and Chinese are chosen to represent individualist
and collectivist cultures based on Hofstede’s (1980, 1983) classifications of countries’
cultural orientations.
American participants will be recruited from the undergraduate students in a
northeastern university who were born and grew up in the United States. Chinese
participants will be recruited from the international students who were born and grew up
in Chinese societies but are currently in the United States. These participants should be
bilingual (i.e. fluent in both Chinese and English) so they can understand the
English-language ad appeal without linguistic obstacles. An alternative approach would
be to recruit participants from a university at Hong Kong. Chinese who live in Hong
Kong will also be bilingual and are more collectivistic than individualistic (e.g. Aaker &
Sengupta, 2000).
There has been constant criticism against using just student samples for empirical
studies because it could limit the external validity of the research. But I think a student
sample could be ideal for these two studies because that campus life always involves
activities and frequent contacts within a high-density population and a limited space
which might cause the quick transmission of the flu virus. Therefore, the issue itself is
highly relevant to the targeted population.
Experimental design
Study 1
The study will employ a 2 (loss versus gain frames)  2 (self versus
other appeals), between-subjects factorial experimental design in which participants read
one of the four designed print advertisements about influenza and flu shots. Subsequent
7
to reading the ads, participants will be asked to spend sometime to finish a questionnaire.
Study 2
The study will employ a 2 (socially proximal appeal versus socially
distant appeal) x culture orientation (Chinese v. Americans), between-subjects factorial
experimental design in which participants from both cultures will read one of the two
designed print advertisements about influenza and flu shots. After reading the ads,
participants will be asked to spend sometime to finish a questionnaire.
Message design
Study 1
The four messages will be formatted to look like public service
announcements (PSAs) on magazines or newspapers. The PSAs will contain text and
pictures and will be specifically created for this study. The four different combinations of
PSAs will comprise: 1) a loss frame with an emphasis on socially proximal objects (i.e.
self); 2) a loss frame with an emphasis on socially distant objects (i.e. other); 3) a gain
frame with an emphasis on socially proximal objects (i.e. self); and 4) a gain frame
appeal with an emphasis on socially distant objects (i.e. other).
When PSAs are loss-framed, the message will focus on the consequences of
getting influenza because of not getting a flu shot (e.g. flu symptoms, hospitalization,
impacts on routine work or life schedule, and flu complications, etc). When PSAs are
gain-framed, the message will focus on the good outcomes of getting a flu shot to prevent
influenza (e.g. free from all the bad consequence of getting influenza). The manipulation
of the socially proximal and socially distant appeals will be revealed both in pictures and
the text. When the PSAs emphasize on socially proximal objects, a picture of one
individual would be used, and bad consequences of getting influenza (or the good aspects
of not having influenza) will target on “you (i.e. self).” When the PSAs emphasize on
socially distant objects, a picture of a group of people would be used, the consequences of
8
getting influenza (or the good aspects of not having influenza) will target on “average
people (i.e. other).”
Other aspects of the message features will be held constant through the four ad
appeals such as font, size of the pictures, layout of the PSAs and length of the text, etc. I
am hoping to provide a more detailed description of the message stimuli can be provided
when the PSAs are completed. In sum, both of the IVs (i.e. social distance and frame
valence) of the first study will be manipulated in the PSAs. Due to the lacking of previous
examples of how social distance can be manipulated in the message, a pre-test using a
different sample from the studies will be conducted before finalizing the message design.
Study 2
The study 2 will use two PSAs with valence-neutral frames but with
different emphasis on socially proximal objects (i.e. self) or socially distant objects (i.e.
other). The manipulations of “self” and “other” will be similar to the ones used in study 1.
Therefore, only one IV (social distant) is manipulated in study 2. The other IV (cultural
orientations) will be measured using existing measurements (Singelis, 1994).
Manipulation check
The purpose of conducting manipulation check is to investigate whether
participants can actually detect the differences that are manifest in the manipulation. The
questions related to the manipulation check will be placed at the end of the questionnaire
to prevent participants’ possible awareness of the purposive manipulation. Participants
will be asked to evaluate the degree to which they agree or disagree with each of the four
statements concerning manipulation checks on a 10-point (1=strongly disagree to
10=strongly agree) Likert-type scale. To examine the manipulations of loss- versus gain
frames, participants will be asked to evaluate statements like “the ad communicates the
positive (negative) implications of (not) getting flu shots.”
9
To investigate the manipulations of the message emphasis on self or other,
participants will be asked to evaluate statements such as “the PSA primarily indicates the
influenza is a health issue that is close to me (average people) ,” and “the PSA primarily
suggests that getting flu shots will be beneficial to me (average people).” Given that the
manipulation of “self” and “other” is similar in study 2, I will use the similar measures
for the manipulation check of the social distance in the second study.
Procedure
Study 1
Participants will be invited to participate in an online study. They will
first see a Web page containing brief introduction of the research and an informed consent
letter. The informed consent letter will indicate 1) participants’ rights to quit the study at
any time they want; 2) the guarantee of confidentiality protection; and 3) the contact
information of the researcher. Participants are told that they will evaluate a health-related
print PSA online. They will also be asked to read the PSA as they normally would. By
clicking the link to enter the formal study, participants will be able to indicate their
consent of their participation. At the same time, the click will randomly assign each
participant to one of the four PSAs that are designed for the first study.
Subsequent to reading the PSAs, participants will be asked to complete an online
questionnaire with measures of post-experimental attitudes, affective responses and
behavioral intentions. The manipulation-check questions and demographic questions will
be placed after the questions regarding the post-experimental effects. The participants
will be thanked and debriefed when they submit their answers online. The entire session
will take approximately 30 minutes.
Study 2 will follow a similar procedure as the one in study 1. The only difference
is that participants, both Chinese and Americans, will be randomly assigned to one of the
10
two designed PSAs when clicking the link to enter the study. The online questionnaires
for the measures of cultural orientations (Singelis, 1994) will be added before participants
see the PSAs.
For both studies, students will receive an invitation email before their
participations and they will be asked to register for the research. In the following email,
the students will be offered a link which leads them to the informed consent letter and the
formal study as described above. Participants will not be asked to come to a lab to
complete the study. They can join the study at any computers that have Internet
connections at their convenience. The potential risk is that I cannot monitor the process in
which they read the PSAs and complete the questionnaires.This may introduce confounds
into the study. But I am expecting the random assignment can minimize these influences.
The advantage of asking participants to do it on their own time and computer is to reduce
the effects of lab environment and allows the surrounding to be as neutral as possible.
Measures
Measures for post-experimental effects will be identical for both studies.
Intention to prevent influenza
A set of behavioral intent measures will be used
to evaluate the likelihood that participants would take actions to prevent flu, including
taking a flu shot before flu seasons; washing hands frequently; avoiding touching nose,
eyes and mouths; avoiding close contacts with people who have influenza, eating healthy;
getting enough sleep; exercising regularly, covering mouth and nose when coughing and
sneezing, staying home when having influenza. Each behavioral intention will be rated on
three 10-point semantic differential scales with paired words: not likely/likely,
improbable/probable, and impossible/possible.
Perceived severity
Three items will be constructed to evaluate the degree to
11
which participants believed that influenza is a severe disease. Participants will be asked
to rate on 10-point semantic differential scales on the following word pairs: not
dangerous/very dangerous, not fearful/very fearful, and not a severe health problem/a
very severe health problem. An index of perceived severity will be constructed by
calculating the mean of the three items if the Cronbach’s α is above .07.
Perceived fear
Affective reactions to the messages will be assessed by asking
participants to rate how they feel after reading the PSAs on a 10-point scale (0=not at all
and 10= a lot). Five items, including frightened, tense, nervous, anxious and
uncomfortable, will be used to encompass fear. The mean of the five items will be
computed to create an index of perceived fear if the Cronbach’s α is above .07.
Perceived external efficacy
Four statements will be constructed to evaluate
participants’ beliefs that influenza can be prevented by taking the flu shots and living
healthy. Participants will be asked to indicate the degree to which they agree or disagree
with the statements on a 10-point (1=strongly disagree to 10=strongly agree) Likert-type
scale. The three statements propose that by taking the flu shots and living healthy the
influenza is 1) preventable, 2) difficult to prevent, 3) easy to prevent if taking the flu shots
4) easy to prevent if live healthy. An index of perceived external efficacy of influenza
prevention will be formed by reverse-coding the item of difficult to prevent and
calculating the mean of these four items if the Cronbach’s α is above .07.
Perceived internal efficacy
Three statements will be included to assess
participants’ beliefs about their own ability to prevent influenza. Participants will be
asked to indicate the degree to which they agree or disagree with the statements on a
10-point (1=strongly disagree to 10=strongly agree) Likert-type scale. The two statements
are: 1) “I am capable of preventing influenza;” 2) “Preventing FASD is easy for me;” and
12
3) “Preventing FASD is not difficult for me.” A reverse coding of the third item will be
needed before creating an index of perceived internal efficacy of influenza prevention
with the mean of these three items (with a Cronbach’s α greater than .07).
Tentative strategies for data analysis
Study 1
When the data is collected, the first step would be to make the data as
“clean” as possible before conducting further data analysis. The “cleaning” step should
include defining missing values, recoding the reverse-coded items, and combining
multiple items to created a new index that represents one DV (only if Cronbach α is
above .07). Since all the DV measures are on the same metric, I don’t have to worry
about converting them to Z scores.
I believe that using MANOVA as the first step of data analysis strategy for study
1 is appropriate because both of the IVs are categorical.There are multiple DVs all of
which are outcome variables and measured at interval level. MANOVA is good to test the
effect of multiple IVs on a combination of DVs while protecting the family-wise alpha.
However, before I conduct the MANOVA test, I need to check whether there are
any violations of assumptions of MANOVA. First, I need to check whether there are
univariate outliners by computing Z scores for each DV. A value that is greater than
+3.29 or less than -3.29 would be an outliner. Second, I need to check for multivariate
outliers by testing the Mahalanobis Distance. Third, I need to check for multivariate
normality assumption. If the sample size for each cell is greater than 20 and all the cell
sizes are really equal, this assumption will be satisfied. Forth, I can use Levene’s test to
check Homogeneity of Variance assumption and use Box’s M test to check the
assumption of Homogeneity of the Variance and Covariance Matrices. However, both
tests can be too sensitive. Therefore, if the cell sizes are relatively equal (min: max < 1:4),
13
and at the same time the Fmax (=S2max/ S2min) is less than 10, then the two assumptions will
be satisfied. If all the assumptions that were mentioned are checked and no violations are
found, I can go ahead to perform MANOVA.
Following the checks for assumptions, a 2 (loss versus gain frames) X 2 (self
versus other appeals) multivariate analysis of variance (MANOVA) will be conducted to
examine participants’ responses toward perceived external efficacy, perceived internal
efficacy, perceived severity, perceived fear, and behavioral intentions to prevent
influenza.
If any main effects of interaction effects are found significant at .05 level for any
of the DVs in the MANOVA test, the next step will be to conduct a factorial ANOVA on
the each of those DVs. For example, in the scenario of this study, if any main or
interaction effect on perceived severity is found significant at .05 level in the previous
MONOVA test, I will continue to conduct a 2 (loss versus gain frames) X 2 (self versus
other appeals) factorial ANOVA on the perceived severity.
The assumptions in factorial ANOVA include: 1) multiple IVs that are categorical
and one interval or ratio DV; 2) cases are independents and randomly sampled or
assigned; 3) normality of one DV on each level of the IV (if cell size is relatively equal
or >20); and 4) Homogeneity of Variance (Levene’s test or Fmax<=10). I need to check all
of these before I can conduct factorial ANOVA for study 1.
Other types of statistic analysis, such as Chi-square, independent-sample t-test,
correlations and regression are not suitable for this study basically because the type of
data in this study doesn’t fit the assumptions for these analyses. More specifically:
1) Chi-square test is normally applied when both IV and DV are categorical;
2) Independent-sample is used for nominal IV with two levels, and one interval
14
and ratio DV;
3) Correlation is for only two variables that are interval, ratio or dichotomous;
4) Regression (simple or multiple regression) can be used when the study has one
DV and one or more IVs. However, it requires large and sufficient sample size. It’s not
commonly used when the goal is to compare group differences.
The current study contains two IVs that are both categorical, and the outcome
DVs that are interval. Therefore, using MANOVA as the first step and factorial ANOVA
as the second step for data analysis fits the type of data in this study and satisfies the
related assumptions.
Study 2 will follow a similar procedure of analysis in study 1. After checking the
required assumptions, I will perform a 2 (self versus other appeals) X 2 (Chinese versus
Americans) multivariate analysis of variance (MANOVA) on perceived external efficacy,
perceived internal efficacy, perceived severity, perceived fear, and behavioral intentions
to prevent influenza. If any main effects of interaction effects are found significant at .05
level for any of the DVs in the MANOVA test, the next step will be to conduct a 2 X 2
factorial ANOVA on the each of those DVs.
(Author’s note: A majority of the descriptions and terms related to data analysis
strategies are derived from the class lecture of Data Analysis taught by Dr. Oliver).
15
References
Aaker, J. L. & Sengupta, J. (2000). Additivity versus attenuation: The role of culture in
the resolution of information incongruity. Journal of Consumer Psychology,
9(20), 67-82.
Chandran, S. & Menon, G. (2004). When a day means more than a year: Effects of
temporal framing on judgments of health risk. Journal of Consumer Research,
31, 375-389.
Hofstede, G. (1980). Culture’s consequences: International differences in work-related
values. Beverly Hills, CA: Sage.
Hofstede, G. (1983). Dimensions of national cultures in fifty countries and three regions.
In J. Deregowski et al. (Eds.) Explications in cross-cultural psychology. Lisse,
The Netherlands: Swets and Zeitlinger.
Influenza: The disease. (n.d.) Centers for Disease Control and Prevention. Retrieve
March 12, 2008, from http://www.cdc.gov/flu/about/disease/index.htm
Liberman, N., Trope, Y., & Stephan, E. (2007). Psychological distance. In A. W.
Kruglanski & E. T. Higgins (Eds.), Social psychology: Handbook of basic
principles (Vol. 2, pp. 353-383). New York: Guilford Press.
Markus, H. R. & Kitayama, S. (1991). Culture and the self: Implications for cognition,
emotion, and motivation. Psychological Review, 98(2), 244-253.
Rothman, A. J., Bartels, R. D., Wlaschin, J., & Salovey, P. (2006). The strategic use of
gain- and loss-framed messages to promote healthy behaviors: How theory can
inform practice. Journal of Communication, 56, 202-220.
Shavitt. S., Lalwani, A., Zhang, J., & Torelli, C. (2006). The horizontal/vertical
distinction in cross-cultural consumer research. Journal of Consumer Psychology,
16(4), 325-342.
Singelis, T. M. (1994). The measurement of independent and interdependent
self-construals. Personality & Social Psychology Bulletin, 20(5), 580-591.
16