Fitting Research Design to
Research Purpose
• The overall purpose of most research is to
investigate a predicted relationship between
the occurance of some variation of one
variable, A, and the occurance of variations of
another variable, B, in the same setting.
•
•
•
•
Variables may be states of the physical or social environment (e.g., weather
conditions, the number of people present in the situation), properties of a
stimulus (e.g., the facial expression in a photograph, the content of a message), or
characteristics of a person or a person’s behavior (e.g., mood state, degree of
aggression).
Relationships can be between two environmental variables (e.g., the relationship
between variations in the coldness of the weather on the number of people who
are in an outdoor setting), between an environmental or stimulus variable and an
individual characteristic or trait (e.g., the relationship between the state of the
weather and the average mood of people exposed to it), or between two
characteristics of an individual (e.g., the relationship between mood and
aggressiveness).
To say that there is a relationship between two such variables means that if the
state of one variable differs or changes, we can expect that the state of the other
will also change or differ.
So, for example, if we measure people’s mood on a sunny day and then again on a
cloudy day and there is a difference in mood such that mood is more negative on
the second occasion, then we can say we have shown a relationship between the
state of the weather and individuals’ moods.
The nature of the relationship may be specified in terms
of the form it will take, that is, what kind of changes in B
will accompany particular changes in A and what the
causal direction of the relationship will be. Directionality
may be differentiated into three types.
1. Unidirectional causation, in which changes in A are
predicted to produce subsequent changes in B, but
changes in B are not expected to influence A
(e.g., increases in the temperature-humidity index are
accompanied by an increase in aggressive responses of
rats, but the degree of aggressiveness of rats does not
affect weather conditions).
2. Bidirectional causation, in which changes in A
lead to changes in B and, in addition,
changing B produces changes in A
(e.g., perceiving threat produces feelings of
anxiety, and increasing anxiety enhances the
perception of threat).
3. Noncausal covariation (or third-variable
causation), in which changes in A are
indirectly accompanied by changes in B
because both A and B are determined by
changes in a third variable, C
(e.g., birth rate and consumption of beef steak
rise or fall with increases or decreases in the
cost of living index).
Moderators and Mediators
• In addition to specifying the nature and
direction of a causal relationship under study,
it also is important to distinguish between two
different types of “third variables” that can
influence causal relationships—moderators
and mediators
• Sometimes causal relationships can be either augmented or
blocked by the presence or absence of factors that serve as
moderator variables.
• To take another weather-related illustration, consider the
causal relationship between exposure to sun and sunburn.
Although there is a well-established cause–effect link here,
it can be moderated by a number of factors.
• For instance, the relationship is much stronger for fairskinned individuals than for dark-skinned persons. Thus,
fair skin is a moderator variable that enhances the causal
relationship between sun exposure and burning. However,
this does not mean that the sun–sunburn relationship is
spurious.
• The moderator variable (skin pigmentation) does not
cause the effect in the absence of the independent
variable (sun exposure). Other moderator variables can
reduce or block a causal sequence.
• For instance, the use of effective suntan lotions literally
“blocks” (or at least retards) the causal link between
the sun’s ultraviolet rays and burning. Thus, a
researcher who assesses the correlation between sun
exposure and sunburn among a sample of fair-skinned
people who never venture outdoors without a thick
coat of 30 SPF sunblock would be ill-advised to
conclude that the absence of correlation implied the
absence of causation.
Moderator relationships can be represented
notationally as follows:
It is important here to distinguish between third variables that
serve as moderators and those that serve as mediators of a
cause–effect relationship.
With moderator effects, the causal link is actually between X
and Y, but the observed relationship between these two
variables is qualified by levels of variable C, which either
enhances or blocks the causal process. A mediational relation,
on the other hand, is represented as follows:
In this case, the presence of C is necessary to
complete the causal process that links X and Y.
In effect, varying X causes variations in C, which,
in turn, causes changes in Y.
To return to our weather examples, the effect of rain (X)
on depression (Y) may be mediated by social factors (C).
Rain (X) causes people to stay indoors or to hide behind
big umbrellas, hence reducing social contact (C). Social
isolation (C) may, in turn, produce depression (Y).
However, rain may not be the only cause of social
isolation.
In this case, rain as an independent variable is a sufficient,
but not necessary, cause in its link to depression. To
demonstrate that X causes Y only if C occurs does not
invalidate the claim that X and Y have a causal
relationship; it only explicates the causal chain involved.
FORMS OF VALIDITY
• The research strategy should be be guided by
considerations of two types of validity—internal
and external validity
• Internal validity has to do with the certainty with
which one can attribute a research outcome to
the application of a treatment or manipulation
that is under the rigid control of the researcher.
• Internal validity is about the extent to which
causal inferences can legitimately be made about
the nature of the relationship between the
treatment and the outcome.
• Just as the choice of research method must be
conditioned on considerations of the nature of
the phenomenon of interest, so too must the
role of statistical techniques be evaluated with
respect to the general goal of eliminating or
reducing the plausibility of rival alternative
hypotheses for the events under investigation.
• One potential rival explanation that plagues social
research at all stages of investigation is the operation
of “chance.”
• The phenomena of interest to the social sciences are
generally subject to considerable nonsystematic
variation, that is, variations from individual to
individual and, within individuals, from time to time.
• The purpose of most inferential statistical tests is to
assess the validity of this rival explanation of results in
terms of the probability, or likelihood, that the
obtained data pattern could have occurred by chance.
• The results of a statistical inference test tell us
the probability of a Type I error of inference—
the likelihood that a result would be obtained
when the null hypothesis (no true relationship
between the independent and dependent
variable) is actually valid.
• Statistical significance is achieved when this
probability is so low as to render the chance
explanation implausible.
• External validity is concerned with the issue of
generalizability. Assuming that a research
finding is internally valid, external validity has
to do with the extent that it can be
generalized to other respondent groups, to
other settings, and to different ways of
operationalizing the conceptual variables.
• Even when internal validity is high, however,
there may arise questions about the validity of
interpretations of causal effects obtained in any
given study, particularly their applicability or
generalizability outside of the experimental
setting. These concerns constitute questions of
external validity, which can be further divided
into questions of
(1) generalizability of operationalizations and
(2) generalizability of results to other places
and participant populations
• Validity of Operationalizations corresponds to
misusing a statistical technique such as scale
vs. technique conflict or failure of assumptions
that the technique rests on.
• Once a research study has been completed, the investigator
is usually interested in reaching conclusions that are
generalizable across people and across settings.
• Threats to this form of external validity arise from possible
interaction effects between the treatment variable of
interest and the context in which it is delivered, or the type
of participant population involved. (a’ka random sampling)
• An experimental finding lacks external validity if the nature
of the effect of the independent variable would be reduced
or altered if the setting or the participant population were
changed.
• Need to cover basic concepts in probability
and statististics for reviewing reliability.