Quasi-Experimental
Designs
Chapter 13
Quasi-Experimental Designs
 In a quasi-experimental design, the researcher lacks
control over the assignment to conditions and/or does
not manipulate the causal variable of interest.
 A quasi-independent variable is not manipulated by
the researcher but rather is an event that occurred for
other reasons.
Examples
 Does smoking cause cancer?
 Did 9/11 cause an increase in prejudice against people
of middle-eastern decent?
 Do Republican vs. Democratic presidents affect the
economy?
 Do extreme events (i.e., winning the lottery or being
paralyzed) affect day-to-day happiness?
 Does campus crime affect applicants to a university?
Questionable Internal Validity
 No control over assignment of participants to the
independent variable, so internal validity of quasiexperiments is always questionable.
 However, some quasi-experimental designs are more
internally valid than others.
 The extent to which a quasi-experimental design can
eliminate possible threats to internal validity
determines its usefulness.
Quasi-Experiments and Internal Validity
To infer that X causes Y
1. X must precede Y in time
2. X <-> Y must be related to each other
3. all other alternative explanations of the results
are eliminated through random assignment or
experimental control
With quasi-experimental designs, you can’t rule out ALL
alternative explanations, but you can try to minimize
them.
One-group pretest-posttest design
Time 1
Measure
O1
Treatment
X
Time 2
Measure
O2
 Example: Sample of violent adolescent women
 Treatment: anger management class
 Measures: aggressive behavior 1 year pre/post
 If aggressive behavior is lower at T2 than T1, can we
conclude this decrease is caused by the treatment?
One-group pretest-posttest design
Time 1
Measure
O1
Treatment
X
Time 2
Measure
O2
 Internal validity problems?
Maturation effect: between observations, participants
could have grown out of aggressive behavior
History effect: something else in the school
environment caused a decrease—less overcrowding
Testing effect: the act of assessing aggression led to
awareness of their own aggression…
One-group pretest-posttest design
Time 1
Measure
O1
Treatment
X
Time 2
Measure
O2
 For these reasons, simple pretest-posttest design
should never be used without a control condition
and other efforts to control third variables.
Nonequivalent Control Group Designs
Researcher obtains a groups of participants who are
similar to the group that receives the quasiindependent variable.
Example: Do extreme events (winning lottery, being
paralyzed) have a long-term effect on day-to-day
happiness?
Nonequivalent groups: posttest only design
Measure both groups after one receives the quasiindependent variable.
Time 1: Event
X
--
Time 2: Measure
O
O
Event (X): winning lottery, paralyzing accident
Advantage: unlike pre-post design, not subject to testing
effect.
Threat to Internal Validity: Selection bias, you can not be
sure the groups were the same before the treatment.
How could we try to address this?
Time Series Designs
Measure the dependent variable on several occasions
before and after the quasi-independent variable occurs.
Simple interrupted time series design
8 Observations
2 Observations
T1
T2
T3
T4
T5
T6
T7
T8
T9
O1
O2
O3
O4
X
O5
O6
O7
O8
4.0
4.3
3.9
4.2
2.9
3.1
3.0
3.3
4.2
2.9
Example: Does a well publicized crime spree (X) around
Howard campus impact on number of observed applicants
(O1-O8) to the university?
What if we only had the two observations?
Simple interrupted time series design
T1
T2
T3
T4
T5
T6
T7
T8
T9
O1
O2
O3
O4
X
O5
O6
O7
O8
Advantage: Can rule out maturation.
Possible Threat to Internal Validity: Contemporary history –
could be confounded with some other event that occurred at
the same time.
Interrupted Time Series with a Reversal
Reputation
T1
2
3
4
5
6
7
8
9
10
11
12
13
O1
O2
O3
O4
X
O6
O7
O8
-X
O9
O10
O11
O12
5
4
6
5
10
10
9
6
7
5
4
Example: Howard graduates earn 3 metals in one
Olympics, and earn 0 metals in the next Olympics. What
impact does that have on Howard’s reputation for
athletics?
Shows the effect of adding AND removing quasiindependent variable.
This pattern of results rules out maturation and history.
Time Series Designs
Comparative Time Series Design
 Examines two or more variables over time in order to
understand how changes in one variable are related to
changes in another variable
 Provides indirect evidence that the change in one variable
may be causing the change in the other variable
Example: Comparative Time Series Design
(Friedberg, 1998)
Time Series Designs
Control Group Interrupted Time Series Design
 nonequivalent control group included that does not
receive the quasi-independent variable
 helps rule out certain history effects
 but could still get local history effects
Experimental:O1 O2 O3 O4 X O5 O6 O7 O8
Control
:O1 O2 O3 O4 -- O5 O6 O7 O8
Control Group Interrupted Time Series
Example: Economic Output of Cities
FEMA went into New Orleans
120
100
80
New Orleans
60
Tampa FL
40
20
0
2001 2002 2003 2004 2005 2006 2007 2008 2009
Longitudinal vs. Cross-sectional Designs
Design
1990
Participant #
1995
2000
2005
Age 10
Age 15
Age 20
Longitudinal
P’s 1-25
Age 5
Cross-sectional
P’s 1-25
Age 5
P’s 26-50
Age 10
P’s 51-74
Age 15
P’s 75-100
Age 20
Longitudinal Designs
 The quasi-independent variable is time; nothing has
occurred from one observation to the next except for
the passage of time
O1
O2
O3
O4
O5
 Mostly used by developmental psychologists to study
age-related changes in how people think, feel, and
behave.
 Developmental changes in memory, emotions, selfesteem, personality…
Marital Satisfaction
Longitudinal Design
Example: Marital Satisfaction (Kurdek, 1999)
Wife
Husband
1
2
3
4
5
6
7
8
9
10
Years of Marriage
Note: Years is within-participants. Would you offer to collect
these data as a graduate student?
Longitudinal Designs
Three drawbacks:

Difficult to obtain participants who agree to be in a study
over a long period of time

Difficult to keep track of participants once they are in the
study

Repeatedly testing a sample requires time, effort, and
money
Cross-Sectional Designs
Drawback: Age is confounded with generation. Examples:
1. Texting ability by age (10, 20, 30, 40, 50, and 60 year
old Ps)
•
Younger people are faster texting
•
Does texting speed decrease with age?
•
Or are there generational effects
with experience with texting?
2. Prejudice as a function of age.
3. Attitudes toward sex in the media as a function of age.
Evaluating Quasi-Experimental Designs
Quasi-experimental designs can show that:
the presumed causal variable preceded the effect in time
cause and effect covary
Quasi-experimental designs do not:
eliminate all other alternative explanations for the results
reason: no random assignment and no experimental control
Increasing Confidence in QuasiExperimental Results
1. Measure both the effects of the quasi-IV and the
processes assumed to explain the relationship.
2. Include a noncomparable control condition to rule out
global history effects.
3…
4…
3. Find a complex pattern of results that cannot be explained by
anything except for your quasi-experimental factor.
Complex patterns of data are not sufficient by themseves…
Increasing Confidence in QuasiExperimental Results
4. Critical multiplism – employ multiple approaches
that may converge to yield conclusions that are as
concrete as those obtained in experimental research.
•
Longitudinal and cross-sectional
•
quasi and full experiments
•
different quasi-experiments for the same factor
•
replications in different populations...
Design your own Quasi-Experiment
Points to consider
-Pick an event or set of events & one DV variable
-Is it longitudinal, cross-sectional, time-series,
pre-post test, or post-test?
-Is a non-equivalent control group or reversal included?
-What is you hypothesis (prediction)?
-What confounds threaten internal validity?
-Can you improve the design to address the confounds?