Other Quasi-Experimental Designs
Design Variations
Show specific design features that can be
used to address specific threats or
constraints in the context
Proxy Pretest Design
N
N


O1
O1
X
O2
O2
Pretest based on recollection or
archived data
Useful when you weren’t able to get a
pretest but wanted to address gain
Separate Pre-Post Samples
N1
N1
N2
N2



O
X
O
O
O
Groups with the same subscript come from the same
context.
Here, N1 might be people who were in the program at
Agency 1 last year, with those in N2 at Agency 2 last year.
This is like having a proxy pretest on a different group.
Separate Pre-Post Samples
N
N



R1
R1
R2
R2
O
X
O
O
O
Take random samples at two times of people at two
nonequivalent agencies.
Useful when you routinely measure with surveys.
You can assume that the pre and post samples are
equivalent, but the two agencies may not be.
Double-Pretest Design
N
N



O
O
O
O
X
O
O
Strong in internal validity
Helps address selection-maturation
How does this affect selection-testing?
Switching Replications
N
N



O
O
X
O
O
X
O
O
Strong design for both internal and
external validity
Strong against social threats to internal
validity
Strong ethically
Nonequivalent Dependent Variables Design (NEDV)
N
N


O1
O2
X
O1
O2
The variables have to be similar enough that
they would be affected the same way by all
threats.
The program has to target one variable and
not the other.
NEDV Example
80
70
Algebra
Geometry
60
50
40
Pre



Post
Only works if we can assume that geometry scores
show what would have happened to algebra if
untreated.
The variable is the control.
Note that there is no control group here.
NEDV Pattern Matching




Have many outcome variables.
Have theory that tells how affected
(from most to least) each variable will
be by the program.
Match observed gains with predicted
ones.
If match, what does it mean?
NEDV Pattern Matching
80
Algebra
Geometry
60
Arithmetic
Reasoning
Analogies
40
Grammar
Punctuation
20
Spelling
Comprehension
0
Creativity
Exp
Obs
r = .997


A “ladder” graph.
What are the threats?
NEDV Pattern Matching



Single group design, but could be used
with multiple groups (could even be
coupled with experimental design).
Can measure left and right on different
scales (e.g., right could be t-values).
How do we get the expectations?
Regression Point Displacement (RPD)
N(n=1) O
N
O



X
O
O
Intervene in a single site
Have many nonequivalent control sites
Good design for community-based evaluation
RPD Example




Comprehensive community-based AIDS
education
Intervene in one community (e.g.,
county)
Have 29 other communities (e.g.,
counties) in state as controls
measure is annual HIV positive rate by
county
RPD Example
0.07
0
1
Y
0.06
0.05
0.04
0.03
0.03
0.04
0.05
0.06
X
0.07
0.08
RPD Example
0.07
0
1
Y
0.06
Regression line
0.05
0.04
0.03
0.03
0.04
0.05
0.06
X
0.07
0.08
RPD Example
0.07
0
1
Y
0.06
Regression line
0.05
Treated
community point
0.04
0.03
0.03
0.04
0.05
0.06
X
0.07
0.08
RPD Example
0.07
0
1
Y
0.06
Regression pine
0.05
Treated
community point
0.04
0.03
0.03
0.04
0.05
0.06
X
Posttest
displacement
0.07
0.08