C H A P T E R
18
Experimental and
Quasi-Experimental
Research
Chapter ??
Chapter Outline
•
•
•
•
•
•
Sources of invalidity
Threats to internal validity
Threats to external validity
Controlling threats to internal validity
Controlling threats to external validity
Types of designs
Experimental Research Tries
to Establish Cause and Effect
• Selection of a good theoretical framework
• Application of appropriate experimental
design
• Use of correct statistical model and analysis
• Proper selection and control of independent
variables
• Appropriate selection and measurement of
dependent variables
• Correct interpretation of results
Three Criteria for Cause and Effect
1. The cause must precede the effect in time.
2. The cause and effect must be correlated
with each other.
3. The correlation between cause and effect
cannot be explained by another variable.
If the condition is necessary and sufficient
to produce the effect, then it is the cause.
Reviewing Important Terms
•
•
•
•
•
Independent variable (IV)
Dependent variable (dv)
Categorical variable
Control variable
Extraneous variable
Distinguishing Between
Types of Validity
• Internal validity: did the treatments (IV)
cause the change in the outcome (dv)?
• External validity: to what populations,
settings, or treatments can the outcome be
generalized?
• Is there a trade-off between internal and
external validity?
• Can a series of studies address the tradeoff?
Threats to Internal Validity
• History: events that are not part of
treatment
• Maturation: events due to passage of time
• Testing: effects of more than one test
administration
• Instrumentation: change in calibration of
measurements
• Statistical regression: selection based on
extreme score
(continued)
Threats to Internal Validity
(continued)
• Selection biases: nonrandom participant
selection
• Experimental mortality: differential loss of
participants
• Selection–maturation interaction: passage
of time influencing groups differently
• Expectancy: influence of experimenters on
participants
Threats to External Validity
• Reactive or interactive effects of testing:
Pretest may make participants sensitive to
treatment.
• Interaction of selection biases and treatment:
Treatment may work only on participants
selected on specific characteristic.
• Reactive effects of experimental arrangements:
Setting constraints may influence
generalizability.
• Multiple-treatment interference: One treatment
may influence the next treatment.
Controlling Threats
to Internal Validity
• Randomization
– Real randomization
– Matched pairs (not matched groups)
– Randomizing treatments or counterbalancing
• Placebos
• Blind setups
(continued)
Controlling Threats to
Internal Validity (continued)
• Double-blind setups
• Reactive effects of testing: eliminate
pretest.
• Instrumentation
– Calibration and test reliability
– Halo effects
• Experimental mortality: keeping participants
Controlling Threats
to External Validity
• Selecting from larger population
– Participants
– Treatments
– Situations
• Ecological validity: does the setting capture
the essence of the real world?
Types of Designs:
Pre-Experimental Designs
One-shot studies
T
O
One-group pretest-posttest
O1
T
O2
Statistical analysis?
Static group comparison
T
O1
--------------O2
Statistical analysis?
Types of Designs:
True Experimental Designs
Randomized-groups design
R
T
O1
Statistical analysis?
R
O2
Extending the levels—randomized-groups
design
R
R
R
T1
T2
O1
O2
O3
Statistical analysis?
(continued)
Types of Designs:
True Experimental Designs (continued)
(continued)
Types of Designs:
True Experimental Designs (continued)
A factorial design with a categorical factor (B)
R
A1
B1
R
A2
R
A3
---------------------R
A1
B2
R
A2
R
A3
O1
O2
O3
Statistical analysis?
O4
O5
O6
(continued)
Types of Designs:
True Experimental Designs (continued)
Pretest—posttest randomized-groups
R
R
O1
O3
T
O2
O4
Statistical
analysis?
Extending the design on the RM factor
R
O1 T
O2 T
O3
R
O4
O5
O6
Statistical
analysis?
(continued)
Types of Designs:
True Experimental Designs (continued)
Extending the pretest—posttest randomized
groups design on both factors
R
R
R
O1
O4
O7
T1
T2
O2
O5
O8
T1
T2
O3
O6
O9
Statistical analysis?
(continued)
Types of Designs:
True Experimental Designs (continued)
Solomon four-group design—purpose
R
O1
T
O2
R
O3
O4
R
T
O5
R
O6
Statistical analysis (factorial ANOVA)
No treatment Treatment
Pretested
O4
O2
Unpretested O6
O5
Quasi-Experimental Designs:
Reversal (Figure 18.1)
Quasi-Experimental Designs:
Time Series (Figure 18.2)
Quasi-Experimental Designs:
Ex Post Facto
This is one of the preexperimental designs,
but with the treatment not under the control of
the experimenter.
T
O1
-------------------O2
Statistical analysis?
Quasi-Experimental Designs:
Switched Replication
Quasi-Experimental Designs:
Single Participant
Identify participant and follow over time.
• Does the treatment produce the same effect
each time?
• Are treatment effects cumulative, or does
participant return to baseline?
• Does participant’s response become less
variable over treatment times?
(continued)
Quasi-Experimental Designs:
Single Participant (continued)
• Is participant’s magnitude of response
sensitive to multiple treatment
applications?
• Do varying intensities, frequencies, and
lengths of treatment produce varying
responses?