Experimental and QuasiExperimental Designs
Chapters 9 & 10
Research Design

It is the outline, plan, or strategy for
the procedures you will use to address
your research question.
Research Designs – What are
some limitations of these?
One-group after design
1.

Treatment
Response measure
One-group before-after design
2.



Response Measure 1
Treatment
Response Measure 2
Compare
Research Designs – What are
some limitation of these?
Nonequivalent posttest-only design
Cardiac Bypass &
Revascularization
Cardiac Revascularization
Response Measure
Cognitive Functioning
at 6-months follow-up
Response Measure
Cognitive Functioning
at 6-months follow-up
What are the requirements of
true research designs?
Requirements of True
Research Design



Design is adequate to answer the research
question (i.e., test the hypothesis).
Control for extraneous variables
Results are generalizable.
Why Pre-testing Your
Participants





Increased sensitivity of the study
Ceiling effect
Initial position
Initial comparability
Evidence of change
What are the limitations of
this design?
True Research Design
Randomized to:
Anxiety
Management
Usual Care
Response
Measure
Decisional Regret &
Distress
Response
Measure
Decisional Regret &
Distress
Factorial Design

Two or more independent variables are
studied in order to determine their
independent and interactive effects on
the dependent variable.
Interaction Effect

The effect of one factor (independent
variable) depends on the level of the
other factor (other independent
variable).
Main Effects & Interactions
High
High
High
Low
Lightness
Seasonal Affect Disorder
Non-Seasonal Affect Disorder
High
Low
Lightness
Advantages of Factorial
Designs



More than one hypothesis can be tested.
Potentially confounding variables can be
built into the design as factors.
Enables interaction effects to be tested.
Within Participant After-Only
Design





Same research participants in all
experimental treatment conditions.
Repeated measures design.
Overcomes concerns about creating
equivalence between groups.
Requires fewer participants.
Most serious limitation is the confounding
influence of a sequence effect.
Combining Within and
Between Participant Designs


Factorial design based on a mixed
model.
Can include as many independent
variables as is necessary.
Combining Within and
Between Participant Designs
Before - After Design
Participants
Selected
Pre-treatment
Measures
May want to match before
randomizing
Between-Subject
Factor
Randomized to:
Experimental
Group
Control
Group
Post-treatment
Measure
Post-treatment
Measure
Repeated
Measure
Factorial Design
Does stress management work similarly in medicated and
unmedicated hypertensives?
Unmedicated hypertensives
Randomized to:
Beta Blocker &
Stress management
Beta Blocker, no
Stress management
Placebo &
Stress management
Placebo, no
Stress management
Post measure
at 6-months
Post measure
at 6-months
Post measure
at 6-months
Post measure
at 6-months
• Main effect for beta blocker
• Main effect for stress management
• Interaction effect for drug and stress management
Main & Interaction Effects
Drug & Stress Man.
Placebo & Stress Man.
Drug, No Stress Man.
Placebo, No Stress Man.
112
108
104
100
96
92
88
84
80
Drug & Stress Man.
Placebo & Stress Man.
Drug, No Stress Man.
Placebo, No Stress Man.
112
108
104
100
96
92
88
84
80
Pre
Main Effect
Post
Pre
Post
Interaction Effect
Selecting the Appropriate
Design



The design must be one that addresses
your research question.
What control techniques can and should
you apply to help you arrive at an
unambiguous answer.
Between or within-in design or mixed
model.
Quasi-Experimental Design
Quasi-Experimental

Does not meet all of the requirements
necessary for controlling the influence of
extraneous variables.


Most common criteria not met is random
assignment.
While you cannot infer cause and effect,
well designed quasi-experiments enable
you to demonstrate that rival
interpretations are implausible.
Non -Equivalent Control Group
Design: Typical Rival Hypotheses


Increasing treatment effect I outcome
A selection-maturation effect
Increasing Treatment Effect I
Outcome
Experimental Group
Control Group
Pre-test
Post-test
Increasing Treatment and
Control Group Outcomes

Both groups’ scores increase over time
but one group changes to a greater
extent than the other group.

Effect could be due to a treatment effect or
to a selection-maturation interaction.
HADS Depression
14
12
10
Time 1
8
Time 2
6
Time 3
4
2
0
Experimental
Control
ANOVA – time and time by group effect, depression
decreased in both groups but levelled off in control
Group at follow-up while continuing to decrease in experimental.
Increasing Treatment Effect II
Outcome
Control Group
Experimental Group
Pre-test
Post-test
Cross-Over Effects
Control Group
Experimental Group
Pre-test
Post-test
Time Series Analyses

Useful when you cannot randomize
participants and where it is possible to
obtain a series of assessments of the
dependent variable at pre-treatment
and post-treatment.
Time Series Analysis
Treatment
Applied


 
   


 




Classic Studies: Effect of Reduced
Speeding on Traffic Accidents in
Connecticut



In 1955 there were a record number of
traffic accidents ( n = 324) so the Governor
(Abraham Ribicoff) introduced a law to
reduce the speed limit.
In 1956 there were 284 traffic accidents, a
reduction of 12.3%.
Governor concluded that his intervention
worked but the effects could just as easily
been due to regression to the mean.
Classic Studies: Effect of Reduced
Speeding on Traffic Accidents in
Connecticut


Campbell and Ross (1968) used interrupted
time series design to test if the reduced
trend in traffic accidents was plausible.
Compared traffic accident trend in
Connecticut with control States.
Classic Studies: Effect of Reduced
Speeding on Traffic Accidents in
Connecticut
15
Control State
14
13



•


•
•

•
10

•
12
11



•
•
•
Connecticut
•
9
‘51
‘52
‘53
‘54
‘55
‘56
‘57
‘58
•
‘59
Class Exercise




Specify your research question
Your scientific hypothesis
Specify your design
Break into groups of 5