Research Design
Formal and practical
considerations
Ch 9 M&H
Experimental design
Three aspects of the experiment play the
biggest part in determining the design
1.
2.
3.
Number of independent variables
Number of treatment conditions
Are treatments between- or withinsubject
Experimental design
General plan or structure of the
experiment
Building a floor plan analogy
How many rooms
How they are connected
Does not specify the content of rooms
Is the analogy perfect?
The analogy
My first apartment (a 1 ½)
Indpenedent Variable: Drug X
Control
Dpenedent Variable:
Virtual Water Maze
Treatment
How many Subjects?
 Dependent variable – to what degree can responses
vary?
 Greater variability, more subjects are needed
 Less variability, fewer subjects are required
 Effect size – statistical estimate of the size or magnitude
of the treatment effect.
 Large effect – 10-20 subjects/group
 Moderate effect – 20-30 subjects/group
 Small effect - 30 or more subjects/group
 Practical considerations
 Length of testing sessions
 Review prior research
 As a general rule, 15-20 subjects/treatment is advisable.
Obtaining subjects
Rosnow and Rosenthal (1976) have suggested some
methods to get subjects to volunteer.
•
the appeal should be interesting, nonthreatening
and meaningful (it’s a green-friendly exp)
•
get someone who knows the subjects to ask
(preferably a woman of high status)
•
point out that the research can help others and that
lots of people volunteer (“lots” could be more than
one, right?)
•
cold hard cash (if you have lots to spare) or a token
treat like a stick of gum (for college students course
credit)
Who not to recruit
Not a good idea to recruit
friends because:
1.
it could cause problems
2.
friends know you and may be able to
figure out the hypothesis if you
haven’t already told them
3.
they might feel obliged to participate
One independent variable
Two groups design
1.
2.
Independent groups design
Matched groups design
Differ in how subjects are assigned to
treatment conditions
Random selection and
assignment
Random selection
-
All members of the population have an
equal chance of being selected
Random assignment
-
After sample is drawn, similar
randomization procedures are used to
assign subjects to treatment conditions
One independent variable
Two independent groups – the makeup of
one group is independent of the other
group
1.
2.
Experimental group – control group
design
Two experimental group design
Experimental group –
control group design



Random assignment
Treatment is applied to the experimental
group
Control group does not receive the
treatment
Experimental group –
control group design
What is the proper control group in a drug study?
Placebo or vehicle control
1) physiological saline
2) other vehicle (peanut oil)
3) inactive compound
4) selectively active compound
The ethical dilemma of clinical trials
Two experimental groups
design



No control group, rather two
experimental groups
Used in follow-up studies where an effect
relative to a control group has already
been established
The Vioxx-naproxen comparison
http://www.npr.org/templates/story/story.php?storyId=4753601
Two Matched Groups



Groups are matched on a variable that is likely
to affect the DV. (e.g., body weight in a diet
study).
Can match either before or after the experiment
Different kinds of matching



Precision matching - pairs must have identical scores
Range matching – pairs fall between a specified range
of scores
Rank-ordered matching – scores are rank-ordered and
pairs derived from adjacent scores.
Two Matched Groups
Advantages:

can specifically eliminate a source of
confounding

Easier to get significance with statistical
tests

Useful with small numbers of subjects
Two Matched Groups
Caution/Disadvantages:

The matching variable has to be highly related
to the DV

Matching on one variable could produce
differences on other extraneous variables, which
could reduce effects produced by the IV

Major disadvantage – other confounds that we
do not think about ahead of time are not
controlled for.
Multiple groups design
More than two groups of subjects.

Each group is given a different treatment
condition

Most common type is the Multiple independent
groups design
E.g., a drug study with a control (zero condition)
and at least two other groups receiving different
doses of a drug).

IV: Drug effect
Control
Dose 1
Dose 2
Choosing levels of treatment



Control (zero condition) is definitely
appropriate if it has not been done or if
you want to replicate a basic effect
Levels used by others are acceptable
New levels may be introduced (pilot
studies)
Research Design:
Between-subjects
Factorial designs
Designs with one IV

Two Group Designs (two levels)

Two independent groups




Experimental group - control group
Two experimental groups
Two matched groups
Multiple Groups (more than two levels)


Control versus two or more experimental groups
Three or more experimental groups
Random assignment with multiple
groups



Two groups – easy, just flip a coin
Three or more groups – random numbers
table or random numbers generation in
Excel ‘=RAND() command.’
Block randomization – each condition is
randomly selected once for each block.
Randomly assign subjects to fill in the
blocks. This guarantees an equal number
of subjects for each condition.
Block randomization

Randomly select numbers corresponding to
treatment conditions from a random
numbers table
Blocks for 3 treatments (n = 8)
1
3
2
2
2
1
1
1
2
1
3
1
1
2
3
2
3
2
1
3
3
3
2
3
Multiple groups
Hormesis

Often used when there is
a nonlinear relationship
between the IV and DV
E.g. the effect of caffeine
(cups of coffee) on
learning performance
8
8
7
7
6
6
Learning index
Learning index

is a dose-response phenomenon
characterized by a low-dose effect
opposite to a high-dose effect, resulting
in either a J-shaped or U-shaped dose
response curve.
Linear dose-response
Inverted U-shaped dose-response
5
4
3
2
5
4
3
2
1
1
0
0
1
2
Cups of coffee
3
1
2
3
Cups of coffee
4
Factorial design


More than one IV (called factors)
Advantages:



One experiment instead of two
Fewer subjects may be needed
Can study interactions in addition to main
effects
Simple Factorial Design
Two factors
 Two treatments (a.k.a., conditions or levels) for
each factor
Example (water maze):
Sex x Cue condition (2 x 2)
Hypothesis: Females will perform better than
males when navigating in the presence of distal
cues but not when navigating in the presence
of proximal cues.

Simple Factorial Design
Cue condition
Sex
Distal
Proximal
F
Female
Distal
Female
Proximal
M
Male
Distal
Male
Proximal
Simple Factorial Design
How many results?
 Two main effects (Sex and cue
condition)



Did males perform better than females?
Is distal better than proximal?
One interaction (Sex x cue condition)

Does the effect of sex differ at different
levels of the cue condition?
Simple Factorial Design
Cue Condition
Distal
Proximal
F
27.3
12.2
17.95
M
15.4
11.8
13.60
19.55
12.00
Sex
Cell means
Marginal means
Simple Factorial Design
Mean Escape Latency
Results:
Main effects
Sex – not sig
Cue – sig
Interaction
Sex x Cue - sig
30
25
20
Male
15
Female
10
5
0
1
Distal
2
Proximal
Cue condition
Hypothesis: Females will perform better than males when navigating
in the presence of distal cues but not when navigating in the
presence of proximal cues.
Simple Factorial Design
Mean Escape Latency
Results:
Main effects
Sex – not sig
Cue – not sig
Interaction
Sex x Cue - sig
30
25
20
Male
15
Female
10
5
0
1
Distal
2
Proximal
Cue condition
Outcome: Females will perform better than males when navigating
in the presence of distal cues but worse than males when navigating
in the presence of proximal cues.
Simple Factorial Design
Mean Escape Latency
Results:
Main effects
Sex – not sig
Cue – sig
Interaction
Sex x Cue – not
sig
30
25
20
Male
15
Female
10
5
0
1
Distal
2
Proximal
Cue condition
Outcome: Females and males do not differ, there is no interaction
but a significant effect of cue condition.
Simple Factorial Design
Mean Escape Latency
Results:
Main effects
Sex – sig
Cue – not sig
Interaction
Sex x Cue – not
sig
30
25
20
Male
15
Female
10
5
0
1
Distal
2
Proximal
Cue condition
Outcome: Females perform significantly better than males but there
is no significant effect of cue condition and no interaction.
Simple Factorial Design
Mean Escape Latency
Results:
Main effects
Sex – not sig
Cue – not sig
Interaction
Sex x Cue – not
sig
25
20
15
Male
Female
10
5
0
1
Distal
Outcome: no sig main effects or interaction
2
Proximal
Cue condition
Simple Factorial Design
Results:
Main effects
Sex – sig
Cue – sig
Interaction
Sex x Cue – sig
Mean Escape Latency
30
25
20
Male
15
Female
10
5
0
1
Distal
2
Proximal
Cue condition
Outcome: All effects are significant.
Three Factor Experiment
Sex x Cue x Temp (2 x 2 x 2)
Warm water (28°C)
Cue condition
Cold water (18°C)
Cue condition
Proximal
Distal
F
Female
Distal
Female
Proximal
Female
Distal
Female
Proximal
M
Male
Distal
Male
Proximal
Male
Distal
Male
Proximal
Sex
Distal
Proximal
Three Factor Experiment
Sex x Cue x Temp (2 x 2 x 2)

How many results?


Main effects
Interactions


Main effects



Sex
Cue condition
Water temp
Interactions




Sex x Cue
Sex x Temp
Cue x Temp
Sex x Cue x Temp
A three-way interaction
Look at two-way interactions at different levels of the third factor
Warm water (28°C)
Sex x Cue interaction
Cold water (18°C)
No interaction
25
25
20
Male
15
Female
10
5
0
Mean Escape Latency
Mean Escape Latency
30
20
15
Male
Female
10
5
0
1
Distal
2
Proximal
Cue condition
1
Distal
2
Proximal
Cue condition
The 4-Factor Facial Expression
(try to avoid at all costs)