Chapter 7
Control Techniques in
Experimental Design
Control: techniques and design
features that serve to eliminate the
influences of confounding
extraneous variables.
ch8(1)
1
Sampling
In an ideal world we would randomly select
our subjects from the larger population we are
interested in studying (ensures External
Validity) and then we would randomly assign
participants to the control and experimental
conditions (ensures Internal Validity). Rarely
when studying people is this possible.
ch8(1)
2
More emphasis on Internal
validity than External Validity.
• In psychology we are generally more
concerned with whether the IV is truly
causing a change in the DV than with the
generalizability of the effect.
• External Validity is often established by
replicating the study with other populations.
ch8(1)
3
Controlling for Confounding
Extraneous Variables
Ensures that the DV is the only
variable which has a differential
influences on conditions being
compared in the study.
ch8(1)
4
Randomization
(Random Assignment)
Each participant in the study has an equal and
unbiased chance of being assigned to any of
the conditions being compared in the
experiment.
Equates Groups by controlling for both known
and unknown extraneous variables.
ch8(1)
5
Between Subjects Designs
Each participant serves in only one condition in
the study. Comparisons are made between
conditions therefore the biggest potential
confound is that there may be systematic
differences between the individuals being
compared other that those created by
manipulating the IV.
Confound: Individual Differences
ch8(1)
6
Best Solution!
Random assignment of individuals to
conditions.
Random processes only work when you have a
large enough sample size to work with.
In General if Number if participants per
condition is greater than 15 Random processes
will work.
ch8(1)
7
Matching Techniques
Techniques for equating subjects assigned to
different conditions for specific extraneous
variables.
- can be used for Smaller Sample Sizes.
ch8(1)
8
Other Matching Techniques
Matching by Holding Variable Constant
E.g., I only include subjects with IQ scores of
100. Differences in IQ are eliminated as an
extraneous variable.
Weakness – limits external validity because
we do not know if the finding generalizes to
people of higher and lower intelligences.
ch8(1)
9
Matching by Treating Extraneous Variable as a
Second Factor
Control
Treatment
High IQ
Avg.IQ
Low IQ
Like running 3
separate studies at
the same time, each
with subjects in
different
intelligence groups.
Allows us to control for intelligence and still
determine if the results generalize.
ch8(1)
10
Statistical Control - extraneous variables that
might be confounds are measured during the
study. The effects of these variables are then
controlled for (removed) during statistical
analysis.
More often used with natural
manipulation studies
(correlational studies) to
statistically control for
confounds that cannot be
otherwise controlled.
ch8(1)
11
Matching by Yoked Control
The control group
mirrors the experimental
group on some variable.
REM sleep deprivation study. Subject woken up
when ever they enter REM. How do you produce
a meaningful control group?
Example –sleep study (begin at 8.11)
ch8(1)
12
Matching by Equating Participants
a) Precision Control
Matched on sets of participants on case-by-case
basis. Participants are matched on a
characteristic that the researcher wants to control
difference in. Then the pairs are randomly
assigned each to one of the two conditions.
ch8(1)
13
e.g., If I have a small number
of M&Ms (12) and there are 6
colors, if I use random
processes I could end up with
Two groups that look very different from each
other. Creating pairs that are matched for color
and assigning 1 member of each pair to each
group will insure that the distribution of colors in
each group are the same.
ch8(1)
14
While we know that the two groups are Matched for
color – this does not ensure that they are matched for
other variables. Therefore, we randomly assign each
member of a pair to the groups.
e.g., Twin Studies:
Identical Twins are biologically
matched for DNA. They differ
however on many other factors.
If we were to do a twin study we
would take sets of twins and then
us a random process to determine
which twin is assigned to which
condition.
ch8(1)
15
B) Frequency Distribution Control
Mean, and Standard deviation of the Control and
Experimental groups are matched.
Caution: Groups that have the same means and
St. dev can have very different distributions.
(See Text Figure 7.5).
ch8(1)
16
Within Subjects (Participants) Designs
All participants serve in each condition of the
study.
Advantage: Individual Differences are not a
Confound.
ch8(1)
17
Types of WS Designs
1) Pre-Post Design with no treatment control
group.
Used when the purpose is to show a change
between conditions (i.e. treatment or an
outcome) and the order of the conditions is
logically set (i.e., pre test must proceed post
test).
ch8(1)
18
2) WS design with counterbalancing for
order.
Used when comparing conditions and the
conditions can be measured in any order.
Weakness: Sequencing Effects:
Participation in one condition may effects
participation in a subsequent condition(s).
ch8(1)
19
A) Order Effects
Changes in performance due to practice,
familiarity, boredom that are due to repeated
testing and not to differences in the IV.
ch8(1)
20
B) Carry-over Effects Long-lasting effects
participation in one condition that carry-over
to effect performance in subsequent
conditions.
e.g., Learning, Expectation, attitude, physical
changes due to experimental condition.
E.g., Reward Contrast effects in Rats
ch8(1)
21
Example of Carry Over effects: Reward
Contrast effects in Rats
ch8(1)
22
Controlling for Sequence Effects (WS Design)
1) Counterbalance - controls for order effects
and for unexpected carry-over effects.
Counterbalancing: Within your study, each
condition occurs equally as often in each
ordinal position, and each condition occurs
equally as often before and after each other
condition.
ch8(1)
23
Randomized Counterbalancing
Each Subject gets a different order.
ABC
ACB
BCA
BAC
CAB
CBA
Subjects are randomly assigned to order
conditions.
ch8(1)
24
Complete Randomized
Counterbalancing
All possible orders of conditions are used in
and participants are randomly assigned so that
there are equal numbers in each order
condition.
ch8(1)
25
Equal numbers of subjects are assigned to each order
condition, thus order effects are controlled by
equalling the effects of order for each condition.
Condition A
Condition B
Condition C
1st
2nd
3rd
1st
3rd
2nd
2nd
3rd
1st
2nd
1st
3rd
3rd
2nd
1st
3rd
1st
2nd
ch8(1)
26
Incomplete Counterbalancing
With 4 conditions all orders would total 24.
ABCD
BACD
CABD
DABC
ABDC
BADC
CADB
DACB
ACBD
BCAD
CBAD
DBAC
ACDB
BCDA
CBDA
DBCA
ADAB
BDAC
CDAB
DCAB
ADBA
BDCA
CDBA
DCBA
ch8(1)
27
Rules for Incomplete Counterbalancing
1) Each condition must appear an equal
number of times in each ordinal position.
2) Each condition must precede and be
followed by every other condition equal
numbers of times.
A
B
D
C
B
C
A
D
C
D
B
C
D
A
C
B
ch8(1)
28
Intrasubject counterbalancing
Each subject gets each possible order.
ABCCBA
ch8(1)
29
Between Subjects Designs
– Each participant serves in only one
condition in the study.
Confound: Individual Differences
ch8(1)
30
Control of Individual Differences e.g., IQ,
personality, sex, race, age etc.
Random Assignment - Each member of the
sample has an equal chance of being assigned
to any of the conditions.
ch8(1)
31
Power
Ability to find a difference
between conditions when a
difference actually exists.
When designing a study we
want high power!!
ch8(1)
32
Power and Designs
1) WS designs have higher power than BS
designs and should ALWAYS be
considered first.
ASK
Can I counterbalance for order?
If yes, then counterbalance.
If no, use pre-post test with no-treatment
control group.
ch8(1)
33
BS designs are used only when participants
cannot be in more than one condition of the
study.
• Being in each condition permanently
changes a person so that they cannot be in
other conditions of the study.
ch8(1)
34
Pre
Post
Change
Cognitive Therapy
Drug treatment
Combined Treatment
No treatment Control
Comparison between treatment types here is a
BS comparison
ch8(1)
35
BS designs are commonly used for Natural
Manipulation studies (e.g., sex differences,
personality differences). These are not
experiments and they have low Internal
Validity.
ch8(1)
36