Chapter 11



The general plan for carrying out a study
where the independent variable is changed
Determines the internal validity
Should provide for limited differences
between control and experimental groups

Controlling threats to internal validity
◦ Preexperimental
◦ Quasi-experimental
◦ Experimental

Number of independent variables
◦ Single variable – one manipulated IV
◦ Factorial – two or more IVs & at lease one IV is
manipulated

One group pretest-posttest
◦
◦
◦
◦
◦
Give a pretest that measures the DV
Administer the treatment
Give a posttest that measures the DV
Compare the pretest score to the posttest score
Validity concerns




History
Maturation
Instrumentation
Regression

Static group comparison
◦ Uses 2 or more preexisting intact or static groups
◦ One is exposed to the treatment
◦ The posttest is administered to both groups and
the results compared
◦ Validity concerns




Non randomization – can’t generalize
Selection bias
Maturation
Mortality




Nonrandomized control group (pre-post)
Counterbalanced design
Time-series with one group
Time-series with a control group








Select two or more intact groups
Use similar groups
Randomly assign groups as experimental and control
Give pretest to both groups
Administer the treatment
Give posttest to both groups
Compare the test results and analyze
Validity concerns
◦ Selection bias can be controlled with the pretest and
ANCOVA
◦ Interaction of selection and
 maturation,
 regression or
 instrumentation

Something in the selection process that
causes one group to possess a higher or
lower level of maturity
◦ Asking for volunteers


May occur if each group represented a
different population
ANCOVA with the pretest scores as the
covariate is the best way to analyze



Ceiling effect of tests could cause students
with high scores on the pretest to show little
improvement
students with lower scores on the pretest to
show more improvement
ANCOVA with the pretest scores as the
covariate is the best way to analyze





Uses intact groups
Groups rotate positions
Contains a series of replications
Use when several treatments need to be
studied
Rotation helps remove differences between
groups.
Experimental Treatments
Replication
X1
X2
X3
1
Grp 1
Grp 2
Grp 3
2
Grp 2
Grp 3
Grp 1
3
Grp 3
Grp 1
Grp 2
Col. Mean
Col. Mean
Col. Mean


Periodic measurement on one group
Introduces a treatment at some time period
Looking for changes in the pattern when
treatment is added
Dependent

X
6
4
Example 1
2
Example 2
0
T1
T2
T3
T4
time
T5
T6
Example 3

Validity concerns
◦ History

Strengths-repeated measures help rule out
these threats if they don’t cause fluxuations
◦ Maturation
◦ Testing
◦ regression


Contains a control and experimental group
Helps control history effect
Can have multiple control and experimental
groups
6
Dependent

4
Control
2
Experimental
0
T1
T2
T3
T4
time
T5
T6






Randomized Subjects, Posttest-Only Control
Group
Randomized Matched Subjects, Posttest-Only
Control Group
Randomized Subjects, Pretest-Posttest
Control Group
Solomon Three-Group
Solomon Four-Group
Simple Factoral







Randomly assign participants to groups(30 or
more per group)
Give experimental group the treatment
Measure both on the dependent variable
All other variables are held constant
Uses randomization and a control group to
control the threats to internal validity
Randomization ensures that initial differences
between groups are due to chance
Controls for history, maturation, regression,
pretesting

Use for
◦ Studying changing attitudes
◦ Where pretests are not appropriate or available
◦ Can include more that 2 groups



Possible threats are subject effects and
experimenter effects
Mortality could be a threat because there is
no pretest to know if those that dropout are
different from those that stay
Can’t be used to measure change


Similar to previous design
Uses a matching procedure to create
equivalent groups
◦ Matching variables should correlate to DV
◦ Matched pair must be randomly assigned one to the
E group and one to the C.
◦ Could use a pretest in this model
◦ This procedure is useful for small groups (<30)

Threats to validity same as previous design








Randomly assign participants to the experimental
and control groups
Give a pretest on the DV to both groups
Administer the treatment to the experimental
group
Give a posttest on the DV to both groups
Compare the pre and post test results using a ttest or F-test
ANCOVA is preferred statistic
Internal validity threat– test sensitizing
Main concern is with external validity involving
interaction between the pretest and treatment.




Use 3 groups
Random assignment
Same as randomized pretest-posttest, control
group
Has second control group
◦ Not pretested
◦ Exposed to treatment


Compares all three posttest scores
Both control groups should be similar unless
there is test sensitizing










Use 4 groups
Random assignment
Same as randomized pretest-posttest, control group
Has third control group
◦ Not pretested
◦ Not Exposed to treatment
Compares all four posttest scores
Two groups take the pretest and two do not
Treatment is given to one pretest group and one
none pretested group
Both control groups should be similar unless there is
test sensitizing
Drawback is difficult to conduct and time consuming
Compare the posttests using ANOVA