Introduction to Educational Research (5th ed.)
Craig A. Mertler & C.M. Charles
Chapter 14
Experimental, Quasi-Experimental, &
Causal-Comparative Research
1
The (Never-Ending) Search for
Causation
Establishing causation among variables :
Produces increased understanding of those variables
Results in the ability to manipulate conditions in order
to produce desired changes
2
Experimental Research
Can demonstrate cause-and-effect very convincingly
Very stringent research design requirements
Experimental design requires:
Randomly selected participants who are randomly
assigned to groups (experimental and control)
Independent treatment variable that can be applied to
the experimental group
Dependent variable that can be measured in all groups
3
Quasi-Experimental Research
Used in place of experimental research when random
assignment to groups is not feasible
Otherwise, very similar to true experimental research
4
Causal-Comparative Research
Explores the possibility of cause-and-effect relationships
when experimental and quasi-experimental approaches
are not feasible
Used when manipulation of the independent variable is
not ethical or is not possible
5
Fundamentals of Experimental and
Quasi-Experimental Research
Cause and effect:
Incorporates a temporal element—the cause is a
condition that exists prior to the effect; effect is a
condition that occurs after the cause
There exists a “logical connection”—cause-and-effect
is demonstrated when manipulation of the
independent variable results in differences in the
dependent variable (as evidenced by comparing the
experimental group to the control group)
6
Fundamentals of Experimental and
Quasi-Experimental Research (cont’d.)
Random selection and random assignment :
Distinguish between “selection” and “assignment”
Both help to ensure that groups are equivalent and to
control for extraneous variables
If you incorporate random selection and random
assignment
Experimental research
» If you incorporate random selection but not random
assignment
Quasi-experimental
research
7
Fundamentals of Experimental and
Quasi-Experimental Research (cont’d.)
When to use experimental research design :
If you strongly suspect a cause-and-effect relationship
exists between two conditions, and
At least randomly assigned group is available to
participate, and
The independent variable can be introduced to
participants and can be manipulated, and
The resulting dependent variable can be measured for
all participants
8
Fundamentals of Experimental and
Quasi-Experimental Research (cont’d.)
Commonly used experimental design notation :
X1 =
treatment group
X2 =
control/comparison group
O=
observation (pretest, posttest, etc.)
R =
random assignment
9
Common Experimental Designs
• Single-group pretest-treatment-posttest design:
O
X
O
» Technically, a pre-experimental design (only one
group; therefore, no random assignment exists)
» Overall, a weak design—no control group (for
comparison purposes); pretest sensitization;
experimenter effects (therefore, cannot be sure
about conclusions)
10
Common Experimental Designs
(cont’d.)
Two-group treatment-posttest-only design:
R
R
X1
X2
O
O
» Here, we have random assignment to experimental,
control groups
» A better design, but still weak—cannot be sure that
groups were equivalent to begin with
11
Common Experimental Designs
(cont’d.)
Two-group pretest-treatment-posttest design:
R
O
X1
O
R
O
X2
O
» A substantially improved design—previously
identified errors have been reduced
12
Common Experimental Designs
(cont’d.)
Solomon four-group design:
R
O
X1
O
R
O
X2
O
R
X1
O
R
X2
O
» A much improved design—how??
» One serious drawback—requires twice as many
participants
13
Common Experimental Designs
(cont’d.)
Factorial designs:
R
O
X1
g1
O
R
O
X2
g1
O
R
O
X1
g2
O
R
O
X2
g2
O
» Incorporates two or more factors
» Enables researcher to detect differential differences
(effects apparent only on certain combinations of
levels of independent variables)
14
Common Experimental Designs
(cont’d.)
Single-participant measurement-treatment-measurement
designs:
O
O
O
|
X
O
X
O
|
O
O
O
» Purpose is to monitor effects on one subject
» Results can be generalized only with great caution
15
Common Quasi-Experimental Designs
Posttest-only design with nonequivalent groups:
X1
O
X2
O
» Uses two groups from same population
» Questions must be addressed regarding
equivalency of groups prior to introduction of
treatment
16
Common Quasi-Experimental Designs
(cont’d.)
Pretest-posttest design with nonequivalent groups:
O
X1
O
O
X2
O
» A stronger design—pretest may be used to establish
group equivalency
17
Common Quasi-Experimental Designs
(cont’d.)
Time series designs:
O
O
O
O
X
O
O
O
O
» Uses a single group of participants
» Examines possible changes over time
18
Similarities Between Experimental and
Quasi-Experimental Research
Cause-and-effect relationship is hypothesized
Participants are randomly selected and randomly assigned
(experimental) or randomly selected and nonrandomly
assigned (quasi-experimental)
Application of an experimental treatment by researcher
Following the treatment, all participants are measured on
the dependent variable
Data are usually quantitative (exception is single-subject
designs); analyzed by looking for significant differences
on the dependent variable
19
Internal and External Validity
“Validity of research” refers to the degree to which the
conclusions are accurate and generalizable
Both experimental and quasi-experimental research are
subject to threats to validity
If threats are not controlled for, they may introduce error
into the study, which will lead to misleading conclusions
20
Threats to Internal Validity
Internal validity—extent to which differences on the
dependent variable are a direct result of the manipulation
of the independent variable
Differential selection of participants—participants are
not selected/assigned randomly
History—when factors other than treatment can exert
influence over the results; problematic over time
Maturation—when changes occur in dependent
variable that may be due to natural developmental
changes; problematic over time
21
Threats to Internal Validity (cont’d.)
Testing—also known as “pretest sensitization”; pretest
may give clues to treatment or posttest and may result
in improved posttest scores
Attrition (mortality)—loss of participants
22
Threats to External Validity
External validity—extent to which the results can be
generalized to other groups or settings
Population validity—degree of similarity among sample
used, population from which it came, and target
population
Personological variables—application of findings based
on individual differences
Ecological validity—physical or emotional situation or
setting that may have been unique to the experiment
23
A Published Example of Experimental
Research
Meyer, M., Linville, M., and Rees, G. (1993). The
development of a positive self-concept in preservice
teachers. Action in Teacher Education, 15(1), 30–35.
See “Additional Examples of Published Experimental
Studies” for 7 additional articles, available through
Research NavigatorTM (http://www.researchnavigator.com)
24
The Nature of Causal-Comparative
Research
Conducted to explore possible cause-and-effect
relationships
Differs from experimental and quasi-experimental
research:
Independent variable is not manipulated
Focuses first on the effect, then tries to determine
possible causes (ex post facto)
Questions will remain about the effect following the
cause, or vice versa
Other conditions must also be considered as “plausible
causes”
25
A Published Example of CausalComparative Research
Kelly, K. (1993). The relation of gender and academic
achievement to career self-efficacy and interests.
Gifted Child Quarterly, 37, 59–64.
See “Additional Examples of Published Causal-Comparative
Studies” for 6 additional articles, available through
Research NavigatorTM (http://www.researchnavigator.com)
26
Applying Technology…
Web sites covering topics related to experimental,
quasi-experimental, and causal-comparative research
Dr. Trochim’s discussion of "Experimental Designs"
(http://trochim.human.cornell.edu/kb/desexper.htm)
Dr. Trochim’s discussion of "Two-Group Experimental Designs"
(http://trochim.human.cornell.edu/kb/expsimp.htm)
Dr. Trochim’s discussion of "Factorial Designs"
(http://trochim.human.cornell.edu/kb/expfact.htm)
Dr. Trochim’s discussion of "Randomized Block Designs"
(http://trochim.human.cornell.edu/kb/expblock.htm)
Dr. Trochim’s discussion of "Covariance Designs"
(http://trochim.human.cornell.edu/kb/expcov.htm)
27
Applying Technology…
More web sites covering topics related to experimental,
quasi-experimental, and causal-comparative research
•
Dr. Trochim’s discussion of "Hybrid Experimental Designs"
(http://trochim.human.cornell.edu/kb/exphybrd.htm)
•
Dr. Trochim’s examination of quasi-experimentation
(http://trochim.human.cornell.edu/kb/quasiexp.htm)
•
Dr. Trochim’s discussion of "The Nonequivalent Control Group Design"
(http://trochim.human.cornell.edu/kb/quasnegd.htm)
•
Dr. Trochim’s discussion of "Other Quasi-Experimental Designs"
(http://trochim.human.cornell.edu/kb/quasioth.htm)
•
Dr. Beckelhimer’s discussion of "Experimental Research"
(http://cmhs.cabe.k12.wv.us/~sbeckelh.staff/research/EXPER.HTML)
•
Dr. Beckelhimer’s discussion of "Quasi-Experimental"
(http://cmhs.cabe.k12.wv.us/~sbeckelh.staff/research/QUASI.HTML)
28