HOMEWORK NINE: PART E: EXPERIMENTAL DESIGN
Topics 37-42: Experimental Design
Isabel Cabrera
EDCI-6300.61 Foundations of Research in Education
Dr. Alberto Jose Herrera
The University of Texas at Brownsville
March 24, 2012
1
HOMEWORK NINE: PART E: EXPERIMENTAL DESIGN
2
Exercise on Topic 37: True Experimental Designs
1. What is the purpose of an experiment?
The purpose of an experiment is to explore the cause-and-effect relationships.
2. How can researchers ensure that there are no biases in the assignment of
participants to groups in an experiment?
Researchers are assured that there are no biases in the assignment by assigning
participants at random to groups.
3. In a diagram for an experimental design, the symbol “O” stands for what?
The “O” stands for observation of measurement.
4. In a diagram for an experimental design, the symbol “X” stands for what?
The “X” stands for experimental treatment.
5. What is the name of the potential problem caused by the pretest in the
pretest-posttest randomized control group design?
This problem is called the pretest sensitization or the reactive effect in testing because
participants will find out what material will be covered and to what extent.
6. What is the name of the true experimental design that has no pretests?
The experimental design that has no pretests is called the posttest-only randomized
control group design.
7. What is the potential drawback to the Solomon randomized four-group
design?
The only drawback is that a researcher must begin with a reasonably large pool of
participants so that when they are divided into four groups, each of the groups will have a
sufficient number to yield reliable results.
8. True experimental designs are easy to spot because they are all characterized
by what?
True experimental designs are easy to spot because they are all characterized by random
assignment to treatments.
Exercise on Topic 38: Threats to Internal Validity
HOMEWORK NINE: PART E: EXPERIMENTAL DESIGN
3
1. What is the name of the threat that indicates that taking a pretest may affect
performance on a posttest?
Testing is an internal threat that indicates that taking a pretest many affect the
performance on a posttest.
2. Suppose an experimental group is being taught letters of the alphabet as a
treatment. At about the same time, the students are watching an educational
program on television, from which they learn the names of the letters. What
is the name of the threat that this problem illustrates?
History is a threat when other environmental influences have on the participants between
the pretest and the posttest.
3. If observers are more tired and less astute when making posttest
observations than when making pretest observations, what threat is
operating?
Instrumentation is another threat when observers are more tired and less astute when
making posttest observations than when making pretest observations.
4. What is the name of the threat posed by nonrandom assignment of
participants to experimental and control groups?
The researcher is using intact groups when groups are not assigned at random.
5. If infants naturally improve in visual acuity and thus perform better at the
end of an experiment than at the beginning, what threat is operating?
Maturation is the threat because the participants are maturing during the period between
the pretest and the posttest.
6. Under what circumstance will statistical regression operate?
Statistical regression is another threat that occurs only if participants are selected on the
basis of their extreme low scores.
7. How can researchers overcome all the threats to internal validity?
All threats to internal validity can be overcome by using a true experimental design, in
which participants are assigned at random to experimental and control conditions.
Exercise on Topic 39: Threats to External Validity
HOMEWORK NINE: PART E: EXPERIMENTAL DESIGN
4
1. Which type of validity deals with the question of whether a researcher can
generalize with confidence to a larger population in a natural setting?
External validity is when a researcher can generalize results with confidence to a larger
population in a natural setting.
2. Which type of validity deals with whether the treatment is directly
responsible for any changes observed in the experimental setting?
Internal validity is concerned when treatment is directly responsible for any changes
observed.
3. What is the name of the threat that warns researchers to be careful in
generalizing the results to a population when an experiment is conducted on
a nonrandom sample?
This threat is called the selection bias because when participants are not chosen at
random the experiment is bias and results cannot be generalized and applied to all
populations.
4.
Suppose a random sample of workers in a factory is exposed to five different
reward systems, with each system being used for one month. What is the
name of the threat that reminds researchers that the results may not
generalize to the population of workers if the population is to be exposed to
only the last reward system tried in the experiment?
Multiple-treatment interference is a threat that occurs when a group of participants is
given more than one treatment.
5. Suppose an experimental classroom has research observers present at all
times. What is the name of the threat that reminds researchers that the
results may not generalize to other classrooms without observers?
The reactive effects of experimental arrangements is when an experimental setting is
different from the natural setting in which the population usually operates, the effects that
are observed in the experimental setting may not generalize to the natural setting.
6. If a pretest causes a change in participants’ sensitivity to a treatment, what
threat is operating?
The reactive effect of testing also known as pretest sensitization is a threat that refers to
the possibility that the pretest might influence how the participants respond to the
experimental treatment.
Exercise on Topic 40: Pre-experimental Designs
HOMEWORK NINE: PART E: EXPERIMENTAL DESIGN
5
1. Are pre-experimental designs valuable for identifying cause-and-effect
relationships?
Pre-experimental designs are of very limited value for investigation cause-and-effect
relationships because of their poor internal validity.
2. Suppose a researcher administered a new program to all students in a school.
At the end of the school year, the researcher administered a standardized test
to the students in the school as well as to students in another school who were
serving as a control group. Is the comparison of the average scores for the
two groups of students useful for determining the effects of the program?
No the comparison would not be useful for determining the effects of the program. The
researcher used a static-group comparison design where he/she used two groups of
participants that were not assigned at random. The groups would be called intact groups
because they were already pre-existing groups. The internal threat would be selection.
Furthermore, without a pretest, the researcher does not have a basis for knowing whether
the two groups were similar at the beginning of the experiment.
3. What is the name of the pre-experimental design used in Question2?
The pre-experimental design used in the scenario would be the static-group comparison
design.
4. If a researcher gives a pretest on knowledge of child abuse to a group of
social workers, then gives them a series of seminars on child abuse followed y
a posttest, what is the name of the pre-experimental design the researcher
used?
This scenario is using the one-group pretest-posttest design. There is nothing wrong with
this design if it is for instructional purposes only.
5. Is the design used in Question 4 useful for determining cause-and-effect
relationships?
It is of no value of cause-and-effect relationships because it fails to make comparisons.
Exercise on Topic 41: Quasi-Experimental Designs
1. What is the name of the design diagrammed immediately below?
OXO
HOMEWORK NINE: PART E: EXPERIMENTAL DESIGN
O
6
O
It is called the quasi-experimental design because it has two intact groups not assigned at
random.
2. In the design shown in Question 1, what indicates that the participants were
not random to the groups?
Not assigned by random as indicated by the dashed line.
3. If a researcher uses matching to form the two groups in the design in
Question1, would the resulting experiment be superior to a true experiment?
It is far from ideal but is superior to the pre-experimental designs, which are not directly
interpretable in terms of causal effects.
4. What is a major advantage of the equivalent time-samples design?
The major advantage is that the participants are both the control and experimental group
and are identical in all ways at the beginning of the experiment.
5. What is a major disadvantage of the equivalent time-samples design?
A major disadvantage of this design is the strong possibility of multiple-treatment
interference.
6. In Psychology, what is an ABAB design?
In Psychology, a single-group design in which treatments are alternated is often called the
ABAB design.
Exercise on Topic 42: Confounding in Experiments
1. A confound is a source of confusion regarding what?
A confound is a source of confusion regarding the explanation for a given difference.
2. What is the name of the effect that refers to the possibility that the control
group might become aware of its “inferior” status and respond by trying to
outperform the experimental group?
HOMEWORK NINE: PART E: EXPERIMENTAL DESIGN
7
The John Henry effect refers to the possibility that the control group might become aware
of its “inferior” status and respond by trying to outperform the experimental group.
3. What is the formal name of what is characterized as the “attention effect” in
this topic?
The Hawthorn effect is also known as the attention effect because there were two
intertwined explanations for the experimental results on the industrial psychology study,
one being the amount of lighting and the other being the special attention participants
received during the study.
4. In addition to an experimental and a traditional control group (with no
treatment), what other group can be used to control for the Hawthorne
effect?
To control the Hawthorn effect, some researchers use three groups: 1) an experimental
group 2) a control group that receives attention 3) a control group that receives no
special attention.
5. The term “placebo effect” refers to what tendency?
The placebo effect refers to the tendency of individuals whom feel they are improving
simply because they know they are being treated.
6. In what type of experiment do neither the participants nor the individual
dispensing the drug know which is the active drug and which is the placebo?
In a double blind experiment, neither the participants nor the individual dispensing the
drug know which is the active drug and which is the placebo.