Educational Research
Chapter 10
Experimental Research
Gay, Mills, and Airasian
10th Edition
Topics Discussed in this Chapter




Defining characteristics of experimental
research
Conducting experimental research
Manipulation and control
Threats to validity



Internal validity
External validity
Group designs
Defining Characteristics

Research designed to investigate cause and effect
relationships through the direct manipulation of an
independent variable and control of extraneous
variables



Independent variable – the variable being manipulated
Dependent variable – the variable in which the effect of the
manipulation of the independent variable is observed
Researcher manipulation and control – choice of treatments,
choice of a research design, use of specific procedures, etc.
Experimental Process

Six steps to conducting experimental research

Selection and definition of the problem


Statement of a hypothesis indicating a causal
relationship between variables
Selection of participants and instruments



Random selection of a sample of subjects from a larger
population
Random assignment of members of the sample to each
group
Selection of valid and reliable instruments
Experimental Process

Six steps to conducting experimental research
(cont.)

Selection of a research plan

Three types of comparisons




Comparison of two different approaches
Comparison of new and existing approaches
Comparison of different amounts of a single approach
Execution of the research plan

Two concerns


Sufficient exposure to the treatment
They need to be substantively different treatments
Experimental Process

Six steps to conducting experimental
research (cont.)


Analysis of data
Formulation of conclusions
Manipulation and Control

Manipulation


The researcher’s decisions related to what
constitutes the independent variable
Active and assigned variables

Active variables are those the researcher actively
manipulates



Choice of an instructional strategy
A particular counseling approach
Assigned variables are those that cannot be manipulated
by the researcher but are of interest


Gender
Race
Manipulation and Control

Control


The researcher’s efforts to remove the influence of any
extraneous variables that might have an effect on the
dependent variable
The goal is to be assured the only differences between
groups is that related to the independent variable

Participant variables – characteristics of the subjects



Pre-existing achievement levels
Differences in attitudes
Environmental variables – characteristics of the context


Learning materials
Differences in the time available for treatment between groups
Experimental Validity


Internal validity – the degree to which the
results are attributable to the independent
variable and not some other rival explanation
External validity – the extent to which the
results of a study can be generalized


Population validity – generalizations related to
other groups of people
Ecological validity – generalizations related to
other settings, times, contexts, etc.
Experimental Validity

Relative importance of internal and external
validity

Internal and external validity are related
reciprocally




Controlling internal validity decreases external validity
Controlling external validity decreases internal validity
First demonstrate an effect in a highly controlled
environment (i.e., prioritize internal validity)
Second replicate the study in a more realistic,
natural setting (i.e., prioritize external validity)
Threats to Internal Validity








History:an event occurs not related to IV
Maturation: Ss change over time
Testing: Exposure to pretest might improve scores on
posttest
Instrumentation: Reliability, Validity, and not using
the same test
Statistical regression: Regression to the mean
Differential selection of participants: Groups might
be different outside of IV
Mortality: Ss drop out of the study
Selection-maturation interaction, etc.: Groups grow
at different rates not due to the IV
Threats to External Validity




Pre-test treatment interaction: Taking the pretest impacts the
treatment itself.
Multiple treatment interference: More than one treatment/
experiment performed (old impacts new)
Selection treatment interaction: Who is in your sample impacts
the results.
Specificity of variables: Not specific enough in the following
areas to replicate the study or know if generalizable






Participants
Operational definition of the treatment
Operational definition of the dependent variable
Specific times
Specific circumstances
Treatment diffusion: Two groups talk to one another and share
treatment information so that they are not in effect one group.
Threats to External Validity


Experimenter effects: Something about the
experimenter changes the outcome of the DV.
Reactive arrangements: Something about the Ss
changes the outcome of the DV. Some examples of
this are: reaction to the environment, reaction to the
attention from the researcher, placebo effect, and
novelty effect.
Controlling for Extraneous Variables

Extraneous variables must be controlled
to be able to attribute the effect to the
treatment


Group equivalency must be assured
Four major means to achieve control

Randomization


Selection – controls for representation
Assignment – controls for group equivalency
Controlling for Extraneous Variables

Matching




Identifying pairs of subjects “matched” on specific
characteristics of interest
Randomly assigning subjects from each pair to different
groups
Difficulty with subjects for whom no match exists
Comparing homogeneous groups


Restricting subjects to those with similar characteristics
Restricting subjects results in problems related to
generalization
Controlling for Extraneous Variables

Using subjects as their own controls



Multiple treatments across time
Problem with carry-over effects
Analysis of covariance (ANCOVA)


Statistically adjusting the posttest scores for
the subjects in each group for pretest
differences that existed at the beginning of the
study
Creates statistically equivalent groups
Controlling for Extraneous Variables

Other ways to control extraneous
variables

Holding variables constant




Using only males rather than males and
females
Selecting teachers with only similar levels of
experience
Selecting only one grade level
Stipulating the specific length of a treatment
Group Designs

Two major classes of group designs



Single-variable designs – one independent variable
Factorial designs – two or more independent
variables
Three types of experimental designs



Pre-experimental designs
Experimental designs
Quasi-experimental designs
Pre-Experimental Designs

Three types (X or X1=treatment, 0=test, X2=
control)

One-shot case study


One-group pretest-posttest design


O X O
Static group comparison


X O
X1 O
X2 O
Threats to internal validity – see Table 10.1
True Experimental Designs

Three types (r=random assignment,
x=treatment, 0=test)

Pretest-posttest control group design


R O X O
R O
O
Posttest only control group design

R X O
R
O
True Experimental Designs

Three types (cont.)

Solomon four-group comparison


R O X O
R O
O
R
X O
R
O
Threats to internal validity – see Figure
10.1
Quasi-Experimental Designs


Groups may be randomly assigned, not Ss
Three types

Non-equivalent control group design


Time series design


O O O O X O O O O
Counterbalanced design


O X O
O
O
O X1 O X2 O X3 O
O X3 O X1 O X2 O
O X2 O X3 O X1 O
Threats to internal validity – see Figure 10.2
Factorial Designs

Two independent variables and one
dependent variable



The effect of teaching strategy and gender on
students’ achievement
The effect of a particular counseling technique and
the clients’ ethnicity on the success of the
treatment
The effect of a specific coaching approach and
children in three age groups on the ability to
perform certain physical tasks
Factorial Designs

Interaction



The degree to which changes in the dependent
variable are different depending on the levels of
each of the independent variables
A particular instructional strategy is more effective
for males than females
A particular counseling technique is more effective
when the ethnicity of the counselor and client are
similar
Factorial Designs

Interaction

Visually explained by a graph of
performance of all levels of both
independent variables



Parallel lines indicate no interaction
Non-parallel lines indicate an interaction
See Figure 10.5 in your text
Factorial Designs
Interaction

Visual presentation of a significant
interaction
14
12
10
8
Test Scores

TEST
6
Small Testing Room
Large Testing Room
4
Small Lecture Room
LECTURE
Large Lecture Room
Factorial Designs
Interaction

Visual presentation of a non-significant
interaction
18
16
14
12
Test Scores

TEST
10
Small Testing Room
Large Testing Room
8
Small Lecture Room
LECTURE
Large Lecture Room