Experimental Research
Designs, Part 2
Research Process and Design
Spring 2006
Class 7
Today’s objectives




To answer any questions you have
To further explore quasi-experimental and
single-subject designs
To talk about writing the methods section
To apply knowledge to your group project
Research Process and Design (Umbach)
2
Characteristics of Experimental
Research






Statistical equivalence of subject in different
groups (usually by random assignment)
Comparison of 2 or more groups
Manipulation of at least 1 IV
Measurement of DVs
Inferential statistics
Maximum control of extraneous variables
Research Process and Design (Umbach)
3
Experimental Research—Strengths and
Limitations

Strengths




The only designs capable of examining causal relationships
High degree of control of extraneous variables
Direct manipulation of an independent variable
Weaknesses



High levels of control needed for assuring internal validity
mean very restrictive manipulated conditions which in turn
imply low external validity
High levels of external validity mean little control and thus
low internal validity
Not possible in many situations
Research Process and Design (Umbach)
4
Quasi-Experimental Designs

Good designs that are frequently used in
education


Lack of control related to randomly assigning
subjects to groups (i.e., the use of pre-existing
groups)
“Non-equivalent groups” terminology reflects the
lack of random assignment and the concern about
the selection internal validity threat
Research Process and Design (Umbach)
5
Quasi-Experimental Designs


Basic designs and several variations of them
Non-equivalent groups designs




Non-equivalent groups pretest-posttest experimentalcontrol
Non-equivalent groups pretest-posttest experimentalcomparison
Non-equivalent groups pretest-posttest experimentalcontrol-comparison
Statistical analyses focus on the comparison of groups and
the use of appropriate analyses like ANCOVA
Research Process and Design (Umbach)
6
Quasi-Experimental Designs

Time series designs: groups repeatedly measured
before and after treatments




Single group interrupted time series (e.g., A OOOOOO X
OOOOOOO)
Experimental-control group interrupted time series
Variations of these designs include the use of comparison
groups and different treatment times
Statistical analyses examine the patterns of performance
over time
Research Process and Design (Umbach)
7
Factorial Designs - Definitions




Main effect – the effect of each independent
variable on the dependent variable
Interaction effect – the effect of the interaction of
two or more independent variables on the
dependent variable
Factors – independent variables (e.g., two-factor
design means two independent variables being
studied)
Levels – independent variables vary, often
represented in groups (e.g., high, medium, low)
Research Process and Design (Umbach)
8
How many variables? How many IVs?
How many groups?




2x2
3x2
3x2x2
2x2x2x2
Research Process and Design (Umbach)
9
Single-Subject Designs

The use of one subject studied over time



Typically one subject, but there are situations where two or
three subjects are sampled
Focus on populations where there is a low incidence of
specific characteristics (e.g., severe and profound special
education students, physically-challenged students, etc.)
Need to draw causal inferences
Research Process and Design (Umbach)
10
Characteristics of Single-Subject
Designs





Reliable measurement
Repeated measurement
Explicit descriptions of conditions
Baseline and treatment condition: duration
and stability
Manipulation of a single variable at one time
Research Process and Design (Umbach)
11
Single-Subject Designs

Three types of designs



A-B designs: baseline and treatment
A-B-A or A-B-A-B designs: baseline, treatment,
baseline (i.e., reversal or withdraw designs)
Multiple baseline designs
Research Process and Design (Umbach)
12
Single-Subject Designs—Multiple
Baseline




Use of the A-B logic with the addition of two or more actions,
subjects, or situations
Used when it is impossible or undesirable to remove a treatment
condition
Types
 Multiple baselines across behaviors (i.e., the use of a single
treatment across several distinct behaviors)
 Multiple baselines across situations (i.e., observation of a single
target behavior observed in two or more settings)
 Multiple baselines across individuals (i.e., two or more subjects
receiving treatment where behavior and the situation are held
constant)
Statistical analyses examine the patterns of behavior
Research Process and Design (Umbach)
13
Group Activity



Design a study using 2 of the research
designs described in today’s or last
week’s class.
What are the implications of each design?
What are the pros and cons of each
design?
Research Process and Design (Umbach)
14
The method section of a quantitative
study

Vary somewhat but generally have the
following components





Participants
Variables
Instrumentation/materials
Procedures
Data analysis
Research Process and Design (Umbach)
15
More specifically, methods section
should:








Identify research design
Describe sample, population, and participants
Describe instruments, variables, and materials
Review procedures in detail
Explain plans for preliminary studies or pilot
studies (if necessary)
Provide details about data analysis procedures
Identify ethical issues and your approach to
dealing with them
Describe limitations or threats to validity
Research Process and Design (Umbach)
16
Remember, you need to give enough
information so that the study can be
replicated
Research Process and Design (Umbach)
17
Group project





Begin thinking about the elements of your
method section
What design will you employ?
Who do you plan to study?
What and how will you collect data?
What are your independent and dependent
variables?
Research Process and Design (Umbach)
18
For next week…


Non-experimental research design:
Correlational and ex post facto studies
Readings:



Jaeger – Chapter 4 (p. 61-76)
* Wiersma, W. & Jurs, S. G. Chapter 7
Distribute take-home exam
Reminder: Outline of literature review due
Research Process and Design (Umbach)
19