CONDUCTING EDUCATIONAL RESEARCH
Guide to Completing a Major Project
Daniel J. Boudah
Chapter 5
Designing and Conducting Experimental Research
Lucy B. Houston, Presenter
Chapter Objectives: reader will understand . . .
Characteristics of
experimental and
quasi-experimental
research design.
The link between
research questions and
research design.
Commonly used
experimental and
quasi-experimental
design.
Common ways to
measure dependent
variables and types of
data.
How to align research
questions with design,
measurement, and
data.
How to be prepared to
implement
experimental research
interventions.
The characteristics of
single-subject designs.
Key Points
•
•
•
•
Validity and outcome of the study hinge on selecting the appropriate design.
Experimental designs are the “blueprint” for the study.
The fit of a design must be linked to the research questions and hypothesis.
Know the purpose and characteristics of the common designs used in
experimental research.
Review: Distinguish
Descriptive from Experimental Research
•
•
•
Descriptive
Describes a condition or
phenomenon - Descriptive
Identifies a potential cause/effect
relationship – Casual Comparative
Explores relationship among
variables, i.e. strength of
relationship - Correlational
•
•
•
•
Experimental
Makes inferential connections
between variables
Attempts to “establish cause and
effect relationships” between
variables
Intervenes to determine if X (the
independent) variable will effect Y
(dependent variable).
Participants are a part of an
experimental group (with or
without a control group) and single
subject.
Types of Experimental Research
•
•
•
Experimental
Attempts to determine if a one
variable has an effect on the other.
Independent variable, the
intervention and dependent
variable, the outcome.
Random assignment to sample,
higher level of control of extraneous
variables, and generalization.
•
•
•
Quasi-experimental
Attempts to determine if a one
variable has an effect on the other.
Independent variable, the
intervention and dependent
variable, the outcome.
Do not randomize sample,
limitation to generalizing outcome.
Characteristics of Experimental Designs
Figure 5.1: page 95
Selection of experimental participants
• Direct manipulation of an independent variable
• Control of extraneous variables
• Measurement of outcomes
Also quantitative research
•
Steps in Experimental Research Design
Figure 5.2, page 96
Define what variables you are interested in and how you might address issues of
validity.
Select a design that will enable you to investigate the variables.
Decide how you want to measure the impact of independent variable on the
dependent variable.
Think about the kind of data will want to collect through your measurement.
Decide how you want to analyze the effects of the intervention
Step 1: Define what variables you are
interested in and how you might address
issues of validity.
Questions determine the Design
Questions are the basis for determining the
research design because. . .
•
•
•
•
stem from interest in a problem or phenomenon, i.e. the
outcome or dependent variable
includes a statement of the intervention, i.e. the
independent variable.
names the target population or participants
provides an idea of the methodology or measurement
Interactive
Question: Do teachers participant more in profession development when they
receive a stipend?
Possible hypothesis: There is a significant difference in participation in staff
development when teachers receive a stipend.
What is the problem?
What is the dependent variable?
What is the intervention or treatment?
What is the independent variable?
Choosing an intervention
Brainstorm and weigh the pros and cons
Focus on a specific outcome
Consider the characteristics, planning, conditions and activities
Identify the who, what, where
Materials and resources needed
Draft a timeline
Revisiting validity
Construct
Internal
External
Measurement tool
measures what it claims to
measure.
Accuracy of conclusion that
an intervention made a
significant difference
Extent to which results can
be generalized beyond
investigation.
Examples:
Most often questioned
when differences can be
explained by demographics
Adequate sample of
population
Memory, intelligence,
reading fluency
Feasibility of replication
Step 2: Select a design that will enable
you to investigate the variables.
Group Designs and Single-Subject Designs
Group Design
Posttest Only Design: group → intervention → Posttest
Pretest-Posttest Design/Paired Sample: Group → Pretest → Intervention → Posttest / Threats to external validity due to
lack of a comparison group and using same test for pre and post test
Comparison Group Design/Control group or static group:
Experimental Group→ Pretest→ Intervention→ Posttest
Comparison Group→ Pretest →no intervention → Posttest/Threats to validity is comparability of two groups
(Homegenity of variance) and only difference in experiences of participants is the intervention (confounding variables).
Group Designs
Factorial
Experimental Group → Pretest → Intervention → Posttest
(1, 2, 3, 4): Enables research to study multiple independent variables that may cause observed problem or effect.
Example: study of the effect of four variables with two conditions (control and experimental), 4 x 2 factorial design.
Time-Series
Group → M → M → M → I → M → M → M: Multiple assessments prior to intervention to determine a “valid baseline”.
Threats to valid is the treatment of one group.
Interactive
Mrs. McGrady attended a workshop on vocabulary development. She decided
to test the effectiveness of the multiple word meaning strategy by using it as an
intervention for her Bubble kids during guided reading. In order to determine if
the new vocabulary strategy worked she administered a pretest before the
treatment and a posttest after six weeks.
True or False
Mrs. McGrady used a quasi-experiemental design.
Single-Subject Designs
Characteristics
• Study of an individual
• Repeated measurement of
dependent variable before and
after the intervention
• Baseline and intervention
results charted or graphed
rather than statistical analysis.
Variations
• A-B-A design –
Baseline/Intervention/Baseline
• Multiple-baseline design –
intervention begins at different
points in time for two or more
participants
Step 3: Decide how you want to
measure the impact of independent
variable on the dependent variable.
Measurement tools must be valid, reliable and
meaningful
Selecting a measurement tool
page 106
•
•
•
•
•
Align tightly to the intervention.
Use validated measurement tools along with those you develop.
A priori calculation of coefficient alpha for new measures and observations.
Balance of global and specific measures.
Learn about construct validity as you conduct intervention research
Measurement Tools
Observations
• Formal – identify and record targeted behavior using event-based
observations (frequency of behavior) or time-based observations (duration of
behavior), example: using tally marks to determine exact number of times
behavior occurred with a set time frame or counting time intervals. Excellent
records for conferences with parents or principles about student behavior.
• Informal – anecdotal records and field notes, less structured
• Test and Permanent Records – State performance test, attendance, office
referrals, and grade point average
Measurement Tools continued
Criterion-Referenced Assessments
• Predetermined level of mastery set by school district, state education agency or teacher.
• % correct indicates mastery, e.g. 90% compared to norm referenced, i.e. same age/same
grade level average.
Continuous Assessment
• curriculum-based measurement (CBM)
• Commercial products developed using state and district curricula e.g. TPRI, DRIBELS, DRA
• Teacher administered/ Personal digital assistants (PSAs)
Interviews
ask questions to determine participants’ personal perceptions.
To determine student’s cognition or strategies used. “Think Aloud” technique
In combination with other measures, strengthens validity.
Clearly state the type of measurement used to
measure your dependent variable in you research
proposal or report.
Text to World Connection
Step 4: Think about the kind of data
will want to collect through your
measurement.
Step 5: Decide how you want to
analyze the effects of the
intervention