Experimental and Single-Subject Design
PSY440
May 27, 2008
Definitions:
Consider each of the following terms and
generate a definition for each:
• Research
• Empirical
• Data
• Experiment
• Qualitative Research
Definitions
Research
1. Scholarly or scientific investigation or inquiry
2. Close and careful study
(American Heritage Dictionary)
3. Systematic (step-by-step)
4. Purposeful (identify, describe, explain, predict)
Definitions
Empirical:Relying upon or derived from observation
or experiment; capable of proof or verification by
means of observation or experiment. (American Heritage
Dictionary)
Data: Information; esp. information organized for
analysis or used as the basis of a decision. (American
Heritage Dictionary)
Experiment:A method of testing an hypothesized
causal relationship between two variables by
manipulating one variable and observing the
effect of the manipulation on the second
variable.
Overview of Experimental Design
Based on Alan E. Kazdin. (1982).
Single-Case Research Designs:
Methods for Clinical and Applied
Settings. Chapter IV.
Independent & Dependent Variables
The independent variable (IV) is the variable
that is manipulated in an experiment.
The dependent variable (DV) is the variable
that is observed to assess the effect of the
manipulation of the IV.
What are some examples of IV’s and DV’s
that might be studied experimentally?
Internal and External Validity
Internal validity refers to the extent to which a
study is designed in a way that allows a causal
relation to be inferred. Threats to internal validity
raise questions about alternative explanations for
an apparent association between the IV and DV.
External validity refers to the generalizability of the
findings beyond the experimental context (e.g. to
other persons, settings, assessment devices, etc).
Threats to Internal Validity
History
Maturation
Testing
Instrumentation
Statistical Regression
Attrition
Diffusion of Treatment
History
Any event other than the intervention
occurring at the time of the experiment that
could influence the results.
Example in intervention research: Participant
is prescribed medication during the time
frame of the psychosocial treatment
Other examples?
How can this threat be ruled out or reduced by
the experimental design?
Maturation
Any change over time that may result from
processes within the subject (as opposed to
the IV)
Example: Client learns how to read more
effectively, so starts behaving better during
reading instruction.
Other examples?
How can this threat be ruled out or reduced by
the experimental design?
Testing
Any change that may be attributed to effects of
repeated assessment
Example: Client gets tired of filling out weekly
symptom checklist measures, and just starts
circling all 1’s or responding randomly.
Other examples?
How can this threat be ruled out or reduced by
the experimental design?
Instrumentation
Any change that takes place in the measuring
instrument or assessment procedure over
time.
Example: Teacher’s report of number of
disruptive incidents drifts over time, holding
the student to a higher (or lower) standard
than before.
Other examples?
How can this threat be ruled out or reduced by
the experimental design?
Statistical Regression
Any change from one assessment occasion to
another that might be due to a reversion of scores
toward the mean.
Example: Clients are selected to be in a depression
group based on high scores on a screening
measure for depression. When their scores (on
average) go down after the intervention, this could
be due just to statistical regression (more on this
one later in the course : )
How can this threat be ruled out or reduced by the
experimental design?
Selection Biases
Any differences between groups that are due to the
differential selection or assignment of subjects to
groups.
Example: Teachers volunteer to have their classes
get social skills lessons, and their students are
compared to students in classrooms where the
teachers did not volunteer (both teacher and
student effects may be present).
Other examples?
How can this threat be ruled out or reduced by the
experimental design?
Attrition
Any change in overall scores between groups or in a
given group over time that may be attributed to the
loss of some of the participants.
Example: Clients who drop out of treatment are not
included in posttest assessment - may inflate
treatment group posttest score.
Other examples?
How can this threat be ruled out or reduced by the
experimental design?
Diffusion of Treatment
The intervention is inadvertently provided to
part or all of the control group, or at the times
when the treatment should not be in effect.
Example: Teacher starts token economy before
finishing the collection of baseline data.
Other examples?
How can this threat be ruled out or reduced by
the experimental design?
Internal validity and single-subject designs
In single-subject research, the participant is
compared to him/herself under different
conditions (rather than comparing groups).
The participant is his/her own control
Selection biases and attrition are automatically ruled
out by these designs
Well designed single-subject experiments can rule
out (or reduce) history, maturation, testing,
instrumentation, and statistical regression
Threats to External Validity
Generality Across
•
•
•
•
•
Participants
Settings
Response Measures
Times
Behavior Change Agents
Reactive Experimental Arrangements
Reactive Assessment
Pretest Sensitization
Multiple Treatment Interference
Generality Across Subjects, Settings, Responses, & Times
Results do not extend to participants, settings,
behavioral responses, and times other than
those included in the investigation
Example: Couple uses effective
communication skills in session, but not at
home.
Other examples?
How can this threat be ruled out or reduced by
the experimental design?
Generality Across Behavior Change Agents
Intervention results do not extend to other persons
who can administer the intervention (special case
of previous item)
Example: Parents are able to use behavior
modification techniques successfully but child
care providers are not (child responds differently
to different person)
Other Examples?
How can this kind of threat to external validity be
ruled out or reduced?
Reactive Experimental Arrangements
Participants may be influenced by their
awareness that they are participating in an
experiment or special program (demand
characteristics)
Example: Social validity of treatment is
enhanced by the association with a university
Other examples?
How can this threat be ruled out or reduced by
the experimental design?
Reactive Assessment
The extent to which participants are aware that their
behavior is being assessed and that this awareness
may influence how they respond (Special case of
reactive arrangements).
Example: Child complies with adult commands when
the experimenter is observing, but not at other times.
Other examples?
How can this threat be ruled out or reduced by the
experimental design?
Pretest Sensitization
Assessing participants before treatment sensitizes them to
the intervention that follows, so they are affected
differently by the intervention than persons not given the
pretest.
Example: Pretest administered before parenting group
makes participants pay attention to material more closely
and learn more
Other examples?
How can this threat be ruled out or reduced by the
experimental design?
Multiple Treatment Interference
When the same participant(s) are exposed to more
than one treatment, the conclusions reached
about a particular treatment may be restricted.
Example: Clients are getting pastoral counseling at
church and CBT at a mental health center.
Other examples?
How can this threat be ruled out or reduced by the
experimental design?
Evaluating a Research Design
No study is perfect, but some studies are better than
others. One way to evaluate an experimental
design is to ask the question:
How well does the design minimize threats to
internal and external validity? (In other words,
how strong a causal claim does it support, and
how generalizable are the results?)
Internal/External Validity Trade-Off
Many designs that are well controlled (good
internal validity), are more prone to
problems with external validity (generality
across settings, behavioral responses,
interventionists, etc. may be more limited).
Random selection/assignment
Random: Chance of being selected is equal for each
participant and not biased by any systematic factor
Group designs can reduce many of the threats to internal
validity by using random assignment of participants to
conditions.
They can (in theory) also limit some threats to external
validity by using a truly random sample of participants
(but how often do you actually see this?)
Single-Subject Designs
More modest in generalizability claims
Can be very strong (even perfect) in reducing threats
to internal validity
Examples:
Reversal (ABAB)
Multiple Baseline
Changing Criterion
Multiple Treatment
Reversal Design (ABAB)
A=Baseline
B=Treatment
A=Return to Baseline
B=Reintroduce Treatment
Reversal Design
Baseline:
Stable baseline allows stronger causal
inference to be drawn
Stability refers to a lack of slope, and low
variability (show examples on white board)
ABAB Design: Baseline
If trend is in reverse direction from expected
intervention effect, that’s OK
If trend is not too steep and a very strong
effect is expected, that may be OK
For reversal design, relatively low variability
makes it easier to draw strong conclusions
Threats to internal validity?
History & maturation (return to baseline rules these
out)
Testing, instrumentation, and statistical regression
(return to baseline rules these out, but use of
reliable measures, trained & monitored observers
strengthens the design)
Selection bias & attrition (not an issue with singlesubject research!)
Diffusion of treatment (a concern but can be
controlled in some cases)
What about generalizability?
Can’t easily generalize beyond the case.
Need to replicate under different
circumstances and with different
participants, or follow-up with group
design.
Disadvantages to Reversal Design
Diffusion of treatment:
If the intervention “really worked” the
removal of the intervention should not
result in a return to baseline behavior!
Ethical concerns:
If the behavior change is clinically
significant, it may be unethical to remove it,
even temporarily!
Multiple Baseline Design
Collect baseline data on more than one
dependent measure simultaneously
Introduce interventions to target each
dependent variable in succession
Multiple Baseline Design
Each baseline serves as a control for the other
interventions being tested (Each DV is a “mini”
AB experiment of the previous baseline)
May be done with one participant when multiple
behaviors are targeted, or with multiple
participants, each receiving the same intervention
in succession.
Example with one participant
Multiple baseline design across behaviors to
increase assertive communication:
– Increase eye contact
– Increase speech volume
– Increase number of requests made
Example across participants
• More than one client receives the same
intervention, beginning baseline all at one
time, and introducing the intervention in
succession, so that each participant serves
as a control for the others.
• Example from textbook: intervention to
increase appropriate mealtime behavior in
preschoolers
Example across situations, settings, and time
• Measure target behavior in multiple
settings, and introduce the intervention into
each setting in succession, while collecting
baseline data on all settings
Advantages of Multiple Baseline
+ No need to remove intervention
+ Interventions can be added gradually
(practical utility in clinical setting)
Disadvantages of Multiple Baseline
- Interdependence of baselines (change in one
may result in change in another)
- Inconsistent effects of interventions (some
are followed by changes in behavior and
others are not)
- Prolonged baselines (ethical &
methodological concerns)
Changing Criterion Designs
Intervention phase requires increasing levels
of performance at specified times.
If the performance level increases as expected
over several intervals, this provides good
evidence for a causal relationship between
intervention and outcome.
Multiple Treatment Designs
More than one treatment is implemented
during the same phase, in a manner that
allows the effects of the treatments to be
compared to each other