CHAPTER 4
ISSUES IN SINGLE
SUBJECT RESEARCH
Pelaez, EDP 7058
THE CONCEPTS OF PREDICTION,
VERIFICATION, AND REPLICATION
• The concepts of prediction, verification, and replication
relate to the issues of reliability and validity as
specifically applied to single subject research.
• When these concepts can be demonstrated within a
single subject research design, the functional relationship
between the independent and dependent variables is
evident.
• The reliability and validity of the study are verified
assuming the extraneous variables and systematic bias
cannot be reasonably perceived to account for the
changes that have occurred.
2
Prediction
• Prediction refers to the idea that if there is no effect
attributable to the independent variable, the dependent
variable’s data path will remain unchanged; and one could
predict that it will remain unchanged.
– EX: A researcher has collected baseline data on an individual’s
target behavior. After stability has been achieved during the
baseline phase, the intervention phase would be introduced.
– If the intervention had no effect on the dependent variable, one
could logically assume the data path from baseline to
intervention phases would depict no appreciable change.
Therefore, one could predict that the data path will remain
unchanged despite a phase change.
– Should the data path change and that change be maintained
one could reject the prior notion and possibly verify that the
intervention has an effect on the dependent variable or target
behavior.
3
Verification
• Verification is the confirmation that the dependent variable is
changing in a predictable fashion as the independent variable
is systematically applied.
– EX: A baseline phase is completed and an intervention is
introduced. If the independent variable will have no
effect, we could predict there would be no change in the
data path with that phase change.
– We may predict conversely that the data are likely to
reveal a change in the desired direction as a result of the
implementation of the independent variable.
– When this happens, we have verified our hypothesis that
the dependent variable will change predictably with the
introduction of the independent variable.
– Replication is needed also to complete the demonstration
of a functional relationship.
4
Replication
• Replication refers to the repeating of the observed
predictions and verifications within the same study.
– EX: A practitioner gathers baseline data on math accuracy
and implements a reinforcement program, and the desired
changes are observed. One could state that both a
prediction and a verification are present, but replication is
not present.
– For replication to occur, the researcher might return to a
baseline condition and measure the outcome. Should the
target behavior change during this phase prediction has
been verified.
– The researcher may then reintroduce the intervention,
which allows for replication of the original prediction and
verification. This makes a stronger case for the
demonstration of a functional relationship (ABAB design).
5
REPLICATION CONT.
• The more replications included within a study, the less
changes in the dependent variable are attributable to
extraneous or confounding variables.
• Also, the case for the power (robustness) of an independent
variable to influence a dependent variable is strengthened
when other researchers seek to replicate effects with other
individuals, with similar behaviors, in different settings, and so
on.
6
RELIABILITY AND VALIDITY
• There are three major areas of concern relating to
reliability and validity in single subject research.
Interobserver reliability (sometimes referred to as
interrater reliability or interobserver agreement) is
important to establish confidence in the measurement
of the dependent variable.
Internal validity is important to establish the
believability of the functional relationship and,
therefore, establish confidence in the results and
conclusions drawn.
External validity is related to the confidence others
may have that the same independent variable will yield
similar results in similar studies.
7
Interobserver Reliability
• The researcher must identify and define the target behavior in
such a way that there is confidence that at least two people
can observe the individual and agree whether or not the
behavior has occurred, or to what extent, or for how long, and
so on.
• The researcher identifies at least two people to serve as
observers (one can be the researcher). Then the observers
are trained and practice observing the individual and scoring
the dependent variable.
• Following practice sessions, the observers should compare
results for each and every occurrence or measurement of the
target behavior (dependent variable).
8
Interobserver Reliability cont.
• They should reach an understanding of both why and why not
the target behavior has occurred. When disagreements occur,
it is particularly important to understand and reach
agreement as to what the correct measurement was and why.
• Interobserver reliability should be calculated among and
between observers. The coefficients should be calculated in a
rigorous manner to ensure adequate reliability.
• Some experts (e.g., Barlow & Hensen , 1984) argue that
observers should be naïve to the intervention and purpose of
the study, should not be involved in calculating interobserver
agreement, and should remain unaware of when reliability
checks are scheduled. However, since the researcher is often
involved in data collection such precautions are not always
possible.
9
Reactivity
• Reactivity refers to the individual being observed altering his
or her behavior (i.e. target behavior) as a response to being
observed.
• Reactivity generally diminishes as the number of observations
increases.
• Practice observations in the experimental setting can reduce
the possibility of reactivity when the official data collection
begins, although the presence of observers regularly before
the introduction of the independent variable may have an
unknown and confounding result.
• Reactivity may also be overcome by extending baseline
measurements until there is stability, but also a reasonable
assumption that the performance on the dependent variable
truly represents previous levels unaffected by observer
presence.
10
Observer Drift
• Observer drift refers to a change in interpretation of
when the target behavior occurs or not (or at what
level, or how intensely, or for how long, etc.) from the
original operational definitions.
• Observer drift occurs generally when a number of
observations have been made, particularly if the
communication and comparison of scores among
raters has not been regularly reviewed and discussed.
– EX: Two observers are recording “sitting in the seat”
behavior, and one observer begins to score occurrences
even if the individual has begun to slump over and props
herself with her hand. The other observer, simultaneously
scoring the individual’s performance, may disagree on
what the actual performance was and therefore threaten
interobserver reliability.
11
VALIDITY
• There are several types of validity associated with
measurement in particular (e.g., face validity, content validity,
and predictive validity). These are used less frequently in
single subject research then in group studies, but
understanding them is important.
• Internal and external validity are concepts that apply to both
group and single subject research.
12
Internal Validity
• Internal validity refers to the degree to which the researcher
has adequately controlled the independent, dependent, and
extraneous variables so that changes in the dependent
variable are directly attributable to the presence or absence
or intensity of the independent variable.
– EX 1: A researcher is attempting to help an individual
improve his articulation. The researcher implements a
treatment that is intended to accomplish that objective.
After several weeks, the researcher’s measurements verify
that the individual’s articulation improves, and there is no
reason to suspect that that any other factor or variable
accounted for the improved articulation . The researcher
has convincingly demonstrated the functional relationship
between the independent variable and the dependent
variable.
13
Internal Validity cont.
– EX 2: As an example of diminished internal validity, the
same change in the target behavior occurs. However, the
individual reports that he has been reading materials that
tell how other people overcame their articulation
difficulties. Also, he and his mother have been praying for
divine assistance. The researcher must now acknowledge
that the treatment alone may not account for the changes
in the target behavior.
14
Extraneous Variables
• Extraneous variables refer to virtually anything that may
affect the demonstration of the functional relationship
between the independent and dependent variables.
– History is used to refer to the passage of time and
both foreseen and unforeseen events that arise.
– Maturation refers to the natural development of an
individual that occurs over time.
– Attrition is the loss of subjects during the course of
the study.
15
Extraneous Variables cont.
– Multiple treatment interference is a very real threat in
many single subject studies. Interference may occur when
more than one independent variable (e.g., positive
reinforcement followed by negative reinforcement) is
used.
EX: A package treatment is used (e.g., positive
reinforcement plus verbal prompting plus response
interruption). Because more than one intervention is
being used, it may be difficult if not impossible for the
researcher to accurately determine which of the
components of the independent variable or variables
actually accounted for changes in the dependent
variable.
16
Extraneous Variables cont.
Treatment drift refers to individuals involved in administering
the independent variable producing personal modifications
(consciously or unconsciously) that may influence the impact
of the independent variable on the dependent variable.
Systematic bias refers to the researcher introducing some
element to the experimental conditions that is likely to
influence changes in the dependent variable regardless of the
manipulation of the independent variable.
17
External Validity
• External validity refers to the degree to which the researcher
may have confidence that she or he or other researchers will
obtain the same or similar results if they use the same or very
similar experimental procedures with other individuals, with
other target behaviors, or in other settings.
• The more an experimental effect is replicated by the same or
other researchers, the greater the external validity.
18
Types of Replication
• Direct replication: The researcher attempts to duplicate the
procedures as precisely as possible.
• Intrasubject replication: The same subject is used in a
subsequent study.
• Intersubject replication: The researcher maintains every
aspect of an earlier study, but with different although similar
subjects.
• Systematic replication: The conditions from a previous study
are varied, but still obtaining similar results.
19
Educational Significance
• Educational significance refers to the concern that, although
statistically significant results may be achieved the results
should merit conclusions that the interventions used also
translated into real world significance.
• The question is asked, “Did the interventions result in
outcomes that are meaningful to practitioners and to the lives
of the participants in the study?”
20
ETHICS
• There are many considerations in the use of applied behavior
analysis and single subject research; empirical and social
validity are equally important.
• Social validity refers to the degree to which other people
think that the targeted changes in behavior are important and
that the methods used to encourage behavior change are
acceptable.
• Empirical validity refers to the measurements that actually
demonstrate that the proposed behavioral changes will
indeed positively affect the individual’s life.
21