SOWK 6003 Social Work
Research
Week 6
Designs for Evaluating Programmes and
Practice – Single case study eveluation
By Dr. Paul Wong
Purposes of Research
1.
2.
3.
4.
5.
Exploration
Description
Explanation
Evaluation
Multiple purposes
Relationships between Variables
• Correlations
• Correlational relationship simply says that
two things perform in a synchronized
manner.
Types of Variables
Independent Variables (IVs) vs. Dependent Variables (DVs)
IV is a variable that explains or causes something,
and is not depending on the other variables. It is
what the researcher (or nature) manipulates.
DV is the variable which is the effect, being
explained or caused.
When we talk about study design, we need
to consider the following aspects:
1.
2.
3.
4.
Nature
Setting
Time
Participants
Nature
– Experimental Design - investigation that involves manipulation and
control of an independent or treatment variable with the intent of
assessing whether the independent variables causes a change in the
level of a dependent variable. Note: Randomization : random
assignment to the control and experimental group, which is not equal to
random sampling. It is about the reduction of systematic error such as
selection bias.
– Quasi-Experimental Design - research study set up to resemble a true
experiment but that does not involve random assignment of the
participants to a group or manipulation and control of a true independent
variable, instead relying on measuring groups based on pre-existing
characteristics. (Beins, 2009)
– Pre-Experimental Design - the features of experimental design and
quasi-experimental design are not present.
Setting
• Applied Research: take place in a natural environment
where people are acting as they normally do. It usually
attempts to address practical questions rather than
theoretical questions
•
Basic Research is more likely to occur in a laboratory
or other controlled setting. It tests or expands on
theory, with no direct application intended. (Beins, 2009)
Time
• One time: Cross-sectional/One
shot/Posttest only
• Two time: pretest – posttest design
• More than two times: Longitudinal
• Many many times: Time Series
Participant/Subject
• One person – Case study
• One group – A cohort
• More than one person
• More than one group –
Comparison study
Treatment/intervention
Data Source
• Qualitative
–
–
–
–
–
–
–
Observations
Journal/Diary/Self report
Indepth Interview
Focus Group
Art work
Archives
Documents
• Quantitative
– Measurements through self-administered questionnaires (by
hand, by mail, or online)
– Interview survey (telephone, home visit)
– Health indicators (BP, skin temperature, saliva cortisol etc.)
Internal validity
• Refers to the confidence we have that the
results of a study accurately depict
whether one variable is or is not a cause
of another.
Threats to internal validity
• Means that whenever anything other than the
independent variable can affect the dependent
variable.
–
–
–
–
–
History
Maturation or the passage of time
testing
Instrumentation changes
Statistical regression, check pp.234-235 for an
interesting example
– Selection bias
– Ambiguity about the direction of causal influence
External Validity
• Refers to the extent to which we can
generalize the findings of a study to
settings and populations beyond the study
conditions.
• *** a major factor that influences external
validity is the representativeness of the
study sample, setting, and procedures.
Research designs that can help us
to do evaluation
• Single-case design
• Quasi-experimental
design
• Experimental design
Why single case study?
• Challenges faced by experimental group comparison
design
–
–
–
–
–
–
Ethical objections
Practical problems
Generality of findings
Inter-subject variability
Resources implication: effort, manpower, time and fundings
Complex interactions among clients, therapist, technique and
socio-environmental variables
– More applicable in demonstrating the impact of one set of
techniques is greater than another set under broadly defined
conditions. There is no information about specific mechanisms of
psychological change, and specific technique necessary for such
changes.
Single Case Study
• Also known as: single subject designs, single-case
designs, or single-system designs
– is a method widely adopted in the early years of psychology in
clinical work
– studies with single individuals, whether human or nonhuman
– can involve multiple cases, but likely to be reported individually
– can involve controlled observations, like experiment
– data can be collected from questionnaires, naturalistic
observations, or physiological measurements
– mainly used by behaviorists and clinicians
– taking repeated measures of the dependent variable,
experimental groups can serve as their own controls
Concepts related to single case design
• Baseline Phase (A): repeated measures
that occurs before intervention, serves as
control phase
• Intervention Phase (B): repeated
measures that occurs during the
intervention
What, How and Who to measure?
Operational indicator(s) based on operational definition of problems or
goals
(1) When measured by standardized assessment instruments:
– Participants might lose interest in completing them carefully over and
over again
– Biased for social desirability
– Test-retest effect
(2) When measured by direct behavioral observation:
– Availability of observers: practitioners vs significant others vs self
– Inter-rater reliability
– Unobtrusive vs obtrusive measure
(3) When studied by available record:
– Consistency
– Start intervention earlier with baseline available
Single case study designs
A-B Design: The basic single case design
– popular design with greater feasibility
– can be replicated with multiple clients or at
multiple settings
– provide immediate feedback to clinicians
– criticism: it provides no information about the
natural course of the behavior would have
been had we not intervened with the
treatment condition
A-B Design with follow-up
A-B with follow up and booster treatment
• during the follow up period, there is a booster treatment
introduced- referred as withdrawal design:
• withdrawal of the treatment variable that has been
applied after baseline measurement has concluded.
• withdraw after change has been successfully
demonstrated
• a decrease or increase trend in the baseline after the
intervention is withdrawn.
• more rigorous than the A-B-A design
• ethical: providing good clinical care
• 2 A-B analysis + 1 B-A analysis
A-B with multiple target measures and follow up
• attempt to control for extraneous
variables by having more than one
baseline and intervention phase.
• instead of having more than one
baseline, multiple baselines are
collected simultaneously.
• can be done by measuring different
client, different measurements, or
across different settings
• usually, the baseline starts at the
same time, the introduction of
intervention at different point of time.
Qualitative data in single case study
• use open-ended interviews with the client or the
significant others to try to learn
• whether important extraneous events in the client’s
social environment coincided with the instability
• interviews before baseline can improve our
understanding of the target problem, how to measure it,
and how best to intervene
• information collected from proxy can be used in
triangulation format:
–
–
–
–
–
indepth interview
video-tape of the intervention
open-end question in self-reported questionnaires
event logs
diary
In class activity 1
Article Critique
In class activity 2
After class article search:
Please find an article/chapter that
describes the disadvantages of singlecase study design and summarize it in
point form and send to paulw@hku.hk