Research methods in clinical psychology:
An introduction for students and practitioners
Chris Barker, Nancy Pistrang, and Robert Elliott
CHAPTER 8
Foundations of design
Design: overview
• Types of designs
• Validity analysis
• Descriptive and correlational designs
– Correlation and causation
• Quasi-experimental designs
• Randomised experimental designs
– Control and comparison groups
Classification of designs
• Non-experimental designs
– Descriptive
– Correlational
• Experimental designs
– Non-randomised (quasi-experimental)
– Randomised
Causal relationships
Central principle:
Correlation does not mean causation
Suppose that A and B are correlated: what
might be the causal relationship between
them?
ctd./
Causal relationships 2
Simple causation:
A
B
B
A
A
Third variable:
B
C
ctd./
Causal relationships 3:
mediating and moderating variables
Mediating variable:
A
D
B
Moderating variable:
A
B
E
(Baron & Kenny, 1986)
Validity analysis
•
•
•
•
Statistical conclusion
Internal
Construct
External
(from Cook and Campbell, 1979)
Statistical conclusion validity
Is the study sensitive enough?
– statistical power
– design quality
Do the variables covary?
If so, how strongly?
Internal validity
If two variables do covary, is there a causal
relationship between them?
Construct validity
Do the outcome variables (and also the
experimental intervention) represent the
underlying constructs that they are supposed
to?
External validity
Do the results of the study generalise (e.g.,
across settings or participants)?
Some non-randomised designs
(Cook & Campbell, 1979)
(Notation:
X = intervention;
O = observation)
One-group posttest-only design
X
0
One group pretest-posttest design
O1
X
O2
Some non-randomised designs (cont.)
Nonequivalent groups posttest-only design
NR
X O
NR
O
Nonequivalent groups pretest-posttest design
NR
O X O
NR
O (Y) O
Some threats to internal validity
•
•
•
•
•
•
•
Endogenous change
Maturational trends
Reactivity of measurement
Secular drift
Interfering events
Regression to the mean
Selection effects
Some threats to the construct validity of
the intervention
• Confounding variables
– “non-specific factors”
• Expectancy effects
– “placebo effects”
• Hawthorne effect
Randomised designs
• Essential feature: randomised assignment to
experimental groups (conditions)
• This controls for the internal validity threat
of selection effects, which is a problem with
quasi-experimental (non-randomised)
designs
RCTs
RCT is a common abbreviation for either:
Randomised Controlled Trial
or
Randomised Clinical Trial
Efficacy (as opposed to effectiveness) research
Example
“Randomised groups pretest-posttest design”
RO
RO
X O
(Y) O
(see Cook & Campbell, 1979)
Randomised designs: terminology
• The dependent variable(s) are the outcome
measures
• The independent variable(s) are the
experimental conditions
• The statistical method is analysis of
variance (ANOVA)
Factors in randomised designs
• Independent variables are arranged as
factors
• each has two or more levels
• multifactorial designs
• within-group (repeated measure) factors
• between-group factors (experimental
conditions)
• blocking factors (participant differences)
Some types of control group
• No-treatment controls
– treatment better than nothing
• Wait-list controls
– control for expectation of benefit
• “Placebo” controls
– credible but inert treatment
– “double- and triple-blind”
• Comparative treatment groups
– Alternative treatment
– “Treatment as usual”
• Dismantling studies
Practical limitations of randomisation
• Non-equivalence of groups
• Attrition
– “intent to treat” analysis
•
•
•
•
•
Leakage
Non-cooperation of staff
Costly and time consuming
Can’t study negative events
Ethical issues
Ethical issues in RCTs
•
•
•
•
•
•
No treatment/placebo controls
Wait-list controls
Random assignment v. choice
Specified treatments v. clinical judgements
Decision on patient inclusion
Referrals at termination
Some good experimental design features (1)
•
•
•
•
Patient homogeneity
Randomised assignment
Groups similar after randomisation
Specific interventions
– manualisation
• Appropriate control groups
• Groups treated equivalently except for
intervention
ctd./
Good design features (2)
• Low attrition
• Patients, clinicians and raters blind
• Follow-up after termination (e.g., 6 months,
one year)
• Independent replication