Experimental Designs
Evidences for Causality
• Evidences for Causality
• Components of Experiments
• Internal Validity and External Validity
• Types of Experimental Designs
• Covariation
• The cause occurs before (or
simultaneously) the effect
• Elimination of other causal factors
1
Components of
Experimental Design
2
Validity
• Treatments: Independent variables
• Test units
• Dependent variables
• Extraneous variables
• Internal validity- Controlling
extraneous variables
• External validity (EV) Generalizability
3
4
1
Symbols
Example 1
• X - the exposure of test units to
treatment
• O – an observation or a measure on
the dependent variable
• R - random assignment of test units
to treatment
• O1 (an exam on the first date of class)
• X (course of research methods)
• O2 (an exam on the last date of class)
•
O1 X
O2
5
Example 2: Selection of
hamsters in a diet experiment
6
Example 2 (Cont.)
How to assign 10 hamsters into E-group
and C-group?
• Alternative A: Put the first 5 hamsters
captured from the cage into E-group.
• Alternative B: Flip a coin to assign
each captured hamster until there is 5
in one cage.
Which alternative will you suggest?
What is the probability that the first
two hamsters are in the same group?
What is the probability that the last
two hamsters are in the same group?
7
8
2
Internal Validity
Internal Validity (Cont.)
• History
• Maturation
• Testing
-Main testing effects: (O1 → O2)
-Interactive testing effect (EV):
→O2)
O1 → (X→
• Instrumentation
• Statistical Regression
• Selection Bias
• Test Unit Mortality
9
Types of Experiments
10
Preexperimental Designs
• One-Shot Case Study
X O
• One-Group Pretest-Posttest Design
O1
X O2
• Static Group Comparison
Group 1:
X O1
Group 2:
O2
• Preexperimental Designs
• True Experimental Designs
• Quasi-Experimentation
11
12
3
True Experimental Designs
True Experiments (Cont.)
• Pretest-Posttest Control Group
E: R O1 X O2
C: R O3
O4
• Posttest-Only Control Group
E: R X O1
C: R
O2
• Solomon Four Group
E1: R O1 X O2
C1: R O3
O4
E2: R
X O5
C2: R
O6
13
14
Quasi-Experimental Designs
Quasi-Experimental Designs
• Only the administration of treatments
and the measurements can be
controlled;
• Subjects are not randomly assigned to
groups.
• Simple Time Series Design
• Equivalent Time-Sample Design
• Nonequivalent Control Group
Design
• Multiple Group Time Series Design
15
16
4
Simple Time Series Design
• Similar to pretest-posttest design
• Repeated measurements of the same
group of test units before and after
the treatment
• Can we have more confidence in the
causal inference in this design than
in the pretest-posttest design?
17
Example
Treatment
Day
Response Y1
Y2
Y3
No
Mon
4
0
8
No
Tue
3
1
21
No
Wed
5
5
5
Yes
Thu
15
15
15
No
Fri
7
25
9
No
Sat
4
40
2
No
Sun
5
60
13
19
18
What would you infer from
Y1, Y 2 and Y3, respectively?
•Maturation
•Instrumentation
•Main testing
•Statistical regression
•History
20
5
Equivalent Time-Sample Design
(Cont.)
Equivalent Time-Sample Design
X0 O1 X1 O2 X0 O3 X1 O4 X0 O5 X1 O6
• Repeatable stimulus
• Assigning the treatment at random
if possible
• Choosing interval between
observations with care
Why can we have more confidence in
the inference than when the
treatment was administered only
once?
21
22
Nonequivalent
Control-Group Design
Example
Treatment
No
No
Yes
No
Yes
No
Yes
Day
Mon
Tue
Wed
Thu
Fri
Sat
Sun
Combine pretest-posttest design with static
group comparison design.
Response Y
3
4
16
3
12
6
15
Time
Example 1
E-group
C-group
T=1
5
5
Treatment is assigned.
T=2
23
15
5
24
6
Example 2
Example 3
Time
E-group
C-group
Time
E-group
C-group
T=1
5
5
T=1
5
0
Treatment is assigned.
T=2
15
Treatment is assigned.
15
T=2
25
15
10
26
7