Bald and Moore 2009, 2012, Chapters 7 and 8:
Experiments and Comparative Observational Studies
Experiments vs. comparative observational studies
Research
Strategy
Model, Roles, of
variables
Same Purpose
Different
Methodologies
Document1
Comparative Observational
Experiment
Explanatory Variable  Response Variable
Explanatory Variable (Treatment)  Response Variable
XY
Y = f(X) + e
f(X) is mean response as a function of X
e = [Y – f(X)] is random “error”,
i.e., random deviation of individual’s response, Y, from
mean response, f(X)
XY
A model such as Y = f(X) + e can be invoked, but the
both X and Y are random, and the role (explanatory or
response) is not intrinsic.
To study a relationship between explanatory (denoted by X) and random response denoted by Y) variables
X is a random characteristic of the units observed, not
X is a non-random “treatment” randomly assigned at to
manipulated or controlled by the investigator. The
investigator models (i.e. thinks of) X and Y as
experimental units.
Experimenter manipulates and controls X.
explanatory and response, but the role (explanatory or
response) is not intrinsic.
Causality
cannot infer causality on the basis of statistics alone
can infer causality on the basis of statistical results
Reference
B&M Chapter 7
B&M Chapter 8
1
2/10/2016
Can have more than one explanatory variable
Y = f(X1) + e
Y = f(X1, X2) + e
Y = f(X1, X2, X3) + e

There may be “lurking” explanatory variables that are were not included in the study.
Why observe rather than experiment?
1. Unethical or impossible to assign units to certain treatments, e.g., cigarette smoking studies
2. Certain explanatory variables are inherent traits, e.g., sex, gender
3. Ecological Validity, e.g., Jane Goodall, Diane Fossey
Design of Experiments, Terminology (from B&M Chapter 8)
The individuals studied in an experiment are often called subjects, particularly when they are people.
The explanatory variables in an experiment are often called factors (B&M, Section 8.1).
A treatment is any specific experimental condition applied to the subjects. If an experiment has several factors, a treatment is a
combination of specific values of each factor (B&M, Section 8.1).
An experimental group is a group of individuals receiving a treatment whose effect we seek to understand (B&M, Section 8.2).
A control is a treatment meant to serve as a baseline to which the experimental group is compared (B&M, Section 8.2).
A placebo is a control treatment that is fake (for example, taking a sugar pill) but otherwise indistinguishable from the treatment in the
experimental group (B&M, Section 8.2).
Document1
2
2/10/2016
Terms used in design of experiments
experimental unit

smallest basic objects to which we can assign and apply different treatments
observational unit

objects or individuals measured in any study
Note: experimental unit and observational unit can be different objects in the same study.
The sample size is the number of experimental units, not the number of observational units.
Example 1: Effect of diet (high protein, high fat, high carbohydrate) on weight gain (kg) of pigs
A total of 48 pigs are raised in 6 pens, with 8 pigs per pen. In each pen, pigs eat from a common
trough. Two (2) pens are randomly assigned to each diet (high protein, high fat, high
carbohydrate). Each pig’s weight gain (kg) is measured at after 10 days on the diet.
Variable
Type
Diet (hi pro, hi fat, hi carb) qualitative
Wt gain (kg)
Role
explanatory
quantitative response
1. experimental unit is (multiple choice)
a. the pig, because each pig is weighed
b. the pen (Not the pig), because pigs are fed from one trough in each pen
2. observational unit is (multiple choice)
a. the pig, because each pig is weighed
b. the pen (Not the pig), because pigs are fed from one trough in each pen
Document1
3
Pen 1
OOOO
protein
OOOO
Pen 2
OOOO
carb
OOOO
Pen 3
OOOO
protein
OOOO
Pen 4
OOOO
fat
OOOO
Pen 5
OOOO
carb
OOOO
Pen 6
OOOO
fat
OOOO
2/10/2016
Example 2: Effect of diet (high protein, high fat, high carbohydrate) on wt gain (kg) of middle-school boys
A total of 36 boys are fed in 6 schools, with 6 boys per school. In each school, children eat
in a common cafeteria. Two (2) boys in each school are randomly assigned to each diet
(high protein, high fat, high carbohydrate). Each boy’s weight gain (kg) is measured at
after 10 days on the diet.
3. experimental unit is (multiple choice)
a. the boy
b. the school
4. observational unit is (multiple choice)
a. the boy
b. the school
Document1
4
School 1
OOO
OOO
School 2
OOO
OOO
School 3
OOO
OOO
School 4
OOO
OOO
School 5
OOO
OOO
School 6
OOO
OOO
2/10/2016
Random Assignment of Treatments to Experimental Units (or vice versa)
Randomized comparative experiments are designed to give good evidence that differences in the treatments actually cause the
differences we see in the response. The logic is as follows (B&M Section 8.2:

Random assignment of subjects forms groups that should be similar in all respects before the treatments are applied. B&M
Exercise 8.51 uses the Simple Random Sample applet to demonstrate this.

Comparative design ensures that influences other than the experimental treatments (i.e., lurking explanatory variables)
operate equally on all groups.

Therefore, differences in average response must be due either to the treatments or to the play of chance in the random
assignment of subjects to the treatments.
Document1
5
2/10/2016
Design
Restricted Randomization (blocking, repeated measures,
before and after, matched pairs, matched k-tuples)
Completely Randomized Design (CRD)
In a completely randomized experimental design,
 all the experimental units are allocated at random
Definition
among all the treatments, such that
 each experimental unit has an equal chance of
receiving each treatment.
More randomization makes it less likely that influences
Why? other than the experimental treatments. I.e., lurking
variables are less likely to be confounded with treatments.
Reference
Under restricted randomization, experimental units are
subdivided into homogeneous “blocks”, such that
 all the units are as similar as possible in each block,
with
 an equal number (often 1) of experimental unit for
each treatment in each block. I.e., number of units
per block is an integer multiple (often 1) of the
number of treatments.
 Treatments are assigned randomly to the
experimental units within each block.
When there is a great deal of variation (noise) among
experimental units, restricted randomization will eliminate
the uncertainty and potential confounding due to that lack of
homogeneity.
School 1
School 2
School 3
School 1
School 2
School 3
OOO
OOO
OOO
OOO
OOO
OOO
OOO
OOO
OOO
OOO
OOO
OOO
School 4
School 5
School 6
School 4
School 5
School 6
OOO
OOO
OOO
OOO
OOO
OOO
OOO
OOO
OOO
OOO
OOO
OOO
B&M Section 8.2
B&M Section 8.3
Continued in Restricted Randomization
Document1
6
2/10/2016