Stats: Modeling the
World
Chapter 13
Experiments
Observational Studies vs. Experiments
Observational study –
A study in which the researcher merely
observes what is happening or what has
happened in the past
Experiment A study in which the researcher imposes a
treatment in order to measure a response
Language of Experiments
Only an experiment can prove causal relationships
Explanatory variable
Response variable
Observational Study or Experiment?
Study: Medicine Mix-ups hurt about 1 in 15 hospitalized kids
Study: Painkillers don’t help older brains
Healthy eating adds up on grocery bills
Ginger helps relieve nausea in chemo patients
Elements of an Experiment
Experimental units:
Subjects or participants:
Factors:
Levels:
Treatments:
Ch
13
Example
A farm products manufacturer wants to determine if the yield of crop
is different when the soil is treated with three different types of
fertilizer. Fifteen similar plots of land are planted with the same type
of seed but are fertilized differently. At the end of the growing
season, the mean yields from the plots will be compared.
a) What are the experimental units?
b) What is the explanatory variable(s)?
c) How many factors are there? How many levels for each factor?
d) What is the response variable?
2004B #2A
The Four Principles of Design
All experimental designs should follow some
basic rules. The four major principles are:
1) Control
2) Randomization
3) Replication
4) Blocking **Not essential, but useful
(we will discuss it later)
Control through Comparison
A control group serves as a reference mark for an
actual treatment to be compared.
Experiments are comparative in nature: We compare the
response to a treatment versus to:
another treatment
no treatment
a placebo
or any combination of the above
Placebos
placebo - A “fake” treatment that looks just like the
treatments being tested.
Blinding
There are two main classes of individuals who can affect
the outcome of an experiment:
(1) those who could influence the results and
(2) those who evaluate the results.
single-blind:
double-blind:
Randomization
Randomization allows us to spread out the effects of
unknown or uncontrollable sources of variation among
the treatments, which helps reduce bias.
**Note: Without randomization, we cannot use the
methods of Statistics to draw conclusions from the
study!
Experiments vs. Samples
How does randomization differ for experiments and
samples?
- Samples
- Experiments
Replication
Two types of replication show up in
experimental designs:
Replication within the study
Replication of the study
Explaining with a Diagram
Completely
Randomized Design
Article Analysis
POD - Article
With your partner, read through the “Placebo Effect”
article
Discuss the main points of the article with your
partner. What did you find interesting?
Write a brief summary of this article in your POD
folder, with the idea that someone NOT in this class
would understand what the article was about.
About the Placebo Effect
The “placebo effect” is an improvement in
health due not to any treatment but only to the
patient’s belief that he or she will improve.
The placebo effect is not understood, but it is
believed to have therapeutic results on up to a
whopping 35% of patients.
Other Important Vocab for
Experiments
Whenever you are
designing an experiment,
there are a few items you
might keep in mind….
Statistical Significance
Do you have ESP?
Statistical Significance
Let’s say Mr. Waddell correctly identified the
hidden card 14 out of 25 times. Do you think this
level of success is statistically significant?
Explain?
How many correct identifications out of 25 would
Mr. Waddellhave to make to convince you that she
has ESP? Explain.
Drawing a Conclusion…
If the goal of an experiment is to compare two treatments,
then how can we tell if a treatment really works?
What does “statistically significant” really mean?
“a difference larger than what we would have expected
due to chance alone.”
Experimental
Units?
Factors? (Exp
Variable)
Treatments?
Response
Variable?
Confounding
Confounding variables are two variables (explanatory or
lurking variables) that are confounded when their effects on a
response variable cannot be distinguished from each other.
studying
CAUS
E?
intelligence
Good grade on test
Confound
ing?
Well-designed experiments take steps to defeat confounding.
POD:
How did
YOU
assign
the
old/new
foods w
respect
to the
temp?
Draw your design in diagram
form…
Blocking – The
th
4
element
When we have a situation in which groups of
subjects are similar, it is often a good idea to
gather them together into blocks.
By blocking, we isolate the variability due to
the differences between the blocks so that we
can see the differences due to the treatments
more clearly.
Blocking
What does it mean to “isolate the
variability”?
Randomized Block Design
We wish to run an experiment on the effects of TV
ads on adults. Since we think that males and
females may react differently, we will separate the
genders first, then run the experiment.
Matched Pairs Design
Choose pairs of subjects that are closely
matched— e.g., same sex, height, weight, age,
and race. Within each pair, randomly assign who
will receive which treatment.
The most closely matched
pairs studies use identical
twins.
Matched Pairs Design
It is also possible to just use a single person
and give the two treatments to this person
over time in random order. In this case,
the “matched pair” is just the same person
at different points in time.
Try it!
All Three Designs…
Ms. Curtis makes soap as a hobby… She wants
to compare her Shea Butter soap to her Goat’s
Milk soap to see which one is better as a facial
soap.
Completely Randomized Design…
Ms. Curtis makes soap as a hobby… She wants to
compare her Shea Butter soap to her Goat’s Milk soap to
see which one is better as a facial soap.
Randomized Block Design…
Matched Pairs Design…