OBSERVATIONAL STUDY VS.
EXPERIMENT
OBSERVATIONAL STUDY

The researcher observes individuals and
measures variables of interest without
influencing the responses.
GOAL: to draw conclusions about the
corresponding population or about differences
between two or more populations
Observational studies of the effect of one variable
on another often fail because of confounding
between the explanatory variable and one or
more lurking variables.
Definition:
A lurking variable is a variable that is not among the
explanatory or response variables in a study but that may
influence the response variable.
Confounding occurs when two variables are associated in
such a way that their effects on a response variable cannot
be distinguished from each other.
Well-designed experiments take steps to avoid confounding.
EXAMLE

Observe women who take hormones vs. those not
taking hormones and note whether a heart
attack has occurred.
Lurking variables:
 Economic status
 Education
 Frequency of visiting the doctor
EXPERIMENT

The researcher deliberately imposes some
treatment on individuals to measure their
responses.
GOAL: to determine whether the treatment
CAUSES a change in the response
WELL-DESIGNED EXPERIMENT

Can provide evidence for a cause-and-effect
relationship
(an observational study cannot draw such
conclusions because we can’t rule out
confounding variables.)
CAUSE AND EFFECT
Children who drink more milk have bigger feet than children who drink
less milk. There are three possible explanations for this association:
Drinking milk causes
children’s feet to be bigger.
Having bigger feet causes
children to drink more milk.
A lurking variable is responsible for both.
LURKING VARIABLES

Lies in the background that
may or may not be apparent at
the outset but once identified
could explain the pattern
between the variables.

The Language of Experiments
An experiment is a statistical study in which we actually do
something (a treatment) to people, animals, or objects (the
experimental units) to observe the response. Here is the basic
vocabulary of experiments.
Treatment : a specific condition applied to the individuals in an
experiment
•If an experiment has several explanatory variables, a treatment is
a combination of specific values of these variables.
Experimental units: the collection of individuals to which
treatments are applied.
•When the units are human beings, they often are called subjects.
Sometimes, the explanatory variables in an experiment are called
factors. Many experiments study the joint effects of several factors. In
such an experiment, each treatment is formed by combining a specific
value (often called a level) of each of the factors.
DISCUSSION

There are now many special courses that claim to
prepare you for the SAT. Suppose that you want to
evaluate a particular course, using SAT scores to
measure the effect of the course. You might find a
reasonably large high school where students are
offered the chance to take the course, and then
compare the SAT scores of those who completed the
course with the scores of those who chose not to take
it.
Suppose that you find that the average SAT score for students who
took the course is 30 points higher than for students who didn’t.
Identify each of the elements in this study: population, response
variable, treatments
POP- students who took the SAT
Response Variable- SAT score
Treatments- taking the course/ not taking the course

Is this study a true experiment?
NO- the students are not randomly assigned to the two treatments

Do you conclude that the course causes an increase in SAT
scores? Why or why not?
NO- those who took the course may be unrepresentative due to
selection bias. Those who take the course may tend to score
higher anyway.

EACH PAIR OF VARIABLES SHOWN ARE STRONGLY
ASSOCIATED. DOES A CAUSE B OR DOES B CAUSE
A, OR IS THERE A LURKING VARIABLE?
1. A: having hip surgery
B: dying within the next 10 years
Lurking variable- person’s age
2. A: the amount of milk a person drinks
B: the strength of a person’s bones
3. A: the amount of money a person earns
B: the number of years a person went to school
4. A. the number of classes taken with Mrs. Sapp
B. level of happiness
THYMUS EXAMPLE

In 1912, Dr. Charles Mayo published an article
recommending removal of the thymus as
treatment for respiratory problems in children.
He made this recommendation even though 1/3 of
the children died after the operation. Look at the
study:
AGE
Thymus
size
CHILD
ADULT
LARGE
Problems
No evidence
SMALL
No evidence
No problems
Why is it impossible to know whether a large thymus causes
respiratory problems?
Age and size of thymus were confounded
THYMUS STUDY
Suppose that the surgeons had examined infants
without respiratory problems and found that their
thymus was generally small.
 Have they now proved that a large thymus causes
respiratory problems in children?
Design an experiment to determine if removal of the
thymus helps children with respiratory problems.
Randomly divide the children with respiratory problems
into two groups: one that gets surgery and one that
does not. Otherwise treat them exactly alike. Record
how many survived under each treatment.

CALCULUS EXAMPLE
Pg. 44
Why was this a poorly designed experiment?
What were some possible extraneous factors?

Model for experiments
Experimental
Units
Treatment
Measure
Response
NOTE: In the lab environment, simple designs often work well.
Field experiments and experiments with animals or people deal
with more variable conditions.
Outside the lab, badly designed experiments often
yield worthless results because of confounding.
EXAMPLES
Suppose Starbucks wishes to find out whether the
population of MHS students prefer hot or cold,
frozen coffee drinks. A random sample of
students is selected, and each one is asked to try
first hot coffee and then frozen coffee, or vice
versa (with the order determined at random).
They then indicate which type they prefer.
Experiment or Observational Study?
A researcher is interested in the effects of
excessive homework on family dinner nights.
She surveys students on their homework load,
and the number of family nights they have been
required to forego. She concludes that homework
load does not directly affect family nights.
Experiment or Observational Study?

Suppose an experiment is designed to investigate
the effects of repeated exposure to TV ads. All
subjects viewed a 40 minute TV program that
included ads for an iPad. Some subjects saw a
30- second commercial; others, a 90-second
version. The same commercial was shown either
1, 3, or 5 times during the program. After
viewing, all the subjects answered questions
about their recall of the ad, their attitude toward
the iPad and their intention to purchase it.
Identify the explanatory and response variables:
List all the treatments: