Experiments vs. Observational Studies
 Experiments
 Observational Studies
 Observe responses to
 Observe responses to
variables
 Administer a treatment
in order to observe the
response to the
treatment
 Can determine causation
variables
 Simply observes
responses, no attempt to
influence them
 Can NOT determine
causation (only
correlation)
Causation
 This may be the most significant reason for conducting an
experiment as opposed to an observational study.
 Most often, people are interested in causation.
 For example:




Will this drug cause my headache to go away?
Does sending my child to daycare cause my child to be behind later
in life?
Does making students pass a test to graduate cause improvement in
our nation’s education?
Will invading Iraq cause long-term peace in that country?
 Unfortunately, experiments (which must control for other
variables) are simply not possible to conduct.
 When it is possible, it is the way to go!
Experiments and Terminology
 Experimental Units
 The things that the experiment is done on
 Also called subjects (when they are human)
 Treatment
 What is actually done to the experimental units
 Factors
 The different types of treatments (which are the
explanatory variables)
 Levels
 Differing amounts of the treatment or factors
Experiments and terminology
 How do the terms apply?
 Consider the experiment:
 A consumer advocacy group is curious about the
effectiveness of pain medication in treating migraine
headaches. They randomly give different doses of aspirin,
tylenol, and ibuprofen to migraine sufferers. They then
measure the results and compare.
 Apply the terms experimental unit, treatment,
factor and level to the scenario above.
Experiments and terminology
 What were the experimental units?
 The migraine patients (since human, subjects)
 What was the treatment?
 The pain medication
 What were the factors?
 The aspirin, tylenol, and ibuprofen
 What were the levels?
 The dosages of the drugs
EXAMPLE
The Characteristics of an Experiment
The English Department of a community college is considering
adopting an online version of the freshman English course. To
compare the new online course to the traditional course, an English
Department faculty member randomly splits a section of her course.
Half of the students receive the traditional course and the other half
is given an online version. At the end of the semester, both groups
will be given a test to determine which performed better.
(a) Who are the experimental units? The students in the class
(b) What is the population for which this study applies? All students who
enroll in the class
(c) What are the treatments? Traditional vs. online instruction
(d) What is the response variable? Exam score
(e) Why can’t this experiment be conducted with blinding?
Both the students and instructor know which treatment they are receiving
1-6
More terminology
 Just like bivariate data, there is typically an explanatory
variable and a response variable in an experiment.
 Consider the scenario:

A teacher wants to see if a new computer program can more
effectively increase the reading ability of students than a traditional
classroom setting. She first tests each student from a 4th grade class.
She then randomly selects half of the class to participate in the
computer program and the other half in the traditional curriculum.
At the end of the year, she tests the students reading ability again
and compares the results.
 Identify the explanatory variable.
 Whether they received the computer program or traditional
curriculum
 Identify the response variable.
 The difference in the results of the reading ability tests.
Comparative Experiments
 Most experiments are comparative.
 That is, the purpose of the experiment is to compare
a treatment to a lack of treatment or to compare two
or more treatments.
Experimental Design Overview
 Simple Experiments (3 models)
Administer
Treatment
Observe
Results
Administer
Treatment #1
Administer
Treatment #2
Observe
Results & Differences
Administer
Treatment #3
Observe
Response Variable
Administer
Treatment
Observe
Response Variable
Nature of Experiments
 The nature of an experiment is to focus in on
causation.
 This is done by controlling variables.
 Variables are controlled through randomization and
the use of control groups.
A completely randomized design is one in which
each experimental unit is randomly assigned to a
treatment.
1-11
Completely Randomized Experiment
 A graphical model is typically used for designing
experiments.
 Consider the question, “Does smoking cause lung
cancer?”
 Unfortunately, a direct experiment would be
unethical, but here is what it would look like.
Smoking and Lung Cancer
 Start with, say, 400 volunteers who have never smoked





before.
Randomly choose 200 of them for the experimental group
and the other 200 form the control group.
The treatment is to smoke 1 pack of cigarettes per day.
The control group does not smoke at all.
Then, track the volunteers for, say 40 years.
At the end of the 40 years, count up how many in each
group developed lung cancer.
 This is a good description of the experiment, the next slide
shows the same thing in diagram form.
Smoking and Lung Cancer
Must state HOW to
randomly allocate –
use a hat
Randomly Allocate
400 Volunteers
Experimental Group (200)
Smoke 1 pack of cigarettes
per day for 40 years
Measure Data
Did lung cancer develop?
Observe Results
Draw Conclusions
Control Group (200)
No smoking for 40 years
Measure Data
Did lung cancer develop?
EXAMPLE Designing an Experiment
The octane of fuel is a measure of its resistance to
detonation with a higher number indicating higher
resistance. An engineer wants to know whether the level
of octane in gasoline affects the gas mileage of an
automobile. Assist the engineer in designing an
experiment.
1-15
Completely Randomized Design
1-16