Sec. 5.2 ~ Designing Experiments
Objectives:

To understand and use the statistical vocabulary involved in designing experiments.

To design experiments that come as close as possible to eliminating lurking variables and bias.
Experimental Units, Subjects, Treatment

A study is an experiment when we actually do something to people, animals, or objects in order to observe the
response.

The individuals on which the experiment is done are the experimental units.

When the units are human beings, they are called subjects.

A specific experimental condition applied to the units is called a treatment.

The _______________________________ in an experiment are often called _______________because we want
to study the effects that these factors have on the ___________________.

In many experiments we want to study the joint effects of several factors.

In such an experiment, each treatment is formed by __________________________(often called a
level) of each of the factors.

See example 5.9 on p.290
Experiments

A placebo is a ___________________that creates a situation so the experimenter can see what the effect of the
subject will be ______________________

In principle, experiments can give good evidence for causation.

Experiments have the advantage of allowing us to study the specific factors we are interested in, while
controlling the effects of lurking variables.


For example, the subjects in the Physicians’ Health Study were all middle-aged male doctors and all
followed the same schedule of medical checkups. These similarities reduce variation among the
subjects and make any effects of aspirin or beta carotene easier to see.
Experiments also allow us to study the combined effects of several factors.

The interaction of several factors can produce effects that could not be ___________________ from
looking at the effects of each factor alone.

For example, the Physicians’ Health Study tells us that aspirin helps prevent heart attacks, at least in
middle-aged men, and that beta carotene taken with the aspirin neither helps nor hinders aspirin’s
protective powers.
Comparative Experiments

Placebo effect (a dummy treatment) – Many patients respond favorably to any treatment, even a sugar pill.

See example 5.11 on p.292

Control group – The group that receives the sham treatment (placebo).

Often times we rely on the controlled environment of a laboratory to protect us from lurking variables.

Control is the first basic principle of statistical design of experiments.

Comparison of several treatments in the __________________________ is the simplest form of control.
Randomization


Experimenters often attempt to match the patients in both groups (treatment and control) by similarities.

For example, medical researchers testing the effects of a new cancer treatment may try to match the
patients in a “new drug” experimental group and a “standard drug” control group by age, sex, physical
condition, smoker or not, and so on.

This is not adequate because there are ___________________________________________ (such as
how advanced a cancer patient’s disease is).
The statistician’s remedy is to rely on ______________________________________ that does not depend on
any characteristic of the experimental units and does not rely of the judgment of the experimenter in any way.

See example 5.12 on p.295

If you assign many units to each group then the ______________________________________and there
will be little difference in the response variable unless the ____________________________________.

Use enough experimental units to reduce chance variation.
Completely Randomized Design

When all experimental units are_______________________________________________, the experimental
design is _______________________________

See example 5.13 on p.297
Principles of Experimental Design

The basic principles of statistical design of experiments are:
1. Control the effects of lurking variables on the response, most simply by comparing two or more
treatments.
2. Randomize – use impersonal chance to assign experimental units to treatments.
3. Replicate each treatment on many units to reduce chance variation in the results.
Statistical Significance

An observed effect is so large that it would rarely occur by chance is called statistically significant.

You will see the phrase “ statistically significant” in reports of investigations in many fields of study. It tells you
that the investigators found ___________________________ for the effect they were seeking.

When all experimental units are allocated at ___________________ among _________________________ the
experimental design is _______________________________
Cautions about Expermientation

The logic of a ______________________________________ depends on our ability to
___________________________________________ in every way __________________________________

In a double-blind experiment neither the subjects nor the people who have contact with them know which
treatment a subject received.

In a single-blind experiment only the subject does not know whether they are part of the treatment or control
group.

The most serious potential weakness of experiments is _______________________. The subjects or treatments
or setting of an experiment _______________________________________________________________
Matched Pairs Design


Completely randomized designs are the simplest statistical designs for experiments.

However, completely randomized designs are often inferior to more elaborate statistical designs.

In particular, matching the subjects in various ways can produce more precise results than simple
randomization.
Match pairs designs compare just two treatments. We choose blocks of two units that are as closely matched
as possible.

See example 5.16 on p.301
Blocking Designs

Matched pairs is an example of blocking design.

A block is a ________________________________________________that are known before the experiment to
________________________________________________________________________________

In a block design, the random assignment of units to treatments is carried out separately within each
block.

Block designs can have blocks of any size

A block design combines the idea of creating equivalent treatment groups by matching with the principle of
forming treatment groups at random.

See example 5.17 & 5.18 on p.302-303