Chapter 1: Introduction to Statistical Inference: One Proportion
I
Chapter 1. Introduction to Statistical Inference:
One Proportion
Learning Objectives:
•
•
•
•
•
•
•
Begin to understand the process of statistical investigations as it relates to the scientific
method: from posing a question to collecting data to analyzing data to drawing inferences
beyond the data to communicating results.
Be able to distinguish population from sample, parameter from statistic.
Be able to state null and alternative hypotheses appropriate to a research question and
conjecture.
Begin to understand the reasoning process of statistical significance, using a 3S strategy:
statistic, simulate, strength of evidence.
Be able to conduct simulations and draw conclusions for inferences regarding strength of
evidence about a (single) proportion.
Begin to understand that a dataset allows for rejecting some hypothesized values of a
parameter while retaining others as plausible values of the parameter.
Begin to recognize that sample size plays an important role in assessing statistical
significance and strength of evidence.
Outline:
Section 1.1: Introduction to Statistics
Topics
• Anecdotal evidence
• Statistics from samples
• Parameters from populations
• Descriptive statistics versus
inferential statistics
• Scientific Method
Section 1.2: Introduction to Statistical
Reasoning: One Proportion
Example: Matching Names to Faces
Activity: A Preference Study: Friend or Foe?
Topics:
• Criminal Justice System as analogy
for Statistics
• Null and Alternative Hypotheses
• Test of significance, P-value
• Simulation
• Strength of evidence
• Stating conclusions
Section 1.3: Statistical Significance for One
Proportion—Other Null Hypotheses
Example: Appetizing Dog Food?
Activity: Cola Discrimination?
Topics:
• Testing null value other than 0.5
Section 1.4: Plausible Values
Copyright © 2010
Example: Competitive advantage of red
uniforms?
Activity: Kissing Right?
• Interval of plausible values
• Not finding evidence against a value
does not mean confirming the value
Section 1.5: Effect of Sample Size
Example: Predicting election results?
Activity: Baseball “big bang”?
• Sample size matters
• All else being equal, larger samples
produce stronger evidence against a
null hypothesis than smaller
samples do.
Case Study: Will skipping breakfast lead to
a world without men?
Research Article: Stock Monkey
Practice Exercises
Exercises
Chapter 1: Introduction to Statistical Inference: One Proportion
II
Section 1.1: Introduction to Statistics
Have you ever heard statements like these?
• “I don’t wear my seatbelt anymore because a friend of mine was in a car accident
and wasn’t wearing a seatbelt when they crashed and she was the only survivor
of the accident.”
• “Don’t get your child vaccinated. I vaccinated my child and now he is autistic.”
• “I’m never going to become a runner, my friend’s dad just started running and he
died of a heart attack last week.”
The people making these statements each use a single case to support a decision about
how they will live their life. They are basing their conclusions on anecdotal evidence.
Key Idea: Anecdotal evidence is information obtained from only one or a handful of
particular cases. It doesn’t take into account that there are many other cases that may
have very different outcomes.
Thought Question: Do you think it is ever reasonable to draw a conclusion based on
anecdotal evidence?
We think most people would agree that drawing conclusions based on a single case is
often a bad idea. However, there are a lot of everyday situations where we use
anecdotal evidence to make decisions. For example, if you’ve been in a windowless
room all day and someone enters the room holding a wet umbrella, you might conclude
that it is raining outside. In this case, the use of a single case to make a conclusion is
reasonable and probably correct.
Of course, there are many scenarios where anecdotal evidence is insufficient for
drawing a conclusion or making a decision. Typically, in these scenarios the risk
involved with being wrong is higher than in situations where we are comfortable using
anecdotal evidence. For example, in the statements earlier, the ramifications of being
wrong about wearing your seatbelt or vaccinating your child are more severe than
getting your hair wet! In these scenarios, we would prefer to see many cases, which
each provide some evidence, in order to make a better conclusion or decision than what
we can with only a few cases.
Specifically, before we decide to stop wearing seatbelt, we would like to know the
survival rates of individuals wearing seatbelts and individuals not wearing seatbelts in a
large number of similar accidents. Before we decide not to vaccinate our child, we
would like to know the rates of autism in a large number of vaccinated and unvaccinated
children. We would also like to know the rates of disease and death in a large number of
vaccinated and unvaccinated children. Before we put the skids on our running, and
before we even look at a large number of runners and non-runners, we might find out a
little bit more about that friend’s dad. Was he overweight? Was he a smoker? Did he
have a family history of heart attacks? Gathering information or data can help us to
make better informed decisions. This is Statistics at work.
Key Idea: Basing decisions on anecdotal evidence is often not appropriate. Instead, it is
often appropriate to draw conclusions only after we have gathered sufficiently large
amounts of data in a carefully planned manner.
What is Statistics?
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
III
Statistics is a discipline that guides us in weighing evidence about phenomena in the
world around us. More specifically, Statistics gives us a formal procedure for gathering
evidence, evaluating that evidence, suggesting conclusions based on that evidence, and
assessing our confidence in those conclusions.
A large part of the discipline of Statistics (capital “S”) is the use of descriptive statistics
(little “s”). You’ve likely come across and used statistics before; they are numbers like
averages and percentages or graphs like bar charts. These numbers and graphs are
called descriptive statistics because they are used to describe data that we have
collected. If we continue beyond the data we have collected and make broader claims,
this process is called Inferential Statistics.
We can use descriptive and inferential statistics when we are trying to learn about a
large and difficult to observe group of people, called the population, but we only have
data on a portion of that population, called the sample. For example, when we are trying
to learn about vaccine use and autism, the population of interest is children, but we
could never investigate all living children. Instead we use a sample of children and
investigate the relationship between vaccinations and autism in the sample. Descriptive
statistics (e.g., percentages of vaccinated children and unvaccinated children in our
sample who are autistic) are calculated, and then used to make conclusions or
inferences about the population of interest, all living children.
Key Idea: Descriptive statistics summarize, with graphs and numbers, what we see in
the sample. Inferential statistics involves weighing the evidence to make conclusions
about the population.
As we’ve already pointed out, numbers that we calculate from our sample are called
statistics. On the other hand, numbers that summarize information about the population
are called parameters. The statistics from our sample can help us draw conclusions
about the corresponding parameters in the population. In our previous example, we use
the descriptive statistic (the percentage of vaccinated children in our sample who are
autistic), to learn about the parameter (the percentage of all vaccinated children who are
autistic).
Key Idea: Descriptive statistics are numerical summaries of the sample, which can be
used to learn about parameters, which are numerical summaries of the population.
Sometimes instead of using statistics to summarize sample data and draw conclusions
about a population, we use statistics to see whether what a researcher’s manipulations
have made an impact on a response of interest. For example, in an experiment we might
see whether taking a newly developed drug lowers blood pressure. Specifically, we
might measure the blood pressure of everyone in our sample, then give them the drug
and measure their blood pressure again. If, overall, we see a decrease in blood pressure
after taking the drug, Statistics can help us decide if we should be convinced that this
change can be attributed to taking the drug.
Key Idea: Statistical methods can be used to tell us whether researcher intervention is a
reasonable explanation for changes in a response.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
IV
In this book we will use Statistics in a variety of real-life situations to help us to weigh
evidence about research questions of interest, including the following:
•
•
•
•
•
•
•
Is swimming with dolphins successful therapy for people diagnosed with severe
to moderate depression?
Do pre-verbal infants have preferences between helper toys and hinderer toys?
Do females have a higher body temperature than males?
Does vitamin C prevent the common cold?
Can people correctly distinguish between two brands of cola?
What is the average weight of newborn babies?
Are there effects of sleep deprivation on learning?
Scientific Method
The scientific method is a series of techniques used to objectively guide scientific
inquiry. When we use the scientific method we start by asking questions, which we then
refine into testable hypotheses based on prior research (if available). Studies are
designed to test the hypotheses. Results from the studies are analyzed, conclusions are
drawn, and often new research hypotheses are formed. Statistics informs all parts of the
scientific method. See Figure 1.1 for an outline of the scientific method.
Specifically, knowledge of Statistics helps to refine questions into testable hypotheses,
guides many study design decisions, is used to evaluate whether the results of the study
are evidence in favor of the research hypotheses, and guides the ultimate conclusions
drawn from the research. In short, Statistics is used in many studies that use the
scientific method
Figure 1.1: Flowchart of Scientific Method
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
V
Example: The Physicians’ health study conducted by the Harvard Medical School began
in 1982 and ended in 1995. One question this study hoped to answer was: Is a bi-daily
aspirin beneficial in the prevention of cardiovascular disease? Approximately twenty-two
thousand male physicians ages 40 to 84 were split randomly into two groups of
approximately 11,000 physicians each. One group took an aspirin every other day for
the duration of the study and the other group took a placebo (sugar pill) every other day
for the duration of the study. It was concluded that aspirin reduced the risk of a first
myocardial infarction (heart attack) by 44%.
The Physician’s health study is an example of applying the scientific method.
Figure 1.2: Flowchart of Scientific Method applied to Physician’s health study.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
VI
Section 1.2: Introduction to Statistical Reasoning: One
Proportion
Ei incumbit probatio qui dicit --- The burden of proof rests on one who asserts
Statistics and the Criminal Justice System
In our criminal justice system, each trial is designed to answer the same question: “Is the
defendant guilty?” The system involves a jury that evaluates the strength of the
evidence suggesting guilt of the defendant. An important initial instruction is given to the
jury, to assume that the defendant is innocent, and not to conclude that the defendant is
guilty unless the evidence is “beyond a reasonable doubt.”
Notice that there are two competing hypotheses here. The first hypothesis is that the
defendant is innocent. It is this first hypothesis that the jury is instructed to assume is
true. The second hypothesis is that the defendant is guilty. It is this hypothesis that the
prosecutor believes to be true. Indeed, if the prosecutor did not have a strong reason to
suspect the defendant’s guilt he would not have brought the defendant to trial. In the
trial, evidence is presented. The jury then examines the evidence in order to gauge
whether the evidence strongly points to the defendant’s guilt. The jury must examine
every piece of evidence assuming that the defendant is innocent; asking themselves the
question “Is it possible that the defendant is innocent and we still see this evidence? Is it
unlikely? How unlikely?”
The logic of the criminal justice system is similar to the approach we use for weighing
evidence in Statistics. In Statistics we have a research conjecture we want to evaluate;
we call this research conjecture the alternative hypothesis. To see whether sufficient
evidence exists to conclude that our research conjecture is reasonable, we start by
assuming that the research conjecture is not correct. The statement of the research
conjecture being incorrect, stated in such a way as to communicate no effect or equality,
is known as the null hypothesis. In short, we assume the null hypothesis is true, and
evaluate the data under that assumption in order to weigh the evidence supporting the
alternative hypothesis (our research question). And so, the null hypothesis is like
assuming the defendant is innocent, the data we gather is the evidence, and we are
looking for strong evidence in favor of the alternative hypothesis (research question; the
defendant is guilty).
Key Idea: The alternative hypothesis is the research conjecture we are trying to
establish. The null hypothesis is a statement contrary to the research conjecture, which
is often a statement of no effect or equality. As in our criminal justice system, we will
assume that the null hypothesis is true unless there is evidence (data), beyond a
reasonable doubt, supporting the alternative hypothesis. If we don’t find strong enough
evidence, the null hypothesis remains plausible.
This process of assuming the null hypothesis (or the “dull hypothesis”) to be true,
gathering data, then analyzing that data to see whether we have convincing evidence in
favor of the alternative hypothesis is called a test of significance.
Key Idea: Tests of significance use the data gathered to assess the strength of evidence
in favor of the alternative hypothesis over the null hypothesis.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
VII
Example: Matching Names to Faces: Bob or Tim?
A study in Psychonomic Bulletin and Review (Lea, Thomas, Lamkin, & Bell, 2007)
presented evidence that “people use facial prototypes when they encounter different
names.” Participants were given two faces and had to determine which one was Tim and
which one was Bob. The researchers wrote that their participants “overwhelmingly
agreed” on which face belonged to Tim and which face belonged to Bob, but did not
provide the exact results of the study. A recent class of statistics students (our sample)
replicated this study and 23 of the 33 students correctly identified the face that belonged
to Tim. So our statistic could be the 23/33 ≈ 0.697 proportion that made the correct
identification.
But do 23 of the 33 students making that choice convince us that there is something to
this theory? Maybe these students just got lucky? What does that mean here?
If our research conjecture is that people in general (our population) have a tendency to
associate certain facial features with a name, this gives us our alternative hypothesis.
Note: We might want to debate what population you are willing to consider this sample
representative of, but we will come back to that.
Alternative hypothesis: In the population, people have a tendency to associate
certain facial features with a name. In other words, the correct face is matched
with Tim more than half the time (i.e., the parameter, the proportion in the
population, is greater than 0.5).
The null hypothesis then becomes there is no such association. This means that people
trying to match names to faces are essentially blindly guessing which name goes with
which face.
Null hypothesis: People are not more likely to match Tim with one face over the
other. In other words, people match Tim with his face half the time (i.e., the
parameter, the proportion in the population, is equal to 0.5).
So, like in a criminal trial, we will begin our analysis by assuming there is nothing special
going on here; in other words that the null hypothesis is true and people are blindly
guessing which face is Tim’s. If that were the case, would it be possible that in a sample
of 33 students, more than half would guess the Tim face correctly just by chance?
Sure, this is possible. But what if all 33 students had matched Tim correctly? Well then,
we would be pretty convinced there was something going on. Why? Because it’s unlikely
that everyone would get the right answer if they were all blindly guessing. How unlikely?
That would be like flipping a coin 33 times and getting all heads! This would happen
less than 1 in a billion times!
So what about 23 people matching names to faces correctly just by chance? How
unlikely is that? We know that each time we toss a coin 33 times we will get a different
value for the number of heads. Is it unusual to get 23 heads? How unusual? We will
explore this by looking at what “could have happened” when we toss a coin 33 times.
Figure 1.3 shows the results from tossing a coin 33 times, sorted by heads or tails.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
VIII
Figure 1.3: Results from tossing a coin 33 times
This time we got 15 heads and 18 tails, pretty close to the 50/50 split we would expect.
But when we did it again we got 20 heads as shown in Figure 1.4.
Figure 1.4: Results from tossing a coin 33 times a second time
There are a lot of different possible “number of heads” we could get when flipping a coin
33 times, and the result will potentially vary every time we flip the coin another 33 times.
So to help us assess whether 23 heads is unusual, we need to flip the coin 33 times
over and over again. Below is a graph of the number of heads we obtained in 33 tosses,
when we performed the 33 coin flips 1000 different times.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
IX
Figure 1.5: A graph showing 1000 repetitions of flipping a fair coin 33 times and
counting the number of heads.
lower tail
6
7 8
outcomes
6 4 typical
44 7
4 4 48
upper tail
6
7 8
Examining the graph in Figure 1.5 (a “dotplot”) confirms that it is virtually impossible to
get all 33 heads (it never happened), there is clearly some chance we will see 23 heads.
Specifically, in these 1000 sets of 33 tosses, we obtained exactly 23 heads 11 times,
and we got 23 or more heads in 17 of the 1000 times. Therefore, a result like 23 is not
very likely. In fact, 23 lies out in the “tail” of the distribution. So our conclusion would be
that a result of 23 heads in 33 tosses is fairly unusual.
Thought Question: How does the above analysis help us answer our research question
as to whether we have convincing evidence that people are able to match names to
faces more than half the time?
So what does this have to do with our study? Well, if the null hypothesis is true and
people can’t match names to faces any better than with a coin toss, we can model their
behavior with a coin. Heads means they matched the names correct, tails means they
did not. This doesn’t mean every sample is expected to have exactly half the people
match the right name; we know there will be some variability, just be chance. By
repeating this process over and over again, like we showed in Figure 1.5 we get a sense
for what the pattern of outcomes looks like if people are blindly guessing at which name
belongs to which face. This allows us to evaluate whether the results of our students
appear to be consistent with the random behavior we would see from coin tosses.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
X
Figure 1.6: A graph showing 1000 repetitions of a group of 33 students picking
Tim’s face and counting the number of correct matches under the null hypothesis
that people are equally likely to match his face correctly as not.
lower7 8
tail
6
typical
outcomes
if null
is
true
6 4 4
4 7 4
4 4
8
upper
tail
6
7 8
The graph shown in Figure 1.6 is the same as the one shown in Figure 1.5, but we have
changed the context from repeatedly flipping 33 coins and counting the number of heads
to repeatedly sampling 33 people and counting the number of times each group correctly
identifies the correct face. In both cases, the simulation is built on the premise that the
probability of a “success” (heads or a correct guess) equals .5. This graph tells us that, if
people are simply guessing between the two faces, getting an outcome of 23 matches is
rather unlikely. But we DID see 23 correct matches. So we can conclude that the
subjects in the study appear to have behaved differently than they would have if they
were making their decisions by the result of a coin toss. Our data did not appear
consistent with the null hypothesis, so we conclude that we have evidence against the
null hypothesis, beyond a reasonable doubt.
The more unusual the outcome is under the assumption that the null hypothesis is true,
the stronger the evidence provided by the observed data against the null hypothesis that
generated the “could have happened” results.
Note that a key step in this process was assuming the null hypothesis was true so that
the people in the study were behaving like they each tossed a coin to make their choice.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XI
To help us determine whether 23 was an unlikely outcome in 33 tosses, we could either
toss a coin 33 times, over and over, or we could use a computer to carry out the coin
tosses. Typically, instead of actually flipping coins many times, we use a computer to
simulate the “could have happened” data by assuming the people behaved like coins
and then generating random coin toss results.
Of course, people don’t behave exactly like coins, even when they are equally choosing
between two choices. But this model gives us a reasonable assessment of how unusual
our sample results are under this null hypothesis.
Key Idea: A simulation is the imitation of a real world process that is typically performed
by a computer. The use of a computer allows us to repeat the process many, many
times very quickly.
To test some different hypotheses we will use a three step strategy called the “three S”
strategy. This process serves as the foundation for a majority of research questions we
will investigate in this book.
Three S Strategy for Test of Significance
1. Statistic: Compute the statistic from the observed data.
2. Simulate: Simulate the process to produce “could have happened” data under the
assumption that the null hypothesis is true and calculate the value of the statistic in that
data. Repeat the simulation process to generate a large number of could have happened
data sets, always assuming the null hypothesis to be true. Examine the “what if the null
hypothesis was true” distribution of these statistics.
3. Strength of evidence: Consider where the observed statistic falls in the “what if the
null was true” distribution. If the statistic falls in the far tail of the distribution, then we
have strong evidence against the null hypothesis. Otherwise, if the value of the observed
sample statistic is not in the tail of the “what if” distribution, then consider the null
hypothesis to be plausible.
For example, in our study the statistic we computed was 23 out of 33 students who
correctly matched Tim to his face. We simulated what “could have happened” if the null
were true (people correctly match Tim with his face half the time and incorrectly half the
time) using a coin flip (heads = correct match). We repeated this simulation several
times, each time counting the number of heads out of 33 tosses (number of correct
matches out of 33 just guessing students). The observed number of correct matches by
the students in our study, 23, was far enough in the tail of the “what if the null was true”
distribution to give us some evidence against the null hypothesis.
How far out in the tail of the distribution does the observed statistic need to be so that we
consider the result “beyond a reasonable doubt?”
Thought Question: Looking at the dotplot in Figure 1.6, how many correct matches
would you need to see in a group of 33 students to convince you that their outcome was
better than what you would expect just by random chance?
One way to quantify how unusual an outcome is in the “what if the null was true”
distribution is to calculate a p-value.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XII
Key Idea: The p-value is the probability that we would get a result as extreme as or
more extreme than the one that was actually observed, if the null hypothesis was true.
In our example, we found 23 or more correct matches occurred 11+ 5 + 1 = 17 times,
giving us an approximation to the p-value of 17/1000 = 0.017. [You can calculate this pvalue more exactly as you will learn in Chapter XX, but 1000 repetitions of our simulation
should give us a very reasonable estimate.] So this gives us a measure for how unusual
it is to get 23 correct matches if everyone in the class simply guessing.
FAQ: Why do we need to include more than when computing our p-value?
Student: So 23 out of 33 students correctly matched Tim’s name to his picture, why isn’t
the p-value the likelihood I would just get 23 heads out of 33 flips of a coin? Why do we
have to also include the likelihood of getting more than 23 heads?
Stat Prof: In flipping just 33 coins, the probability of getting 23 heads is very similar to the
probability of getting 23 or more heads, so I can see how this would be confusing. I
have an idea to help you understand this. Do you have any coins with you?
Student: I might, let me check. It looks like I have a few.
Stat Prof: Good. Do you think you can toss the coin “fairly”?
Student: I don’t see why not.
Stat Prof: Start flipping it. As you do this, keep track of the number of heads you get as
well as the total number of flips. Also think about whether or not your results indicate
whether this is an unfair tossing process.
Student: Okay, I flipped the coin 10 times and got 6 heads. Since this is close to 50-50, I
don’t think I could conclude that my coin tossing is unfair.
Stat Prof: Good. The probability of getting 6 heads out of 10 flips is about 0.20 while the
probability of getting 6 or more heads (our actual p-value) is about 0.37. Either way you
think of this, getting 6 heads out of 10 coin flips is not too unlikely. Let’s keep flipping.
Student: Okay, I’ve flipped it 50 times and …
Stat Prof: Keep flipping.
Student: I’ve flipped it now 100 times and…
Stat Prof: Keep flipping.
(1 hour later)
Student: I think my arm is going to fall off!
Stat Prof: I guess you can stop now. What are your results?
Student: I flipped the coin 1000 times and got 505 heads.
Stat Prof: Do you think your results show that your coin tossing is not fair?
Student: Of course not. Getting 505 heads out of 1000 flips is close enough to 50% that
the result is not unexpected.
Stat Prof: Do you mean you expected to get 505 heads?
Student: Well no, not exactly 505, but I did expect to get something close to 500.
Stat Prof: In fact, the probability of getting exactly 505 heads out of 1000 flips of a fair
coin is only about 0.02. Because there are so many different outcomes possible, the
probability of any one particular outcome is rather small. But we wouldn’t want to look a
the probability of .02 and consider this a surprising outcome, it is definitely among the
typical values. this is better conveyed by noting that the probability of getting 505 heads
or more in 1000 flips is about 0.39.
Student: And since this p-value is so high, I would not conclude my coin tossing is unfair!
Stat Prof: Exactly!
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XIII
Key Idea: The p-value for a test of significance is what we base our conclusion on. A
small p-value means it is unlikely the result in our data would have occurred simply by
random chance, assuming the null hypothesis is true.
What if only 18 out of 33 students had correctly identified Tim?
In the class we reported on 23 out of 33 students correctly identified Tim, but what if that
number had only been 18? In Figure 1.6 we see that 18 is a result that is typical if the
null hypothesis is true. When we compute the p-value we see that it is 0.340 and so we
would not consider this outcome unusual for a class that was simply guessing.
Because the p-value is not small, this says that if people really can’t correctly identify
Tim, it’s a fairly common occurrence to see a number like 18 correctly identify Tim
anyway. Thus, these “new” data (18 out of 33 students correctly identifying Tim) would
not provide enough evidence that the population of students can identify Tim. In this
scenario, it is plausible that the population cannot correctly identify Tim and that we just
happened to see a slight preference in our sample by random chance.
Key Idea: If the p-value for our test of significance turns out to be large, then our data do
not provide evidence that the observed outcome is something other than what we would
have expected to see by chance alone assuming the null hypothesis to be true. Note, we
never conclude that the null is true or even that we have evidence in favor of the null.
We can only say that we don’t have evidence against the null hypothesis and so the null
is plausible.
Again, the legal analogy applies, in a trial, a verdict in favor if the defendant is not
“innocent” but rather “not guilty.” We haven’t proven anyone innocent, only that the
evidence provided does not convince us, beyond a reasonable doubt, of their guilt.
Strength of Evidence
But still, how do we decide whether our p-value is a small number or not? How rare did
the outcome need to be in Figure 1.6 to convince you and your fellow classmates that
the result should not be attributed to random chance? How similar were your answers?
FAQ: What p-value should make us suspicious?
Stat Prof: Did you ever google “Persi Diaconis”?
Student Skeptic: No, why?
Stat Prof: Try it sometime. He’s unusual … even for a statistician he’s unusual. He was
one of the first people to win one of the MacArthur “genius” awards.
Student Skeptic: Is that all? What else can he do?
Stat Prof: Flip a coin and make it come up heads every single time.
SS: I don’t believe anyone can do that. Isn’t this whole dialog just a stat prof’s geeky
way to sneak in p-values?
SP: Sorry, I can’t help it. We stat profs are so transparent sometimes. That’s just the
data. But humor me: How many heads in a row would Persi have to get to make you
begin to think maybe it’s not just chance? Would heads on the first two tosses do it for
you?
SS: Of course not. That happens 25% of the time.
SP: You win a point for extra credit. What about heads on the first three tosses?
SS: You’re making me get serious about this. Three heads in a row wouldn’t happen
very often, but it’s still not unusual enough to be suspicious.
SP: What about four in a row? Five in a row?
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XIV
SS: Now it’s really getting serious … and suspicious. I find it hard to believe that you
can get five heads in a row just by chance. Sure, it can happen, but five in a row is
enough to make me think maybe there’s something else going on.
It turns out that four heads in a row happens about 6% of the time and 5 heads in a row
happens about 3% of the time. This is about when most people would start to think
there is something suspicious going on. And in fact, many studies consider 5% the cutoff value for what we would consider strong evidence against the null hypothesis. Of
course, in some situations you may want even stronger evidence. (A civil trial only
requires “preponderance of evidence” opposed to “beyond a reasonable doubt” in a
criminal trail.) Keep in mind that the smaller the p-value, the stronger the evidence
against the null hypothesis in favor of the alternative hypothesis.
Statisticians have agreed on the following standards for how they evaluate the strength
of evidence conveyed by a p-value.
Standards for judging p-values
0.10 < p-value
0.05 < p-value < 0.10
0.01 < p-value < 0.05
0.001 < p-value < 0.01
p-value < 0.001
not much evidence against null hypothesis
moderate evidence
strong evidence
very strong evidence
extremely strong evidence
When the statistic provides strong evidence against the null hypothesis in favor of the
alternative hypothesis, we often say the result is statistically significant, meaning unlikely
to happen by random chance alone.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XV
Activity 1.2: A Preference Study: Friend or Foe?
As adults we know the difference between naughty and nice, but what about pre-verbal
infants? A study in the November 2007 issue of Nature looked at children less than 1
year old to see whether they recognized and had a preference for nice versus naughty
toys.
In one component of the study, 10-month-old infants were shown a “climber” character
(a piece of wood with “google” eyes glued onto it) that could not make it up a hill in two
tries. Then they were alternately shown two scenarios for the climber’s next try, one
where the climber was pushed to the top of the hill by another character (“helper”) and
one where the climber was pushed back down the hill by another character (“hinderer”).
The infant was alternately shown these two scenarios several times. Then the child was
presented with both pieces of wood (the helper and the hinderer) and asked to pick one
to play with. The helper toy was chosen by 14 of the 16 children. We will consider these
16 infants (our sample) as representative of a larger population of 10-month-old infants
(our population).
To see a video of both the “helper” and “hinder” scenarios as well as to see infants
choosing the “helper,” visit the link below:
http://www.yale.edu/infantlab/socialevaluation/Helper-Hinderer.html
Keep in mind that the shape and colors of the helper and hinderer toys were changed
across the trials, so focus only on the helper vs. hinderer roles rather than on these other
aspects of shape and color.
Setting up the hypotheses
Remember, we said that the research conjecture becomes the alternative hypothesis
and the null hypothesis is then a related statement of “no effect” or “nothing special
going on.” The “dull” hypothesis.
1.
State the null and alternative hypotheses for this research question. (Keep in
mind there are a couple of different ways to state them, but be sure to state them
about the population.)
Examining the data
2.
Statistic: In the study, 14 of the 16 infants chose the helper toy. Convert this to
a proportion. Is this statistic in the direction of the alternative hypothesis? That is,
does it provide initial support for the research conjecture?
Strength of Evidence
Although our data are supportive of the alternative hypothesis, a result as extreme as 14
could have arisen even if the infants were each choosing equally between the helper
and the hinderer. So we again need to consider how unusual such a result would be by
random chance alone.
If the null hypothesis is true, then infants will equally choose either the helper or the
hinderer toy and so each infant’s choice is really just like flipping a coin. Remember that
tests of significance are like the criminal justice system in that we will be making an initial
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XVI
assumption (the null hypothesis is true). So let’s assume, for the time being, that infants
really don’t prefer the helper toy any more than the hinderer.
3.
Simulate: Let’s use coin flipping to simulate “could have happened” results
under this null hypothesis.
a.
Start by simulating a single infant randomly (with equal
probability) choosing the helper or hinderer toy. Explain how you will use a coin to do
this and what heads and tails represent.
b.
Now, simulate 16 infants randomly choosing the helper
or hinderer toy. Explain how you will use a coin to do this, what heads and tails
represent, and how you will obtain the statistic from your simulation.
4.
Pool your results with your classmates to create the “what if the null was true”
distribution for the number of infants choosing the helper toy. Based on these
results, does 14 out of 16 infants choosing the helper toy when the null hypothesis is
true and they have no genuine preference between the toys (or 14 heads in 16 coin
tosses) appear to be an unlikely occurrence?
It’s hard to get a real secure feeling about your conclusion having only replicated the
study a few times. It would also be quite time consuming if we had to do too many more
of these repetitions flipping a coin. We will use the power of the computer to repeat this
simulation of 16 children’s toy choices under the null hypothesis. The more times we
repeat the simulation the more accurately we will be able to simulate the p-value for this
study.
5.
Let’s use the “Coin Tossing” web applet to generate more repetitions.
a.
First, use the applet to simulate 16 coin flips. Report
the statistic you obtained and explain how the simulation relates to the infants
choice of helper or hinderer.
b.
Now change the Number of Repetitions to 20 and press the
“16 Tosses” button to get a sense for how this statistic varies from trial to trial.
c.
Next change the number of repetitions to 979 and
press the “16 Tosses” button to create a total of 1000 replications.
d.
Use the applet to count how many of the dots (sets of
16 tosses) had 14 or more heads by entering 14 in the “As extreme as” box and
press Count. The proportion of repetitions reported by the applet is your
approximation of the p-value – how often, when the null hypothesis is true, do we
get a statistic that is at least as extreme as what the researchers found in the
actual study.
6. Press the Exact p-value button and the applet will calculate the p-value exactly
using probability rules (Think of repeating the above simulating infinitely many times).
How does this value compare to your estimation from the 1000 repetitions?
7. Interpret this p-value in the context of this study. Explain what it measures.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XVII
8. Would you consider this p-value strong evidence against the null hypothesis?
Explain.
Let’s recap what you have done. We suspect you found that the proportion of repetitions
with 14 or more heads was very small. The exact p-value equals 0.002. This means that
if infants in general are just randomly choosing toys and you did this same study with 16
impartial infants 1000 times, only about 2 out of 1000 times will you get a number as
large as or larger than 14 (or 87.5%) infants choosing the helper toy as observed in the
actual study. This provides very strong evidence against the null hypothesis that infants
in the population do not have a genuine preference between the two toys.
Warning: Keep in mind that we might be making an error here. It’s possible that this
study could be one of those 2 in 1000 studies that would produce such an extreme
sample even if the null hypothesis was true. However, the chance of that is sufficiently
small, that we feel more comfortable concluding something else is going on.
Now, let’s step back a bit further and think about the implications of the fact that these 16
infants seem to be demonstrating a preference for the helper toy. Do you think this
means that all ten-month-old infants can tell the difference between naughty and nice?
An important question here is how representative these 16 infants are of all infants.
We’ll discuss this more in Chapter 2. We would also probably need to do some more
sophisticated testing to make the leap from toy preference to knowledge of difference
between naughty and nice.
9. Use the applet to investigate how many infants would need to choose the helper toy
to convince you that their result is more extreme than you would reasonably expect
from chance alone.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XVIII
Section 1.3: Statistical Significance for One Proportion—Other
Null Hypotheses
In the previous section, we discussed testing whether or not a null hypothesis of
“choosing equally between two choices” was believable based on some observed
sample data. But there are situations where we may wish to consider proportions other
than .5.
Example: Appetizing Dog Food?
Does canned dog food make for a suitable and inexpensive alternative to blended meat
products such as spam or liverwurst? As unappetizing as this might sound, researchers
investigated this question in a study that was described in the February 20, 2009 issue
of Science magazine. The researchers presented 18 subjects with 5 unlabeled blended
meat products. One of these products was Newman’s Own dog food, prepared with a
food processor to have the texture and appearance of a liver mousse. After tasting all 5
products, subjects were asked to identify which was their least favorite on the basis of
taste. The research question was whether subjects are more likely to identify dog food
as their least favorite than if they were making selections at random.
Statistic: It turned out that 13 of the 18 subjects (≈72%) identified the dog food as their
least favorite.
This study has much in common with the infants’ toy study: We have 18 subjects instead
of 16 infants, and researchers keep track of which selection each subject makes. Also,
the research question is whether a certain selection is made so often that random
chance can be eliminated as a plausible explanation for that tendency. We’ll analyze
these studies in the same way: We’ll simulate the subjects’ selections assuming a null
hypothesis that selections are made by random chance alone. We’ll repeat this
simulation a large number of times, and we’ll see how often we obtain a result as (or
more) extreme as the actual result from the subjects in the real study. But there’s one
important difference between these studies, which affects how we’ll conduct the
simulation.
Thought Questions
• What is the important difference? Hint: Would it make sense to use a fair coin to
simulate the subjects’ selection process?
• If they are picking among the five foods completely at random, roughly how many of
the 18 subjects would you expect to pick dog food as their least favorite?
Hopefully it is obvious that it would not make sense to use a fair coin here, because
there are five options rather than two available to each subject (and then we will record
whether or not they pick dog food as the least favorite). In order to simulate subjects’
choices of their least favorite product as random selections, we need to give dog food a
1 in 5 chance, rather than a 1 in 2 chance, of being selected. But this is the only change
we need to make to our analysis strategy; the rest will proceed exactly as with the
infants’ toy study.
So our hypotheses will be:
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XIX
Null hypothesis: Subjects in the population pick equally among the 5 foods (the
probability dog food is picked as the least favorite equals 0.20)
Alternative hypothesis: Subjects in the population are more likely to choose the
dog food as their least favorite. (The probability dog food is picked as the least
favorite is larger than 0.20.)
Simulate: To simulate a process with a 0.2 probability, we could use a computer or a
calculator. The results of 1000 sets of such random selections are shown in Figure 1.7.
Figure 1.7: The “what if the null was true” distribution of the number of 18
subjects randomly choosing dog food as their least favorite
As expected under the null hypothesis, the center of this distribution is close to (1/5)×18
= 3.6. Most of the trials in this simulation saw 2, 3, 4 or 5 people picking the dog food as
their least favorite.
Strength of Evidence: In particular, when the null hypothesis is true we never saw a
result as extreme as 13 of the subjects just randomly picking the dog food when subjects
are choosing equally among the five foods. (The exact p-value can be shown to equal
0.0000025, or less than 3 in a million or less often than tossing 18 heads in a row.)
Thought Question
What conclusion would you draw from this very small p-value?
This small p-value suggests that the actual result of the study would be extremely
unlikely to occur if in fact the selections were made purely at random (picking equally
among the five food choices). Therefore, this study provides extremely strong evidence
that dog food really is chosen as least favorite more than 20% of the time.
Follow-up:
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XX
The 18 subjects in this study were also asked to guess which of the 5 products was dog
food. Interestingly, only 3 of the 18 subjects correctly identified which product was dog
food.
Thought Question
Explain why it’s not even necessary to conduct a simulation analysis to investigate
whether this provides strong evidence that people can correctly identify which is dog
food more than random chance.
In our 3S strategy, the first S: statistic involves looking at the sample data. One thing to
consider is whether the data are in the direction conjectured by the alternative
hypothesis. In this case, 3 out of 18 or about 16.7% of the subjects correctly identified
the dog food. Since this is less than 20%, such data will certainly not convince us that
people are more likely to correctly identify the dog food than if they were simply
guessing.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXI
Activity 1.3: Discriminating Colas
Do you think that, overall, the cola drinkers in your class can tell the difference between
two similar colas? Let’s find an answer to this question by running a taste test. Students
who are cola drinkers will be given three plain cups. Two contain the same brand of cola
and one contains a different brand of cola. They will then taste from all three cups and
attempt to correctly determine which one is different from the others. Students who
aren’t cola drinkers can set up and run the taste tests. Record the results to see how
many of the students involved in the taste test correctly identify which cup of cola is
different from the other two. We want to know if the proportion of students that can
correctly identify the cola that is greater than if they were just guessing (making their
decision by random chance). This is similar to the dog food example since we have
more than two outcomes from which to choose, so we won’t be able to use a fair coin to
simulate the student’s choice. (But will classify the response as “correct” or “not.)
1. If we assume that, overall, the class can’t tell the difference between the two colas,
or each individual is just randomly guessing as to which of the three cups is different,
what is the chance an individual will correctly choose the cup that is different? How
many correct identifications do you think you would get on average in your sample if
students are really just guessing?
2. What are your null and alternative hypotheses?
3. Statistic: Let’s examine the data we collected from our taste test.
a) How many students correctly identified which of the three cups was different?
b) What proportion of students correctly identified which of the three cups was
different?
c) Do you suspect that the evidence is strong enough to convince someone that the
alternative hypothesis is true? Why or why not? What other information would
you like to have in order to decide how convincing the evidence is?
Simulate: Let’s use an applet to simulate 1000 repetitions of the class taste tests, where
we assume that students are just guessing which cup is different (probability of 0.3333
as to whether a student correctly identify the different cola).
Go to the “One Proportion Inference” applet which lets you specify the conjectured
population proportion that can correctly distinguish the different cup (as specified by the
null hypothesis) as well as the number of observations in each trial (e.g., the number of
subjects in the study).
4. What value are you entering for the null value? What value are you entering for the
number of tosses?
5. Press the “Randomize” button to generate one version of the results that “could have
happened” if the null hypothesis was true. One dot is added to the dotplot. Clearly
explain what this dot represents.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXII
6. Now change the Number of Repetitions box to 999 in order to generate a total of
1000 repetitions. Describe the resulting “what if the null is true” distribution.
7. How many students correctly identified the different cola in the class results? Enter
this value in the “As extreme as” box and press Count. What proportion of times out
of 1000 did you simulate a number as large or larger than the actual result of the
study?
8. What is your proportion from question 8 called?
9. Strength of evidence: What do you conclude about the strength of evidence that
the cola tasters are able to tell the difference between the two colas better than
random guessing based on our results? Why?
10. Is there a larger population to which you can infer your results?
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXIII
Section 1.4: Plausible Values
In this section, we continue to apply the statistical process to explore a population
proportion in more detail. Often our goal is less about examining one particular
conjectured value for the parameter, but rather to identify a range of plausible values for
the parameter.
Example: Competitive advantage of red uniforms?
Do athletes who wear red uniforms have an advantage over their competitors? To
investigate this question, Hill and Barton (Nature, 2005) examined the records in the
2004 Olympic Games for four combat sports: boxing, tae kwon do, Greco-Roman
wrestling, and freestyle wrestling. Competitors in these sports were randomly assigned
to wear either a red or a blue uniform. The competitor wearing red defeated the
competitor wearing blue in 248 matches, and the competitor wearing blue emerged as
the winner in 209 matches.
1. Statistic: The proportion of matches won by the competitor wearing red is 248 / (248
+ 209) = 248 / 457 ≈ .543.
The question is: Do these data provide strong evidence that competitors wearing red
really do win more than half the time? To investigate this question we ask how unlikely it
would be to obtain 248 or more winners wearing red in 457 matches, if the winner is
really equally likely to be either competitor. In other words, we ask how often a sample
proportion of .543 or higher would result if in fact the population proportion were .5. The
null hypothesis is that 50% of matches are won by the competitor wearing red, and the
alternative hypothesis is that more than 50% of matches are won by the competitor
wearing red.
2. Simulate: We have seen in previous sections that we can use simulation to evaluate
the strength of evidence in favor of the alternative hypothesis. First, we will model the
outcome of the game with a coin flip and we simulate some data that “could have
happened” had the null hypothesis been true. To do this, we flip a coin 457 times,
representing the matches in these sports, and we count how many heads appear,
representing wins by the competitor wearing red. We then repeat this process many,
many times generating the “what if the null hypothesis was true” distribution. We used a
computer applet to simulate 1000 repetitions of flipping a coin 457 times. The results
are shown in Figure 1.8.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXIV
Figure 1.8: A “what if the null is true” distribution showing the number of heads in
1000 repetitions of a coin being flipped 457 times. Note that 248 is in the upper
tail and thus an unlikely event if the red competitor wins 50% of the time.
3. Strength of Evidence: We see that the outcome 248, which is the sample number of
matches won by the competitor wearing red, is in the upper tail of the distribution. In
fact, only 35 of the 1000 simulated samples produced 248 or more heads, so the
simulated p-value is .035. This is small enough to conclude that the sample data
provide fairly strong evidence that the competitor wearing red really does have a better
than 50% chance of winning the match.
Now what? We have reason to believe that the competitor wearing red wins more than
half the time, so a natural question to ask next is: How much more than half? In other
words, we have concluded that .5 is not a plausible value for the underlying probability
that the competitor wearing red wins a match, but what do we think are plausible values
for that underlying probability? To investigate this, we can perform a similar test using a
different potential value for this probability in the null hypothesis.
Let’s start by testing the null hypothesis that the competitor wearing red has a .55
probability of winning a match. We’ll simulate 1000 sets of 457 coin tosses with a .55
probability of landing heads. The results are shown in Figure 1.9.
Figure 1.9: When the probability that a competitor wearing red wins a match is .55,
we find that 248 out of 457 matches won by the red competitor is not out in the
tail, but lies fairly close to the center of this “what if the null is true” distribution.
In this case we see that the observed data value (248) is not in the tail of the distribution.
In fact, 39.5% of the above repetitions had a smaller number of successes (red
competitor wins) than 248. So we don’t have evidence against the null hypothesis that
the probability of winning while wearing red is .55.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXV
Notice, that although any number between 0 and 1 is a possible value for the underlying
probability that the competitor wearing red wins the match, our analyses above have
revealed that 0.5 does not seem to be a plausible (or believable) value in light of the
sample data, whereas 0.55 do appear to be plausible based on the sample data. This is
not surprising, considering that the observed sample proportion of matches won by the
red-wearing competitor is .543, much closer to .55 than to .5. But we still don’t know the
true probability of winning while wearing red. Using the scientific method, let’s rehypothesize. Maybe we could close in on the true probability by finding an upper bound.
Let’s next test the null hypothesis that the competitor wearing red has 53% chance of
winning a match. The simulation results shown in Figure 1.10 reveal that an outcome of
248 is again not a surprising outcome in this “what if the null was true” distribution.
Figure 1.10: When the probability that a red-wearing competitor wins the match is
0.53, we find that 248 out of 457 matches won by the person wearing red among
the typical values of our “what if the null is true” distribution.
So both .53 and .55 are considered plausible values for the probability of the red
competitor winning.
Key Idea: Keep in mind that not gathering enough evidence to support the alternative
hypothesis is not the same thing as concluding the null hypothesis is true. There
are almost always other null hypotheses that would also be plausible.
Let’s next test a 60% chance of winning the match. The simulation result shown in
Figure 1.11 reveals that an outcome of 248 wins (or less, looking in the more extreme
tail) by the red competitor would be very surprising if this null hypothesis were true. The
simulated p-value is .006, so we have very strong evidence that the actual probability of
winning while wearing red is less than .6.
Figure 1.11: When the probability that a red-wearing competitor wins the match is
0.6, we find that 248 out of 457 matches won by the person wearing red is out in
the lower tail of our “what if the null is true” distribution.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXVI
We can continue to use this analysis strategy to investigate many different null
hypotheses, each using a different value for the probability that the red-wearing
competitor wins the match. We’ve already tested whether .5, .53, .55, or .6 are plausible
values for this probability, and we’ve concluded so far that the probability is above .5 but
less than .6. We also found that .55 is plausible, because the observed data are not in
either tail of its “what if null were true” distribution. There must be other values between
.5 and .6 that would not be rejected and thus can also be considered plausible values for
the true probability. Let’s go ahead and test values .51, .52, .53, and so on.
We used the applet to test all of these null hypotheses, with 1000 repetitions for each
analysis. Table 1.1 shows the strength of evidence for each of these hypothesized
values.
Table 1.1: p-values when testing various null hypothesis probabilities that the
competitor wearing red wins the match, based on the sample result of 248 wins in
457 matches
Simulated
p-value
Strength of evidence against null
value
0.5
0.04
Strong
0.51
0.073
Moderate
0.52
0.17
Not much
0.53
0.319
Not much
0.54
0.469
Not much
0.55
0.375
Not much
0.56
0.253
Not much
0.57
0.109
Not much
0.58
0.044
Strong
0.59
0.021
Strong
0.6
0.008
Very strong
Null value
Note: In this table we have always computed the p-value by looking in the tail of the
“what if the null hypothesis was true” distribution. This means that when the null
hypothesis value is less than 0.645 we look in the upper tail and when the null
hypothesis value is greater than 0.645 we look in the lower tail. In Chapter 2 we
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXVII
introduce the idea of computing our p-values using a “two-sided” testing approach where
we measure strength of evidence based on both tails of the distribution.
This table illustrates that there are many values for which the null hypothesis value is
plausible. Specifically, for values from .52 to .57, the observed data do not provide much
evidence against the null hypothesis, so we can say that it’s plausible that the actual
probability that the red competitor wins a match is between .52 and .57. This suggests a
very slight advantage for the competitor wearing red. Notice that these endpoints (.52
and .57) are almost equally far from the observed sample proportion (.543).
We could refine this analysis further by considering more possible values. For instance,
we could consider values from .510 to .530 and from .560 to .580 in order to “zero in”
more precisely on reasonable endpoints. We could also modify this procedure by
deciding that we want to use a more or less stringent rule for evidence against the null
hypothesis, for example by considering a value to be plausible unless the simulation
produces very strong evidence against it.
Key Idea: We consider a value plausible for the population parameter if a simulation
analysis based on that value does not put the observed data in the tail of the “what if null
were true” distribution. This analysis produces an interval of plausible values for a
population parameter based on the observed sample data. This interval of all such
plausible values is called a confidence interval.
We will further explore confidence intervals in Chapters 5, 6, 7, and 8.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXVIII
Activity 1.4: Kissing Right?
Most people are right-handed and even the right eye is dominant for most people.
Molecular biologists have suggested that late-stage human embryos tend to turn their
heads to the right. In a study reported in Nature (2003), German bio-psychologist Onur
Güntürkün conjectured that this tendency to turn to the right manifests itself in other
ways as well, so he studied kissing couples to see if they tended to lean their heads to
the right while kissing. He and his researchers observed couples in public places such
as airports, train stations, beaches, and parks. They were careful not to include couples
who were holding objects such as luggage that might have affected which direction they
turned. For each couple observed, the researchers noted whether the couple leaned
their heads to the right or to the left. They observed 124 kissing couples, finding that 80
leaned to the right.
The question is: Do these data provide strong evidence that kissing couples really do
lean to the right a majority of the time? To investigate this question we ask how unlikely
it would be to obtain 80 or more couples leaning to the right in a random sample of 124
couples, if the couples are equally likely to lean right or left.
1. Statistic: Calculate the proportion of the observed couples who leaned to the right.
2. State the null and alternative hypotheses for this study.
This time we don’t want to ask you to take the time to actually do a lot of coin flipping,
but we do want you to think about how you would conduct this simulation analysis if you
had to.
3. Simulate: Describe how you would use a coin to conduct a simulation analysis to
determine whether these data provide strong evidence that in general kissing
couples really do tend (more often than not) to lean to the right. Provide sufficient
details that someone else could implement the analysis based solely on your
description. You should include statements about how you would make a conclusion
based on the simulation results.
4. Use technology (one proportion inference applet) to simulate 1000 repetitions of 124
couples, assuming the null hypothesis that couples are equally likely to lean right or
left is true. Where is the distribution centered? Also report your simulated p-value.
Would you consider the result of 80 or more out of 124 unlikely assuming the null
hypothesis is true?
5. Strength of evidence: Based on your simulation results, would you conclude that
the researchers’ data (80 of 124 couples leaning to the right) provides strong
evidence that couples in general really do tend to the lean to the right more often
than the left? Explain the reasoning behind your answer.
Now suppose the researchers believed that couples are more likely to lean to the right
when kissing. In fact, they believe this happens about 2/3 of the time, as that is similar to
other right-sided tendencies that have been observed (right handedness, right eye-ness,
embroyos turning to the right in the womb and so on).
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXIX
6. Carry out a test of significance to decide whether you have convincing evidence that
couples turn to the right less than 2/3 of the time. Make sure you state the null and
alternative hypotheses, explain how you generate the simulation, and report the
empirical p-value.
7. What do you conclude about whether 2/3 (roughly .667) is a plausible value for the
probability a couple leans to the right?
8. Carry out a test of significance to decide whether you have convincing evidence that
couples lean to the right less than 75% of the time. Make sure you state the null and
alternative hypotheses, explain how you generate the simulation, and report your
simulated p-value.
9. What do you conclude about whether .75 is a plausible value for the probability a
couple leans to the right?
Confidence Interval: We can continue to use our analysis strategy to investigate many
different null hypotheses, each stipulating a different value for the probability that a
couple leans to the right. We’ve already tested whether 0.5, 0.67, or 0.75 are a plausible
value. These analyses rejected the null hypothesis in two cases and we concluded that
the proportion of all couples that lean to the right when kissing is more than 0.5 and less
than 0.75. So, there must be values in between 0.50 and 0.75 that would not be
rejected and thus be plausible values for the true proportion of couples that lean to the
right when kissing. Let’s go ahead and test values such as 0.51, 0.52, 0.53, and so on.
Granted, this will get tedious, but with technology it’s not too cumbersome.
10. Use the applet to test all of these null hypotheses, starting with values 0.51, 0.52,
0.53, and continuing up to and including 0.74 for the probability that a kissing couple
leans to the right. Use 1000 repetitions for each analysis. Report the hypothesized
values which do not provide strong evidence (p-value > .05) against the null
hypothesis. [Hint: For now, always put the observed result in the “tail” of the
distribution, so you may need to switch the count inequality in the applet.]
11. Where does the observed sample proportion (.645) lie compared to this interval of
plausible value?
12. Why does it make sense that you received a large p-value for your test of the null
hypothesis that 65% of couples lean right? Does this mean that you’ve proven that
65% f kissing couples lean right? Why or why not?
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXX
Section 1.5: Role of Sample Size
In this section we explore the important role of sample size in considering statistical
significance.
Example: Predicting Elections from Faces?
Do voters make judgments about political candidates based on how their faces look?
Can you correctly predict the outcome of an election, more often than not, simply by
choosing the candidate whose face is judged to look more competent? Researchers
investigated this question in a study published in Science (Todorov, Mandisodka, Goren,
and Hall, 2005). Participants were shown pictures of two candidates and asked which
one looked more competent. Researchers then predicted the winner to be the one
whose face was judged to look more competent by most of the participants.
For the 32 U.S. Senate races in 2004, this method predicted the winner correctly in 23 of
them. This “competent face” method therefore succeeded in 23/32 ≈ 0.719, or 71.9% of
the 32 races. Is this significantly higher than 50%? We can use the 3S method to
investigate: Assume that the “competent face” method is no better than flipping a coin,
and simulate the set of 32 races, over and over, for a total of 1,000 times. How often do
you get 23 or more correct predictions? Our results are shown in Figure 1.12.
Figure 1.12: A distribution of the number of correct predictions in 32 races if the
“competent face” method works half the time. In our sample of 32 races we found
the “competent face” method to work 23 times.
number of correct predictions in 32 races (under null hypothesis) Only 9 of these 0,000 simulated sets of 32 races show 23 or more correct predictions, so
the simulated p-value is 0.009. This p-value is small enough to provide strong evidence
against the null hypothesis, in favor of concluding that the “competent face” method
makes the correct prediction more than half the time.
These researchers also predicted the outcomes of 279 races for the U.S. House of
Representatives in 2004. [While all 435 House seats were up for reelection in 2004, the
researchers were only able to obtain pictures for the winter and runner up in 279 of
these races.] The “competent face” method correctly predicted the winner in 189 of
those races, which is a proportion of 189/279 ≈ 0.677, or 67.7% of the 279 House races.
Notice that this percentage is similar but a bit smaller than the 71.9% of correct
predictions in 32 Senate races.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXXI
Thought Question: Do you expect the strength of evidence for the “competent face”
method to be stronger for the House results, weaker for the House results, or essentially
the same for the House results as compared to the Senate results?
Let’s investigate this key question with a simulation of 1,000 repetitions of 279 races,
assuming that the “competent face” method does no better than a coin flip. (See Figure
1.13.)
Figure 1.13: A distribution of the number of correct predictions in 279 races if the
“competent face” method worked half the time. In our sample of 279 races we
found the “competent face” method to work 189 times.
number of correct predictions in 279 races (under null hypothesis) Notice that none of these 1,000 repetitions produced 179 or more correct predictions. In
fact, none came close to that many correct predictions. Let’s try 10,000 repetitions.
Figure 1.14: A larger distribution of the number of correct predictions in 279 races
if the “competent face” method worked half the time. In our sample of 279 races
we found the “competent face” method to work 189 times.
number of correct predictions in 279 races (under null hypothesis) Notice this “fills in” and “smooths out” the distribution a bit, but we still did not receive a
single simulated study that had as many as 189 correct predictions. So, the simulated pvalue from these data is extremely close to zero (less than .0001). These data provide
overwhelmingly strong evidence that the “competent face” method makes a correct
prediction more than half the time.
This analysis reveals that sample size plays a very important role in assessing statistical
significance. Even with a slightly smaller sample proportion of correct predictions, the
data from the 279 House races provide much stronger evidence that the “competent
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXXII
face” method works better than a coin flip, as compared to the data from the 32 Senate
races.
Key idea: Whenever you need to decide whether a sample proportion is strong
evidence against the null hypothesis, be sure to take into account the sample size of the
study. If two studies have the same value of the statistic, a larger sample provides
stronger evidence against a null hypothesis than a smaller sample does.
Also keep in mind that the “number of repetitions” in the simulation gives us a more
accurate estimate of the p-value but does not necessarily make the p-value smaller like
a larger sample size does.
Frequently Asked Question: More extreme versus bigger sample?
House Member: “My big House has a lot more data than your little Senate. So
I’m betting the House evidence is stronger, which means the House p-value will
be closer to 0 than your Senate p-value.”
Senator: “Big House indeed. Your percentage is only 67.7%. Mine is 71.9%,
which is more extreme – farther from 50/50 – so my evidence is stronger.”
Houser: “Hold on! Your sample is so small that there’s a lot of shakiness in your
percentage. It could easily have been lower. My percentage is much more solid,
because it’s based on a lot more data.”
The debate between the Houser and the Senator illustrates that the strength of
evidence, as captured by the p-value, is really driven by two components. The first
component is the distance the proportion is from the null hypothesis proportion, and the
other is the sample size. Both components contribute to how strong the evidence is
against the null hypothesis, and the p-value takes both into account. In later chapters,
we will see how these components are directly associated with the statistical power, a
measurement of how likely the study is to find evidence in favor of the alternative
hypothesis.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXXIII
Activity 1.5: Baseball Big Bang
A reader wrote in to the “Ask Marilyn” column in Parade magazine to say that his
grandfather told him that in 3/4 of all baseball games, the winning team scores more
runs in one inning than the losing team scores in the entire game. (This phenomenon is
known as a “big bang.”) Marilyn responded that this probability seemed to be too high to
be believable.
1. Identify the relevant parameter in this study.
2. State the grandfather’s claim and Marilyn’s response, in terms of this parameter.
Also identify which is the null hypothesis and which is the alternative.
3. Suppose that you take a sample of Major League Baseball (MLB) games and find
that half of them contain a big bang. What more do you need to know before you
can test whether this provides strong evidence against the grandfather’s claim?
4. Suppose that Jose and Maria both take a sample of MLB games and keep track of
which contain a big bang and which do not. Suppose that Jose finds that half of his
sample of 10 games contains a big bang, and Maria finds that half of her sample of
50 games has a big bang. Which person do you expect to have the smaller p-value,
and therefore stronger evidence against the grandfather’s claim and in favor of
Marilyn’s alternative?
5. Use the One Proportion Inference applet to conduct a simulation analysis to simulate
the p-value for both Jose and Maria’s sample data. You will need to toggle the
greater than sign to less than for this test. Report the simulated p-value, and
summarize the strength of evidence, in each case. Also comment on whether your
analysis supports or refutes your answer to question 4.
6. Collect your own data on a sample of baseball games. Use a sample size of at least
20, but preferably much larger, and try to make your sample as representative of the
population as possible. Feel free to use Major League or minor league or college or
high school or Little League games. For each game take note of whether the game
contains a big bang or not. Describe how you collect your sample data, and report
your sample proportion of games with a big bang.
7. Continuing from question 6, conduct a simulation analysis to investigate how much
evidence your sample provides against the grandfather’s claim in favor of Marilyn’s
alternative. Submit a graph of the “what if the null is true” distribution, and report the
simulated p-value. Write a paragraph summarizing your conclusions and explaining
the reasoning process by which they follow from your analysis.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXXIV
Chapter Summary
Basing decisions on data is better than relying on anecdotal information. The study of
statistics provides us with a systematic procedure for gathering data (evidence),
evaluating that evidence, suggesting conclusions based on that evidence, and assessing
our confidence in those conclusions.
Descriptive statistics involves summarizing information from data to detect patterns and
tendencies. Numbers that describe a sample are called statistics, whereas numbers that
describe a population are called parameters. In practice, we almost never know the
values of parameters, and so we use sample statistics to learn about population
parameters. This process is called inferential statistics.
Statistical investigations make use of the scientific method: Ask research question, form a
hypothesis, collect data, analyze results, draw conclusions, communicate results, repeat.
The research conjecture is stated as the alternative hypothesis, to be compared against
the null hypothesis that is often a statement of no effect or equality (the “dull”
hypothesis). As with the presumption of innocence in the criminal justice system, a test
of significance begins with the assumption that the null hypothesis is true. After
gathering and examining sample data, we conduct a test of significance to assess the
strength of evidence that the data provide against the null hypothesis in favor of the
alternative hypothesis. To assess the strength of evidence, we outlined a three step
process that we call the 3S strategy:
1. Statistic: Calculate the value of a statistic from the sample data.
2. Simulate: Use simulation to hypothetically replicate the data collection process many
times, but under the assumption that the null hypothesis is true. Examine the “what if
null were true” distribution to see what are typical and unusual values of the statistic
when the null hypothesis is true.
3. Strength of evidence: If the observed data value falls in the tail of the simulated “what
if null were true” distribution, then the sample data provide strong evidence against
the null hypothesis in favor of the alternative hypothesis.
The key component of this strategy is the p-value, which is the probability of obtaining a
result as extreme as or more extreme than the one from the sample data, assuming the
null hypothesis were true. Small p-values provide us with strong evidence against the
null hypothesis, as they indicate our observed data would be unlikely to occur by chance
alone if the null hypothesis is true. Large p-values don’t provide us with enough evidence
to reject the null in favor of the alternative, so we consider the null hypothesis to be
plausible (but not proven). Also keep in mind the distinction between a possible value for
the parameter and a plausible or believable value for the parameter.
We can use repeated test of significance under different null hypothesis values to
determine an interval of plausible values for a population parameter. If we get a large pvalue and thus can’t reject the null, the parameter under the null is considered to be
plausible. This interval of plausible values is called a confidence interval.
Sample size plays a large role in assessing statistical significance. If the value of a
statistic is the same with two samples, then a larger sample provides stronger evidence
against the null hypothesis than a smaller sample.
So far, you considered how to test values for a population proportion, but the same logic
will apply to other parameter values as well.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXXV
Case Study: Will skipping breakfast lead to a world without men?
In a recent study that made headlines around the world, researchers at Oxford
University (Mathews et al. 2008) explored whether women who had high food
consumption at the time of conception were more like to have boys. Two hundred and
forty-one first-time mothers who were classified as having high-food consumption at the
time of pregnancy were followed for nine months until the gender of their child was
identified.
1. Before learning about how the study turned out, specify the null and alternative
hypotheses you will use to test the researchers’ conjecture that the women who had
high food consumption were more likely to have boys than girls. Since all 241 women
were first time mothers in England, it is important to know that, overall, the proportion
of male babies born to all first time mothers was England is 51.2%.
2. Statistic: So, how did the study turn out? In 135 of the 241 pregnancies, a boy was
born. What proportion of the births to high-food consumption mothers resulted in a
boy being born?
3. Simulate: How can you use the coin flipping approach to test the hypotheses?
Specifically describe the process you would use (remember, you can’t use a 50/50
coin) to carry out the simulation, including how to calculate the p-value and make a
conclusion. Provide sufficient details that someone else could implement the analysis
based solely on your description. Your answer should not simply explain how to use
the applet.
4. Use technology to carry out the simulation. Produce a rough sketch of the number of
correctly predicted outcomes assuming the null hypothesis is true, and indicate
where the result observed by the researchers falls in that distribution. Also report the
p-value.
5. Strength of evidence: Based on your p-value, state your conclusions about whether
high-food consuming mothers were more likely to have boys.
6. Using non-statistical language, explain the process you conducted in order to arrive
at your conclusion.
7. These researchers also explored whether the likelihood of having boys amongst the
lowest food consuming mothers was less than expected. Write the null and
alternative hypothesis you wish to test on this data.
8. One hundred eight of the 240 lowest food consuming mothers had boys. Use
technology to analyze these data with the 3-S analysis strategy. Produce a rough
sketch of the null distribution, and indicate where the observed research result falls in
that distribution. Also report the p-value. Summarize your conclusions.
9. Compare the two p-values you calculated, remembering that the p-value is a
measure of the strength of evidence. Explain why, based on the data, it makes sense
that one of the p-values is smaller than the other.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXXVI
10. In another analysis presented in their paper, the researchers tried to pinpoint specific
foods consumed by the mothers associated with the gender of the child. Of 300
women in the study who consumed at least one bowl of cereal per day during the
period of conception, 181 had boys. State hypotheses, analyze the data with the 3S
strategy, report the p-value and summarize your conclusions.
11. Do you think that this analysis proves that higher food and/or breakfast cereal
consumption is causing women to have male babies, or are other explanations
possible? Why or why not?
12. Perhaps she ate a lot of breakfast cereal, but in the early 1900s Annie Grace
Buckland Jones, married to Grover C. Jones Sr., of Peterstown, West Virginia had
15 children---all boys! The Jones’ large family size and the 15 consecutive male
children raised quite a stir at the New York World fair in 1940, where they were
invited to be guests of then President Franklin Delano Roosevelt and were featured
nationally on the radio. Of course, there were some naysayers who couldn’t believe
this to be possible, they must have just been faking it!
a) Using simulation, estimate the probability of getting 15 boys in a row. Make sure
you describe how you’ve carried out your simulation (including any simplifying
assumptions you have made) and estimated the probability.
b) Do you think that this proves that the Jones’ must have been faking it?
c) What are the chances that after 15 boy babies in a row, the next baby (16th) born
to the Jones’ will be a boy? It was!
d) What are the chances that, after having had 16 children, their next baby would be
a girl? It was!
13. We would be remiss in telling you the story of the Jones’ if we did not mention
something that made the Jones’ even more famous than having 16 boys in a row.
While playing horseshoes with one of his sons, “Punch,” Grover C. Jones and his
son found a bluish rock which they believed to be quartz. They placed it in a cigar
box, where it stayed for 14 years while the Joneses struggled through the Great
Depression. Later, in 1942, Grover brought the rock to a geology professor at a
nearby University, who determined that it was a 34.48 carat diamond—the largest
alluvial diamond ever found in the United States. What are the chances of that?
Don’t worry, you don’t need to calculate that value.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXXVII
Research Article: Stock Monkey
Have you ever heard it said that taking your stock broker’s investment advice is no
different than if a monkey had chosen your stocks? This popular notion was first
introduced by Prof. Burton Malkiel’s in his book A Random Walk down Wall Street.
Since that time a number of serious and not-so-serious studies have taken place that
have placed real monkey’s up against experts to “test” the theory.
Read the article from “The Daily Princetonian” which investigates the results of an
ongoing “game” published in the Wall Street Journal. Answer the remaining questions.
1. Did the experts or randomly chosen stocks perform better in the WSJ’s game? What
measure of stock performance do the authors use to make their point?
2. What is Malkiel’s reason for why the expert picks did better? What justification does
he give for being right?
3. Explain intuitively why Malkiel’s “towel” approach of buying multiple stocks makes
sense.
4. How does Malkiel argue that the price of a stock is determined?
5. Why do some people argue that the collapse of Enron means Malkiel’s theory is not
true?
6. How does Malkiel argue that the collapse of Enron gives his theory more evidence?
James “Jim” Cramer is host of the popular Mad Money investment advice show airing
daily on CNBC. Jim gives investment advice. In a popular segment called “The Lightning
Round” callers phone in and ask Jim’s opinion as to whether a stock will go up or down.
Over a 30-day period, Jim Cramer’s picks of whether a stock would go up or down were
right 124 times and wrong 122 times. Assume in the same 30 day period, 50% of stocks
went up and 50% went down.
7. Carry out the 3S strategy for evaluating the quality of Jim’s picks. Make sure to state
null and alternative hypotheses.
8. To see if Jim Cramer’s advice holds up, folks compared Jim’s picks to those of
Leonard the Wonder Monkey who flipped a fair coin to predict whether the stock
would go up or down over the ensuing 30 day period. Is this a fair comparison of
Jim’s picks? If yes, why? If not, what might you do to make it more fair?
9. Imagine if instead of 50% of the stocks in the market went up over the same 30 day
period, that only 40% did. Carry out the 3S strategy for evaluating the quality of Jim’s
picks. Make sure to state null and alternative hypotheses.
10. Is comparing Jim’s picks to those of Leonard the Wonder Monkey a fair comparison
in this scenario? If yes, why? If not, what might make it more fair?
Imagine that you were to test your own stockbroker to see if they could give good shortterm picks (like the test of Jim Cramer). You have your stockbroker pick some stocks
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXXVIII
that he thinks will go up over the next 30 days. You then track all stocks for the next 30
days, and find that 55% of all stocks increased in price during the 30 day time period.
11. If you had asked your broker to pick 5 stocks, how many stocks would he have
needed to get right for you to be convinced he was “better than a monkey”? Why?
12. If you had asked your broker to pick 15 stocks, how many stocks would he have
needed to get right for you to be convinced he was “better than a monkey”? Why?
13. If you had asked your broker to pick 30 stocks, how many stocks would he have
needed to get right for you to be convinced he was “better than a monkey”? Why?
14. Calculate the proportion of “correct picks” needed for each of your answers to 11 to
13. Why is this different in the different scenarios?
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XXXIX
Practice Exercises
1. Review the graph that summarizes the simulation for the Bob or Tim example. The
graph shows the results of 1000 different sets of 33 tosses of a fair coin. The goal of
the simulation was to find out how likely it is to get 23 or more heads in 33 tosses.
For each of (a) – (d) below, indicate what it corresponds to in the example.
a. Each toss that lands heads
A. In individual student
b. Each set of 33 tosses
B. A class of students
c. Each toss of a coin
C. The number of Tim matches
d. Each bar of the graph
D. A single Tim match
2. Review Activity 1.1, Friend or Foe. For each of (a) – (d) below, indicate what it
corresponds to in the activity.
a. Each toss that lands heads
A. In individual infant
b. Each set of 16 tosses
B. The 16 infants
c. Each toss of a coin
C. The number of helpers chosen
d. Each stack of dots in the graph
D. An infant choosing the helper toy
3. Review the Dog Food example and Discriminating Cola activity. Compare them with
the Bob/Tim and Friend/Foe scenarios. In both the Dog Food and Cola scenarios, a
set of individuals is asked to make a choice from a set of options. In all four,cases,
the _______ (null/alternative) hypothesis says that the _______ (options/individuals)
are equally likely. Explain.
4. Review the graphs for the Bob/Tim, Dog Food, and Kissing examples.
• Each stack of dots corresponds to a value of the __________ (parameter/statistic).
• The height of the stack reveals _____________ (choose one of a-e):
a. How many individuals were asked to make a choice
b. How many options there were
c. How many times the value of the statistic occurred
d . How many times the value of the parameter occurred
e. The total number of repetitions
5. In these graphs, the p-value corresponds to ________ (choose one):
a. The height of the bar above the observed value of the statistic.
b. The area of the bars above or to the right of the observed value
c. The area of the bars above or to the left of the observed value
d. The number of bars above or to the right of the observed value
e. The number of bars above or to the left of the observed value
6. The set of individuals you want to know about is called the ________
(sample/population). The set of individuals you get to see is called the ________
(sample/population).
7. Suppose you take several different samples from the same population. You are
likely to get several different values for the _________ (statistic/parameter), but the
value of the ________ (statistic/parameter) will not change.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XL
8. For almost all practical problems, once you have the data, you can compute the
value of the _________ (statistic/parameter) but you can’t compute the value of the
_________ (statistic/parameter).
9. For an election poll, the set of likely voters is often used as the _______
(population/sample).
10. Which one of the following statements is meaningful? (The other three are
meaningless.)
a. “The null hypothesis was statistically significant.”
b. “The observed difference was statistically significant.”
c. “The parameter was statistically significant.”
d. “The sample size was statistically significant.”
11. True or false, and explain:
a. The p-value depends on the sample.
b. The p-value depends on the null hypothesis.
c. The p-value is the probability that the null hypothesis is true.
d. The p-value is the probability that the alternative hypothesis is true.
12. If the p-value is below 0.05, the evidence against the _______ (null/alternative)
hypothesis is ________ (strong/weak).
13. Statistical testing allows you to reject the ________ (null/alternative) hypothesis
based on the data, but cannot justify rejecting the _________ (null/alternative)
hypothesis.
14. When you use the 3S method for hypothesis testing, the simulation you use to find
the p-value is based on the assumption that the ________ (null/alternative)
hypothesis is true.
15. When you use the 3S method, you use simulation to create a large number of
hypothetical _______ (samples/populations). For each one, you compute a value of
the ___________ (statistic/population) and look at the distribution of these values.
16. Multiple Choice. In a sample of size 20, we observe 12 (60%) of the subjects with
the characteristic of interest. In conducting the related test of significance we have a
computer applet simulate counting the number of heads when 20 fair coins are
flipped. This process is repeated 1000 times and a distribution of the number of
heads is made. This distribution represents:
a. Repeated results if the null hypothesis is true.
b. Repeated results if the alternative hypothesis is true.
c. Repeated results if the population proportion is the same as your sample
percentage of 60%.
d. Repeated results if the population proportion is more than your sample proportion
of 60%.
17. Multiple Choice. Suppose you sample 100 students at your school and find that 60%
of them exercise regularly. A friend at another school sample’s 200 students and
finds that 60% of that sample also exercises regularly. You both conduct tests of
significance and find p-values.
a. The sample with the larger sample size will have the smaller p-value.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
b. The sample with the smaller sample size will have the smaller p-value.
c. Since the sample proportions are the same, the p-values will be the same.
d. There is no way to tell which sample will have the smaller p-value.
Copyright © 2010
XLI
Chapter 1: Introduction to Statistical Inference: One Proportion
XLII
Exercises
Section1.1 Introduction to Statistics
1. Pro-life or pro-choice In the May 16, 2009 edition of the Grand Rapids Press, the
article entitled “Most of us say we’re pro-life, poll finds” reported the following
information: A telephone interview conducted by Gallup involved 1,015 adults
nationwide. The poll found that 51% called themselves pro-life rather than prochoice on the issue of abortion. This is the first time that a majority gave that answer
in the 15 years that Gallup has asked the question.
a. What is the population of interest in this study?
b. What is the sample?
c. Identify the statistic(s) in this study. If you don’t know the value of the statistic,
describe it in words.
d. Describe the parameter(s) in this study. If you don’t know the value of the
parameter, describe it in words.
2. Education and Government In August 2010, a Gallup poll asked a random sample
of 1,013 U.S. adults the following question, “In terms of public education in this
country, do you think the federal government [should be more involved in education
than it currently is, should keep its involvement the same, (or) should be less
involved in education than it currently is]?”
Here’s a bar chart that shows the distribution of responses.
a. Approximately what proportion of respondents want the federal government’s
involvement in public education to remain unchanged?
b. Is your proportion from part (a) a parameter or a statistic? How do you know?
c. How many respondents want the federal government’s involvement in public
education to remain unchanged?
d. Suppose that we want to use these data to test whether less than half of
American adults want the federal government to be more involved in public
education than it currently is. State appropriate null and the alternative
hypotheses for this scenario.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XLIII
3. Practice Suppose that our population of interest is all households in San Luis
Obispo County. Identify which of the following are possible parameters.
a. Mean number of residents in all households in the county
b. Whether there are any children under 12 years of age living in the household
c. Proportion of households with children under 12 years of age
d. Whether household is in an urban area
e. Whether size (number of members) of the household is related to the household
expenditure
4. Oil spill Following the oil spill in the Gulf of Mexico, The Pew Research Center for
the People and the Press surveyed a random sample of 994 U.S. adults in early May
2010, and found that 547 of the respondents thought that the oil spill was a major
environmental disaster. In the context of the study, answer the following.
a. Identify the population of interest.
b. Identify the sample.
c. Describe, in words, the relevant parameter of interest.
d. Describe, in words, the statistic corresponding to the parameter described in part
(c). Also, specify the numeric value of the statistic.
5. U.S. Census Did you know that the U.S. Census is required by law? A survey of
1,504 U.S. adults conducted by The Pew Research Center for the People and the
Press in early January 2010, found only 31% of the respondents knew this fact.
a. Identify the population of interest.
b. Identify the sample.
c. Is 31% a parameter or a statistic? How do you know?
d. Do the data provide evidence that less than a third of U.S. adults are “censusaware”? State appropriate null and alternative hypotheses for this scenario.
6. Practice The data collected from a sample of students are analyzed below with a bar
chart and frequency table. Based on this information:
a. What percentage of the sample are freshmen?
b. What percentage of the sample are sophomores?
c. What percentage of the sample are juniors?
d. What percentage of the sample are seniors?
e. What should be true about these four percentages?
Bar Chart
35
30
25
20
Freshman
25
15
Junior
30
10
Senior
13
5
Sophomore
19
Class
Column Summary
87
Freshman
S1 = count ( )
count ( )
Copyright © 2010
Sophomore
Junior
Class
Senior
Chapter 1: Introduction to Statistical Inference: One Proportion
XLIV
Section 1.2 Introduction to Statistical Reasoning: One Proportion
7. Injection drug use Do a majority of women in Rhode Island Prisons who test
positive for Hepatitis C virus report injection drug use? A recent article in the
American Journal of Public Health, (Macalino, G.F. et al., 2005) reported that in a
representative sample of inmates at the time of intake to the prison system 197
women tested positive for the Hepatitis C virus. Of these 197 women, 110 reported
injection drug use.
a. What proportion of women with Hepatitis C virus in this study reported injection
drug use?
b. Specify a null hypothesis and an alternative hypothesis to reflect the research
question that a majority of women who test positive for Hepatitis C report
injection drug use.
c. Use the coin flipping applet to generate 1000 repetitions of 197 Rhode Island
women prisoners who tested positive for Hepatitis C virus under the null
hypothesis. Save a screenshot of your dotplot and p-value.
d. Based on the study’s result, what is the p-value for this test?
e. What are your conclusions based on the p-value you found in part d?
f. To what population are you willing to generalize these results? Explain.
8. Staying up late? In a random sample of 40 students from a small college, 30 went
to bed after 11 pm the previous night. From these data we want to know whether we
can we conclude that the majority of all students at the college go to bed after 11 pm.
a. What are the null and alternative hypotheses?
b. Statistic: What proportion of the sample went to bed after 11pm?
c. Simulate: Use the coin flipping applet to generate 1000 repetitions of 40 students
under the null hypothesis. Save a screenshot of your dotplot and p-value.
d. Strength of evidence: What are your conclusions based on the p-value you found
in part c?
9. Credit card usage Do a majority of female college freshmen at a certain college
have at least one credit card? To answer this question, some student researchers
collected data from 42 female freshmen at their school. They found that 27 of them
had at least one credit card.
a. State the null and alternative hypotheses for a test of significance of the research
question given above.
b. Use the coin flipping applet to answer the following questions and compute the pvalue for this test by using 1000 repetitions.
i) What value did you enter for the “Probability of heads”?
ii) What value did you enter for the “Number of tosses”?
iii) What value did you enter for the “Number of repetitions”?
c. The graph generated is centered at approximately what number?
d. What is your p-value?
e. Use the p-value to give a conclusion for your significance test.
10. Multiple Choice The p-value of a test of significance is:
a. The probability, assuming the null hypothesis is true, that we would get a result
as extreme as the one that was actually observed.
b. The probability, assuming the alternative hypothesis is true, that we would get a
result as extreme as the one that was actually observed.
c. The probability the null hypothesis is true.
d. The probability the alternative hypothesis is true.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XLV
11. Planning to vote? The following figure shows the results of the coin tossing applet.
Suppose the research question was, “Do a majority of students say they plan to vote
in the next election?” Explain what the following numbers shown in the applet mean
in terms of the students being sampled and the process and results from the test of
significance.
a.
b.
c.
d.
e.
f.
What does the 0.5 for the probability of heads represent?
What does the 100 for the number of tosses represent?
What do the 1000 repetitions represent?
What does the 55 in the extreme as cell represent?
What does the 0.202 for the proportion of repetitions represent?
What would be an appropriate conclusion for this test?
12. Monkey see A recent article (Hauser, Glynn, and Wood, 2007) described a study
that investigated whether rhesus monkeys have some ability to understand gestures
made by humans. In one part of the study, the experimenter approached individual
rhesus monkeys and placed 2 boxes an equal distance from the monkey. The
experimenter then placed food in one of the boxes, making sure that the monkey
could tell that one of the boxes received food without revealing which one. Finally,
the researcher made eye contact with the monkey and then gestured toward the box
with the food by jerking his head toward that box. This process was repeated for a
total of 40 rhesus monkeys. It turned out that 30 of the monkeys approached the box
that the human had gestured toward, and 10 approached the other box.
a. Describe how you could use a coin to conduct a simulation analysis of this study
and its result. Give sufficient detail that someone else could implement this
simulation analysis based on your description. Be sure to indicate how you
would decide whether the observed data provide convincing evidence that
rhesus monkeys can read human gestures better than random chance.
b. Use software to conduct a simulation analysis with at least 1000 repetitions.
Report the approximate p-value, and summarize the conclusion that you would
draw about the research question of whether rhesus monkeys have some ability
to understand gestures made by humans.
13. CPR on pets A national survey conducted on October 1-5, 2009 asked pet owners
whether they would perform CPR on their pet in the event of a medical emergency.
In the sample of 1116 pet owners, 58% said that they are at least somewhat likely to
perform CPR on their pet. Investigate whether this sample result provides strong
evidence that more than half of all pet owners in the U.S. are at least somewhat
willing to perform CPR on their pet.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XLVI
14. Practice Indicate whether each of the following p-values provides extremely strong
evidence against the null hypothesis, moderate evidence against the null hypothesis,
or not much evidence against the null hypothesis.
a. 0.052
b. 0.00035
c. 0.417
Section 1.3 Statistical Significance for One Proportion: Other Null
Hypotheses
15. Practice What are the null and alternative hypotheses in the following scenarios?
a. Do a majority of males believe that cigars smell good?
b. Hershey’s claims they put 45% orange Reese’s Pieces in their Reese’s Pieces
candy mixture. Is this true, or is the true percent different from 45%?
c. Do a majority of Americans believe in life-after-death?
d. Do less than 30% of students at your school read for pleasure during the term?
e. Do more than 12% of students at your school skip breakfast every morning?
f. Do more than 25% of the students at your school exercise at least three times
per week?
g. Do fewer than 5% of people aged 14 or older get arrested per year?
16. Psychic powers Statistician Jessica Utts has conducted extensive analysis of
studies that have investigated psychic functioning. One type of study involves
having one person (called the “sender” concentrate on an image while a person in
another room (the “receiver”) tries to determine which image is being “sent.” The
receiver is given four images to choose from, one of which is the actual image that
the sender is concentrating on.
a. If the subjects in these studies have no psychic ability, what is the probability that
they identify the correct image? Is this a null hypothesis or an alternative
hypothesis?
b. Utts (1995) cites research from Bern and Honorton (1994) that analyzed studies
using a technique called ganzfeld. These researchers analyzed a total of 329
sessions (http://www.ics.uci.edu/~jutts/air.pdf). Use software to simulate 1000
repetitions of this study, assuming the null hypothesis to be true. Produce a welllabeled graph of the results.
c. Based on the graph of your simulation results, about how many of these 329
sessions would have to produce a “hit” (correct identification of the image being
“sent.”) in order to provide very strong evidence against the null hypothesis?
Explain how you arrive at this number based on your graph.
d. Utts reported that Bern and Honorton found a total of 106 “hits” in the 329
sessions. Does this result provide very strong evidence against the null
hypothesis? Explain, and summarize your conclusion in the context of this study.
17. Reese’s pieces An astute statistics student wants to determine whether there are
more than 45% orange Reese’s Pieces in small bags of Reese’s Pieces. Being low
on funds, she purchases only one small bag of Reese’s Pieces. There are 25 total
Reese’s Pieces in her bag. Sixteen are orange. The student quickly simulates the
distribution of the number of orange Reese’s Pieces from 1000 samples of 25 total
Reese’s Pieces assuming there are indeed 45% orange in the population. The
graph resulting from 1000 repetitions of this simulation is displayed below.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XLVII
number of orange Reese’s Pieces a. State a null hypothesis and an alternative hypothesis for this student.
b. What percentage of the student’s Reese’s pieces is orange? Is this more than
the 45% she is testing?
c. Use the graph above to find the p-value for this test. What conclusion would you
draw about the null hypothesis from this p-value?
d. Suppose the student’s original bag had 13 orange instead of 16. Now, what
percentage of the student’s Reese’s pieces is orange? Use the graph above to
find the p-value for this test.
e. Which p-value offers more support for the alternative hypothesis that there are
more than 45% orange Reese’s Pieces? Explain why this makes sense using
the percentage of orange Reese’s Pieces you found in (b) and (d).
18. Fantasy golf A statistics professor is in a fantasy golf league with 4 friends. Each
week one of the 5 people in the league is the winner of that week’s tournament.
During the 2010 season, this particular professor was the winner in 7 of the first 12
weeks of the season. Does this constitute strong evidence that his probability of
winning in one week was larger than would be expected if the 5 competitors were
equally likely to win? Conduct a simulation analysis to investigate this question.
Write a paragraph summarizing your conclusion and explaining the reasoning
process by which your conclusion follows from the simulation analysis.
20. Breakfast Findings at James Madison University indicate that 21% of students eat
breakfast 6 or 7 times a week. A similar question was asked of a random sample of
159 students at a different college. Of the 97 who responded, 35 reported eating
breakfast 6 or 7 times a week. Do students at the other college have healthier
breakfast habits than James Madison students? More specifically, do more than
21% of all students at the other college eat breakfast 6 or 7 times weekly?
a. State your null and alternative hypotheses.
b. Statistic: What sample proportion of students at the other college eat breakfast 6
or 7 times per week?
c. Simulate: Use the coin flipping applet to simulate “could have been” outcomes
under the null hypothesis 1000 times.
d. Based on the study’s result, what is the p-value for this test?
e. Strength of evidence: What are your conclusions based on the p-value you found
in part d?
f. What are your thoughts about the fact that only 97 out of the random sample of
159 responded?
21. Cloning humans In a May 2010 poll of a random sample of 1,029 U.S. adults,
Gallup found that 88% thought that the cloning of humans was morally unacceptable.
Explain what, if anything, is incorrect in the following statements.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XLVIII
a. The population is the 1,029 U.S. adults who were interviewed.
b. The population is U.S. adults who think that cloning of humans is morally
unacceptable.
c. The number 88% is a parameter.
d. The sample is all U.S. adults.
e. The statistic is 1,029 U.S. adults.
f. The statistic is the average number of U.S. adults who think cloning humans is
morally unacceptable.
g. If we repeatedly poll random samples of 1,029 U.S. adults, the percentage of
respondents who think that the cloning of humans is morally unacceptable is
always going to be 88%.
22. Cloning humans (contd.) Reconsider the previous exercise. Suppose that we want
to use the data to test whether the proportion of U.S. adults who think that the
cloning of humans is morally unacceptable is over 0.80. What, if anything, is
incorrect in the following?
a. Alternative hypothesis: The proportion of U.S. adults who think that the cloning of
humans is morally unacceptable is less than 0.80.
b. The p-value is the probability of observing a sample proportion of 0.88, if the
proportion of U.S. adults who think that the cloning of humans is morally
unacceptable is 0.80.
c. Another survey of 1,029 randomly selected U.S. adults, run by a different survey
team found 85% of their respondents think that the cloning of humans is morally
unacceptable. These data would be stronger evidence against the null
hypothesis, than the data collected by Gallup.
23. Pro-choice or pro-life (contd.) Reconsider the exercise. Suppose that we want to
use the data from Gallup’s survey to test whether a majority of U.S. adults are now
pro-life rather than pro-choice. Using the Simulating Coin Tossing applet the
simulated p-value was found to be 0.276. What, if anything, is incorrect in the
following statements?
a. Null hypothesis: A majority of U.S. adults are now pro-life rather than pro-choice.
b. Since the p-value is fairly large, we have strong evidence that exactly half of all
U.S. adults are pro-life and exactly half are pro-choice.
c. Since the p-value is fairly small, we have strong evidence that a majority of U.S.
adults are now pro-life.
d. Since the p-value is fairly large, this survey has provided no information about
the proportion of U.S. adults who are now pro-life.
Section 1.4 Plausible Values
24. Multiple Choice If you don’t have much evidence against the null hypothesis, you
can:
a. Conclude the null hypothesis must be true
b. Conclude that the null hypothesis is one of a set of plausible values
25. Democrat or Republican A political poll finds that 305 of 600 likely voters are in
favor of the Democrat over the Republican in a recent local election between the two
candidates. Is this evidence that the majority of likely voters are in favor of the
Democrat candidate?
a. What is the proportion of likely voters in favor of the Democrat candidate in this
sample?
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
XLIX
b. Use the 3S process to evaluate whether this sample is evidence that the majority
of likely voters in the population favor the Democrat candidate.
c. Does your answer to b) mean that you have proven that the population of all
likely voters is perfectly split (50/50) between the two candidates?
26. Outbreak Recently a small college in the Midwest experienced an outbreak of
norovirus which forced its temporary closure. In a survey of a random sample of
students after the outbreak, 34 out of 187 students reported experiencing symptoms
of norovirus during the outbreak.
a. Find the proportion of students reporting symptoms of norovirus.
b. What is the population parameter of interest?
c. State null and alternative hypotheses and then use the 3S process to investigate
whether this sample provides evidence that more than 10% of students at the
college experienced symptoms.
d. State null and alternative hypotheses and then use the 3S process to investigate
whether this sample provides evidence that less than 30% of students at the
college experienced symptoms.
e. Use your results from c) and d) and more computer simulations to find a range of
plausible values of the population parameter. Remember that a range of
plausible values is a set of numbers against which your sample does not provide
much evidence. Make your range accurate to the nearest 1 percentage point.
Clarify how you determined convincing evidence or not much evidence.
Section 1.5 Effect of Sample Size
27. Phone home Suppose we were testing the hypothesis that more than 50% of
college students call home at least once a week and use the 3S analysis strategy to
estimate the p-value. Also suppose that four students collect data to try to answer
the question. Each student collects a different sample size (one of 10, one of 20,
one of 40, and one of 80). Surprisingly, in each case their results showed that 70%
of the students in their respective samples called home at least once a week.
Use the following dotplots of 1000 repetitions for these simulations to decide whether
each result is statistically significant (small p-value) and they can therefore conclude
that more than 50% of students call home at least once a week. Also comment on
what happened to the p-value as the sample size increased even when the sample
proportion stayed the same?
number of college students calling home This student
found
that 28
out of 40
(or 70%)
of the
number
of college
students
calling
home students called home at least once a week.
This student found that 7 out of 10 (or 70%) of the
students called home at least once a week.
Copyright © 2010
number of college students calling home number of college students calling home This student found that 56 out of 80 (or 70%) of the
students called home at least once a week.
This student found that 14 out of 20 (or 70%) of the
students called home at least once a week.
Chapter 1: Introduction to Statistical Inference: One Proportion
L
28. Story falls flat A legendary campus story tells of two students who miss an exam
because they are off partying. When they return to campus, they sheepishly
approach the professor and say that they missed the exam because of a flat tire.
The students are delighted when the professor grants them an opportunity to take a
make-up exam. But when they are sent to separate rooms to take the make-up
exam, they find that one question is worth 95 points: Which tire was flat? It’s been
conjectured that when students are asked this question and forced to give an answer
(left front, left rear, right front, or right rear) off the top of their head, they tend to
answer “right front” more than would be expected by random chance. To test this
conjecture, this question was asked of a recent class of 32 students, with the
following results:
Left front Left rear Right front Right rear
5
4
18
5
a. State the appropriate null and alternative hypotheses to be tested.
b. Produce a bar graph to display the student responses, and comment on what the
graph reveals.
c. Statistic: Calculate the sample proportion who answered “right front.” Does this
statistic appear to support the research conjecture? Explain.
d. Simulate: Use software to conduct a simulation analysis for investigating whether
the sample data provide strong evidence for the research conjecture. Submit a
graph of the what if the null was true distribution, and report the approximate pvalue.
e. Strength of Evidence: Explain what this p-value means. In other words, this is
the probability of what, assuming what? (Do not draw a conclusion from the pvalue yet; that’s the next question.)
f. Summarize the conclusion that you draw from this test. Also explain the
reasoning process behind your conclusion.
29. Story falls flat (contd.) Reconsider the previous exercise. Suppose another class
conducts the same study with exactly half as many students, and suppose the
proportional breakdown in the four categories is identical to the previous exercise. In
other words, 9 out of 16 students answered, “right front.”
a. Before you analyze the data, would you expect to find stronger evidence for the
research conjecture (that people pick the right front tire more than ¼ of the time),
weaker evidence, or the same strength of evidence? Explain your thinking.
b. Conduct a simulation analysis to produce an approximate p-value. How does it
compare to the p-value from the previous exercise? Is this what you expected?
Explain.
30. Haiti earthquake On January 12, 2010, a 7.0 magnitude earthquake shook Haiti,
affecting about three million people. Aid started pouring in from all parts of the world
in the form of volunteer services, donations, etc. In a February 2010 survey of 1,383
randomly chosen U.S. adults, The Pew Research Center for the People and the
Press found that 719 of the respondents had made a donation to Haiti victims. We
want to use these data to investigate whether more than half of U.S. adults made a
donation to Haiti victims.
a. Identify the population of interest.
b. Describe the relevant parameter of interest.
Copyright © 2010
Chapter 1: Introduction to Statistical Inference: One Proportion
LI
c. State the null hypothesis and the alternative hypothesis.
d. Of the people that the Pew Research Center interviewed, what proportion made
a donation to Haiti victims?
e. Is the proportion you calculated in part (d) a parameter or a statistic? How do you
know?
f. The study description says that we are looking at data from “randomly chosen
U.S. adults.” What is the main advantage of having data from “randomly chosen
U.S. adults”?
g. Investigate whether the data provide evidence that more than half of U.S. adults
made a donation to Haiti victims. Be sure to include the p-value, an interpretation
of the p-value, and your conclusion in the context of the study.
h. A high school student surveys a random sample of 200 adults in his city, and
finds that 52% of the respondents made donations to Haiti victims. The high
school student decides to use his data to test whether more than half of the
adults in the city made a donation to Haiti victims. Will the p-value from his
analysis be larger than, smaller than, or the same as your p-value from part (g)?
Explain your choice.
Copyright © 2010