Topics:
You Pick the Topic:
P-values & Hypothesis Testing
Calculating a p-value
Determining the right test
Stat203
Fall2011 – Week 14, Lecture 1
Page 1 of 15
What the heck are p-values?
Simply, the p-value
is the probability
that the observed
test statistic (or one
more extreme)
could have
occurred if the null
hypothesis was
true
Observed
Test Statistic
Stat203
Fall2011 – Week 14, Lecture 1
Page 2 of 15
What the heck is a test-statistic?
Let’s back up and recall what the idea of hypothesis testing
was.
• Make a null hypothesis: “Nothing happening”
• Make a research hypothesis: “Something interesting
happening”
• Gather information
• Assume the null hypothesis is true
• Use the null hypothesis (and often the central limit
theorem) to get a probability model (t, z, Chi-square, etc…)
• See if the evidence contradicts the assumption that the null
is true?
Stat203
Fall2011 – Week 14, Lecture 1
Page 3 of 15
That makes sense, but what does “evidence contradicts
the assumption” mean?
Let’s back up even farther with a simple example, let’s go
back to Week 8, Lecture 1, the first hypothesis testing
examples.
Think about: what would we observe that supports the null
hypothesis? What would we observe that doesn’t support
the null hypothesis?
Review W8 L1 thoroughly.
Stat203
Fall2011 – Week 14, Lecture 1
Page 4 of 15
So, a test statistic a generic term.
A test statistic is calculated from the data.
A test statistic standardizes our data so that we can
calculate the probability of observing our data if the null is
true.
We learned about three different test-statistics:
- t-test (statistic) (means and correlations)
- z-test (statistic) (proportions)
- Chi-Square test (statistic) (distributions)
Stat203
Fall2011 – Week 14, Lecture 1
Page 5 of 15
To Summarize:
A hypothesis test is a way to determine what samples tell
us about populations:
- 1-sample test: is the population mean the same,
greater or less than some value (µ0)
- 2-sample test: is the mean for one population, µ1, the
same, greater, or less than the mean in another
population, µ2
could make the same sorts of statements as the above for a
proportion p
Stat203
Fall2011 – Week 14, Lecture 1
Page 6 of 15
A test statistic is a way of standardizing a statistic from
our sample (mean, proportion, correlation) so we can
calculate a probability.
Specifically … if the null hypothesis is true, how likely
was it that we could have obtained our sample or one
more consistent with the alternative (ie: more extreme).
Reconsider the hypotheses for hours of TV children
watch, and the duration of cable installation time …
choose some possible values for sample means and
identify them on a bell curve (the sampling distribution of
the mean).
Now determine other values which are consistent with
the alternative  this defines the p-value.
Stat203
Fall2011 – Week 14, Lecture 1
Page 7 of 15
So, a p-value is the probability of obtaining data with the
sample mean we observed if if the null hypothesis was
true.
- if the p-value is small, it is very unlikely we could have
obtained our data if the null is true … but we did
obtain our data!  so decide that the null is not true
- if the p-value is large, it is very likely we could have
obtained our data  so decide the null is true
We never know whether the null is true or not! All we know
is that our data are or are not consistent with the null.
Stat203
Fall2011 – Week 14, Lecture 1
Page 8 of 15
How does this translate to the 2-sample problem?
Remember, the two-sample hypotheses could all be
written in two ways (W8L3).
So, if we were to do the same exercise as in the 1sample case, we’d choose values of the DIFFERENCE
in means that are consistent or inconsistent with the null
hypothesis.
Try this for the BFF example.
Stat203
Fall2011 – Week 14, Lecture 1
Page 9 of 15
Calculating a p-value
- we calculate our test statistic
- we identify where our test statistic is (bell curve for tand z-statistics, chi-square curve for chi-square)
- identify the region we need
- decide whether that region is larger or smaller than 
Stat203
Fall2011 – Week 14, Lecture 1
Page 10 of 15
Remember! A p-value is a probability! It is always between
zero and 1. Don’t confuse a test-statistic with the p-value.
- The test statistic is just another thing we measure
(observe) on the data
- The p-value tells us how far the observed test statistic
is away from null value … ie: how much or how little
evidence there is for the null hypothesis
We’re clear on why an area under a curve is a probability,
right? Let’s look back to W2L1, W3L1, W3L3 and W4L2.
Stat203
Fall2011 – Week 14, Lecture 1
Page 11 of 15
Choosing the correct test
Vs
Choosing the correct hypotheses
These are not quite the same things, but you can’t get the
right test if you don’t use the correct hypotheses.
Remember, the research hypothesis is usually given in the
question. It’s the thing the researcher really want’s to find
out.
The Null hypothesis always says ‘no difference’.
Stat203
Fall2011 – Week 14, Lecture 1
Page 12 of 15
If you think you can determine the research hypothesis in
words, then you need to determine a few more things in
order to do the correct test:
- Do you have 1 sample or 2?
 Are there two sample means in the problem?
 Are you comparing two groups, or one group to a
standard value?
 Are you sure you have two samples instead of
just two variables?
o If one of the two variables
Stat203
Fall2011 – Week 14, Lecture 1
Page 13 of 15
- What parameters are you testing?
 µ - the variable of interest in the scientific
question interval or ratio?
 p - (proportion) is the variable of interest a yes/no
or other two-category nominal variable?
 ρ - (rho) are you interested in a relationship?
 f - (distribution) are you deciding whether all
categories of a nominal or ordinal variable are
equal?
Stat203
Fall2011 – Week 14, Lecture 1
Page 14 of 15
Tests we’ve learned:
-
1-sample t-test
2-sample t-test
1-sample z-test for proportions
2-sample z-test for proportions
t-test for non-zero correlation
Chi-square test for distribution
Look through W12L1 for further details on what to look for
on each type of test.
Stat203
Fall2011 – Week 14, Lecture 1
Page 15 of 15