Hypothesis Testing
A Research Question
• Everybody knows men are better drivers than
women.
• Hypothesis: A tentative explanation that accounts
for a set of facts and can be
tested by further
investigation
– Hypothesis: Men are
better drivers
than women
Erin Crocket
NASCAR
Proof
• How would you prove that men are better drivers?
• Gather information on the dependent variable (driving) and
compare based on the independent variable (gender).
• What if most men were better but not all?
• What about all the people you never asked?
What happens if this is reversed?
• How could you disprove that there was a difference in
driving ability by gender?
• Null hypothesis: stating that no differences would be found
when you really are interested in finding them.
• If the hypothesis says that something is true then the null
hypothesis says that it is not true.
• A short diversion to Popper
Karl Popper: Method of Falsification
• Form a hypothesis
• Try to prove it wrong
• From the results make a new hypothesis
Writing Null Hypotheses
• What is your dependent variable?
• What groups will you compare around the
dependent variable (independent variable)?
• Rewrite your problem statement as an
expected outcome.
• Rewrite the expected outcome statement as
expecting no differences between groups to
appear.
Hypothesis Testing
• Hypothesis: Men are better drivers than
women.
• Null Hypothesis: There is no difference
between men and women as drivers.
• You gather evidence and decide if the
evidence says that men are better drivers.
• You show that the hypothesis is not not true.
• You reject the null hypothesis.
Using Null Hypotheses
• Use a sample (otherwise we would use
descriptive statistics)
• Stop talking about individuals and only
compare groups (mean and standard deviation)
• Measure a bunch of people and if there is no
difference you say, “I have been unable to find
evidence that differences exist.”
• You have been unable to “reject the null hypothesis.”
• If you wanted to show that men drive differently than
women, you failed.
• The problem with understanding this is the double
negative. You are trying to show that a hypothesis is
not “not true.”
• But, of course there will be some difference.
Example
• The purpose of this study is to examine the
impact of reading circles on comprehension
scores.
• Null hypothesis: There is no impact of reading
circles on comprehension scores.
• If there is a difference between group means
you have a problem.
• How big do the differences have to be before
we believe they didn’t just happen by chance?
• The differences need to be significant: that
means they are so big they are unlikely to
happen by chance.
Rejecting the Null
• If the group differences are small then you could
make the case that the differences in comprehension
scores could happen naturally (by chance) and not
because of reading circles. (the null—there is no
difference— is not rejected)
• If the group differences are big enough then they are
unlikely to have happened by chance: there is a
significant difference in comprehension scores when
students are in reading circles. (the null—there is no
difference—is rejected)
Why do this?
• This is Popper’s fault: Falsification Theory
• Inferential statistics uses samples of
populations to determine if differences in
group means could occur by chance.
• Inferential statistics is not used to prove
hypotheses. It is used to demonstrate that null
hypotheses are not true.
Think about this as steps:
1.
You want to know that something is true but you can’t absolutely
know because you can’t test all cases.
2.
Instead you focus on the opposite of what you want to know is true
(the null hypothesis).
3.
Start gathering evidence.
4.
If enough cases show that something is true then saying it was not
true is false.
5.
You have rejected the null hypothesis.
6.
For the time being, the thing you wanted to show is true (your
hypothesis) is the best explanation.
Ok, one more semantic step and we are done:
• What we are interested in knowing is whether something made two
groups different.
• The evidence that you gather to see if that is true are means and standard
deviations of groups.
• The null hypothesis always says there will be no difference between the
groups.
• If there is a very low probability that the difference in the groups could
occur by chance, that is the evidence that the null hypothesis is wrong.
• Low probabilities (p values) show that the thing you wanted to show
(your hypothesis) is more likely to be true than not true. You have
rejected the null hypothesis.
Hypothesis Testing
• Write your problem statement as a quantitative
problem: group, characteristic to be measured
(dependent), intervention (independent).
• Remember we write problem statements like
we don’t know what the answer is: What is the
impact of reading circles on comprehension
scores? (Non-Directional)
Hypothesis Testing
• Now, write your problem statement as a
hypothesis: What you think will be the impact
of the independent variable on the dependent.
(Directional)
• Don’t put this in your paper but the hypothesis
is what you are thinking is true.
Hypothesis Testing
• Now, write the null hypothesis. No observable
impact of the independent variable will occur
on the dependent variable. (No Direction)
• There will always be group mean differences
but you want to show that group mean
differences are large enough that you can reject
the null hypothesis.