Statistics for the Social Sciences
Psychology 340
Spring 2010
Hypothesis testing
PSY 340
Statistics for the
Social Sciences
Reminders
• Don’t forget to complete homework 4 for
Feb 9 (Tues)
• And Quiz 3 (Chapters 5, 6, & 7) by 11AM
Thurs (Feb 4)
• Exam 1 Feb 11 (Thurs)
PSY 340
Statistics for the
Social Sciences
Outline (for the week)
• Review of:
– Hypothesis testing framework
• Stating hypotheses
• General test statistic and test statistic distributions
• When to reject or fail to reject
– Effect sizes: Cohen’s d
– Statistical Power
PSY 340
Statistics for the
Social Sciences
Hypothesis testing
• Example: Testing the effectiveness of a new memory
treatment for patients with memory problems
– Our pharmaceutical company develops a new drug
treatment that is designed to help patients with impaired
memories.
– Before we market the drug we want to see if it works.
– The drug is designed to work on all memory patients (the
population), but we can’t test them all.
– So we decide to use a sample and conduct the following
experiment.
– Based on the results from the sample we will make
conclusions about the population.
PSY 340
Statistics for the
Social Sciences
Hypothesis testing
• Example: Testing the effectiveness of a new memory
treatment for patients with memory problems
Memory
patients
Memory
treatment
Memory 55
Test
errors
No Memory
treatment
Memory 60
errors
Test
• Is the 5 error difference:
– A “real” difference due to the effect of the treatment
– Or is it just sampling error?
5 error
diff
PSY 340
Statistics for the
Social Sciences
Testing Hypotheses
• Hypothesis testing
– Procedure for deciding whether the outcome of a study
(results for a sample) support a particular theory (which
is thought to apply to a population)
– Core logic of hypothesis testing
• Considers the probability that the result of a study could have
come about if the experimental procedure had no effect
• If this probability is low, scenario of no effect is rejected and
the theory behind the experimental procedure is supported
PSY 340
Statistics for the
Social Sciences
Inferential statistics
• Hypothesis testing
– A five step program (note: these steps are different than the
book’s)
•
•
•
•
•
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data
Step 4: Compute your test statistics
Step 5: Make a decision about your null hypothesis
PSY 340
Statistics for the
Social Sciences
Hypothesis testing
• Hypothesis testing: a five step program
– Step 1: State your hypotheses: as a research hypothesis and a
null hypothesis about the populations
• Null hypothesis (H0)
This is the one that you test
• There are no differences between conditions (no effect of treatment)
• Research hypothesis (HA)
• Generally, not all groups are equal
– You aren’t out to prove the alternative hypothesis
• If you reject the null hypothesis, then you’re left with
support for the alternative(s) (NOT proof!)
PSY 340
Statistics for the
Social Sciences
Testing Hypotheses
• Hypothesis testing: a five step program
– Step 1: State your hypotheses
In our memory example experiment:
One -tailed
– Our theory is that the
treatment should improve
memory (fewer errors).
H0: Treatment 
HA: Treatment
 NoTreatment
  NoTreatment
PSY 340
Statistics for the
Social Sciences
Testing Hypotheses
• Hypothesis testing: a five step program
– Step 1: State your hypotheses
In our memory example experiment:
direction
One -tailed
specified
– Our theory is that the
treatment should improve
memory (fewer errors).
 NoTreatment
  NoTreatment
no direction
specified
Two -tailed
– Our theory is that the
treatment has an effect on
memory.
H0: Treatment 
H0: Treatment 
 NoTreatment
  NoTreatment
HA: Treatment
HA: Treatment
PSY 340
Statistics for the
Social Sciences
One-Tailed and Two-Tailed Hypothesis Tests
• Directional
hypotheses
– One-tailed test
• Nondirectional
hypotheses
– Two-tailed test
PSY 340
Statistics for the
Social Sciences
Testing Hypotheses
• Hypothesis testing: a five step program
– Step 1: State your hypotheses
– Step 2: Set your decision criteria
• Your alpha (α) level will be your guide for when to reject or fail
to reject the null hypothesis.
– Based on the probability of making making an certain type of error
– Essentially this is the process of deciding, in advance of collecting your
observations, how big a difference between groups is needed to reject
the null hypothesis
PSY 340
Statistics for the
Social Sciences
Performing your statistical test
• What are we doing when we test the hypotheses?
Real world (‘truth’)
H0: is true (no treatment effect)
H0: is false (is a treatment effect)
One
population
Two
populations
XA
the memory treatment sample are the
same as those in the population of
memory patients.
XA
they aren’t the same as those in the
population of memory patients
PSY 340
Statistics for the
Social Sciences
Error types
There really
isn’t an effect
One
pop
Real world (‘truth’)
H0 is
correct
H0 is
wrong
There
really is
an effect
Two
pops
Reject
H0
Experimenter’s
conclusions
Fail to
Reject
H0
PSY 340
Statistics for the
Social Sciences
Error types
Real world (‘truth’)
I conclude that
there is an
effect
H0 is
correct
Reject
H0
Experimenter’s
conclusions
Fail to
Reject
H0
I can’t detect
an effect
H0 is
wrong
PSY 340
Statistics for the
Social Sciences
Error types
Real world (‘truth’)
H0 is
correct
Reject
H0
Experimenter’s
conclusions
Fail to
Reject
H0
H0 is
wrong
Type I
error

Type II
error

PSY 340
Statistics for the
Social Sciences
Error types
• Type I error (α): concluding that there is a
difference between groups (“an effect”) when
there really isn’t.
– Sometimes called “significance level” or “alpha level”
– We try to minimize this (keep it low)
• Type II error (β): concluding that there isn’t an
effect, when there really is.
– Related to the Statistical Power of a test (1-β)
PSY 340
Statistics for the
Social Sciences
Testing Hypotheses
• Hypothesis testing: a five step program
– Step 1: State your hypotheses
– Step 2: Set your decision criteria
– Step 3: Collect your data
PSY 340
Testing Hypotheses
Statistics for the
Social Sciences
• Hypothesis testing: a five step program
–
–
–
–
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data
Step 4: Compute your test statistics
• Descriptive statistics (means, standard deviations, etc.)
• Inferential statistics (z-test, t-tests, ANOVAs, etc.)
PSY 340
Statistics for the
Social Sciences
Performing your statistical test
• What are we doing when we test the hypotheses?
– Computing a test statistic: Generic test
Could be difference between a sample and a
population, or between different samples
observed difference
test statistic 
difference expected by chance
Based on standard error or an
estimate of the standard error
PSY 340
Statistics for the
Social Sciences
Distribution of sample means
• A distribution of all possible sample means drawn
from the population (of a particular sample size)
Population
Distribution of sample means

X
Mean of all
samples of n = #
3 X
XX
4 X 1
2
 this in the next lecture
Much more detail about
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
• The generic test statistic distribution (a transformation of the
distribution of sample means)
– To reject the H0, you want a computed test statistics that is large
– What’s large enough?
• The alpha level gives us the decision criterion
Distribution of sample means
Distribution of the test statistic
X
Transform to
using statistical
test

α-level determines where
these boundaries go
PSY 340
Testing Hypotheses
Statistics for the
Social Sciences
• Hypothesis testing: a five step program
–
–
–
–
–
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data
Step 4: Compute your test statistics
Step 5: Make a decision about your null hypothesis
• Based on the outcomes of the statistical tests researchers will either:
– Reject the null hypothesis
– Fail to reject the null hypothesis
• This could be correct conclusion or the incorrect conclusion
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
• The generic test statistic distribution (think of this as the distribution
of sample means)
– To reject the H0, you want a computed test statistics that is large
– What’s large enough?
• The alpha level gives us the decision criterion
Distribution of the test statistic
If test statistic is
here Reject H0
If test statistic is here
Fail to reject H0
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
• The alpha level gives us the decision criterion
Two -tailed
One -tailed
α = 0.05
Reject H0
Reject H0
0.025
split up
into the
two tails
0.025
Fail to reject H0
Reject H0
Fail to reject H0
Fail to reject H0
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
• The alpha level gives us the decision criterion
Two -tailed
One -tailed
α = 0.05
all of it in
one tail
Reject H0
Reject H0
0.05
Fail to reject H0
Reject H0
Fail to reject H0
Fail to reject H0
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
• The alpha level gives us the decision criterion
Two -tailed
One -tailed
α = 0.05
Reject H0
all of it in
one tail
Reject H0
0.05
Fail to reject H0
Reject H0
Fail to reject H0
Fail to reject H0
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
An example: One sample z-test
Memory example experiment:
• Step 1: State your hypotheses
One -tailed
• We give a n = 16 memory patients a
H0: the memory treatment
memory improvement treatment.
sample are the same as
those in the population of
• After the treatment they have an
memory patients.
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have
 of memory errors that is
a distribution
Normal, μ = 60, σ = 8?
μTreatment ≥ (μpop = 60)
HA: the memory treatment
sample make fewer errors
the the population
μTreatment < (μpop = 60)
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
An example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a
memory improvement treatment.
• After the treatment they have an
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have
 of memory errors that is
a distribution
Normal, μ = 60, σ = 8?
H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
One -tailed
•
Step 2: Set your decision
criteria
α = 0.05
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
An example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a
memory improvement treatment.
• After the treatment they have an
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have
 of memory errors that is
a distribution
Normal, μ = 60, σ = 8?
H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
One -tailed
•
α = 0.05
Step 3: Collect your data
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
An example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a
memory improvement treatment.
H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
One -tailed
•
Step 4: Compute your test
statistics
• After the treatment they have an
X  X
average score of X = 55 memory errors.
zX 
X
• Why
How do
theythis
compare
to the general
isn’t
just σ?
population
ofthe
memory
patients
This is
standard
error who
(σX) have
rather than
a distribution
of memory
errors
that is (σ). It is
thepopulation
standard
deviation
Normal,
= 60,  deviation
= 8?

thestandard
of the distribution
of
sample means. The formula for this is:
X 

n
α = 0.05
We will cover in detail in the next lecture.
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
An example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a
memory improvement treatment.
• After the treatment they have an
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have
 of memory errors that is
a distribution
Normal, μ = 60, σ = 8?

H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
α = 0.05
One -tailed
•
Step 4: Compute your test
statistics
zX 
X  X
X
= -2.5


55  60
 8



 16 
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
An example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a
memory improvement treatment.
H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
 = 0.05
One -tailed
zX  2.5
• Step 5: Make a decision
• After the treatment they have an
about your null hypothesis
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have 5%
 of memory errors that is
a distribution
Normal, μ = 60, σ = 8?
-2
-1

Reject H0
1
2
PSY 340
Statistics for the
Social Sciences
“Generic” statistical test
An example: One sample z-test
Memory example experiment:
H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
• We give a n = 16 memory patients a
memory improvement treatment.
• After the treatment they have an
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have
 of memory errors that is
a distribution
Normal, μ = 60, σ = 8?
One -tailed
α = 0.05
zX  2.5
•
Step 5: Make a decision
about your null hypothesis
- Reject H0
- Support for our HA, the
evidence suggests that the
treatment decreases the
number of memory errors