P2010 Two Population Tests
Corty – Ch 8,9
Prequel: Ways of conducting Research involving two populations
Suppose I’ve developed a new pain reliever that I want to compare with Tylenol. The research will involve
administration of the pain reliever, a waiting period for it to take effect, the administration of a standard pain
(forcing the participants to listen to a statistics lecture on sampling distributions), then administration of an
“Absence of Pain” questionnaire, with high scores indicating little pain felt. So higher scores are better.
The research can be conducted in three different ways.
Independent groups Design
Two separate (independent; not paired or matched) groups are used.
One group receives the new pain reliever- it's the experimental group.
The other receives the standard pain reliever - it's the control group.
Pretest scores
Matched participants Design
78
77
75
74
73
70
68
67
60
58
57
57
56
The groups consist of matched pairs - each person has a "matched" twin
in the other group.
Matching is performed with respect to one or more pretest variables related to
the dependent variable.
One group receives the new pain reliever- it's the experimental group.
The other receives the standard pain reliever - it's the control group.
Participants as their own controls Design
One group is identified.
Participants in the group are given the treatment at one time.
They're given the control condition at some other time.
Statistical analyses: looking ahead.
The independent groups design requires the independent groups t-test.
The matched groups and participants as their own controls designs are known
collectively as correlated groups designs. They require the correlated groups t-test or as SPSS calls it,
the dependent samples t-test or as Corty calls it, the paired samples t-test.
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 1
2/9/2016
Independent samples Formulas
Moving from one sample to two independent samples
One sample
Observed mean – Expected mean
t = -----------------------------S
--N
Two independent samples – equal sample sizes
Observed mean 1 – Observed mean 2 – 0
t = ------------------------------------S12 + S22
-----------N
Two independent samples – unequal sample sizes
Observed mean 1 – Observed mean 2 – 0
t = --------------------------------------------------(N1-1)S12 + (N2-1)S22
1
+
1
----------------------- ----N1-1 + N2-1
N1
N2
Corty
Equation
8.2.
Obviously, the equal sample-sizes formula is simpler than the unequal sample sizes formula.
But since we NEVER compute t-statistics by hand, the distinction between them is irrelevant in the
computer age.
Since the unequal sample sizes formula yields the same number as the equal sample sizes formula when
sample sizes happen to be equal the computer programs that do the computations for us always use the
unequal sample sizes formula.
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 2
2/9/2016
Independent Groups t-test Example Problem
Based on an example given in Minium, et. al. p. 251.
A student is interested in whether fragrances enhance memory. He has participants read a passage from
a text.
Half the participants read the passage in the presence of a pleasant but unfamiliar fragrance.
The other half read the passage with no experimenter-provided scent present.
One week later, all participants are brought back to the lab, and are given a test of their memory for facts
from the passage they had read. The Scent group was given the test on a sheet of paper scented with the
same fragrance they had experienced when reading the passage. The other group was given the test with no
experimenter-provided scent. The interest was in a comparison of performance of the two groups.
The data are as follows . . .
This is how data are supposed to be entered
for the independent groups t-test.
Note that ALL recall scores are in the
same column of the SPSS data editor.
For this test, we do NOT put the scores in
two different columns.
To tell SPSS which group each score belongs
to, we create a separate GROUP column and
put numbers (0 vs 1 or 1 vs 2) in it to
identify the group.
Mike – demo this
_
Performing the analysis using SPSS . . .
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 3
2/9/2016
Analyze -> Compare Means -> Independent-Samples T Test
The dialog box
Mike – the data file is
MDBT\P201\Independent
Groups t Example.sav
The output
Gr oup S tatis tics
recall
sce ntgrp
.00 No scent
1.0 0 Scent
Biderman’s P201 Handouts
N
15
15
Me an
Std . Deviatio n Std . Erro r Me an
21 .4000
4.2 5609
1.0 9892
26 .2000
4.4 2719
Topic 13: Two Population Tests - 4
1.1 4310
2/9/2016
Argh!!!! Reading the independent groups t output. Start here on 10/15/15.
SPSS gives three tests of significance including TWO t values. You have to pick the correct one.
Independent Sam ples Test
Levene' s Test for E qual ity of
Va riance s
F
recall
Eq ual va riances assume d
Sig .
.00 1
.97 8
Eq ual va riances no t assumed
t-te st for Equa lity o f Me ans
t
-3.0 27
df
-3.0 27
27. 957
28
Sig . (2-t ailed ) Me an Differe nce
.00 5
-4.8 0000
.00 5
-4.8 0000
Std . Erro r
Dif feren ce
1.5 8565
95% Co nfide nce In terva l of
the Diffe rence
1.5 8565
Lower
-8.0 4806
Up per
-1.5 5194
-8.0 4828
-1.5 5172
Three tests of significance are presented in the table.
The first is a test that compares the variances of the two groups. It’s the F on the very left side of the table.
The result of this test determines which of the following two t-tests is to be used.
The second is the “equal variances assumed” t, in the upper row of the table.
The third is the “equal variance not assumed” t, in the lower row of the table.
Use the following decision rule
If the leftmost “Sig.” is > .05, the population variances are equal, so we used the equal variances t in the
top row of the table.
If leftmost “Sig.” is <= .05, the population variances are not equal, so we use the equal variances not
assumed t in the bottom row of the table.
(The reasons for this complexity are beyond the scope of this course.)
Symbolically . . .
No
Is p-value for F
<= .05?
Yes
Interpret the “Equal
Variances assumed” t
Interpret the “Equal
Variances not
assumed” t
In this example, the p value for the variances test is .978 which is larger than .05, so we retain the
hypothesis of equal variances and use the equal variances t.
In this particular example, both the equal variance t and the unequal variances t are the same value, -3.027.
But they won’t always be equal, and their p-values will not always be equal.
Bottom Line: For this course, we will use the “Equal Variances Assumed” t value – the one at the top.
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 5
2/9/2016
The Hypothesis Testing Answer Sheet for the Independent Groups t-test
Give the name and the formula of the test statistic that will be employed to test the null hypothesis.
Independent Groups t-test
If you get the name correct, I’ll assume you could write the formula.
Check the assumptions of the test
Distributions appear to be approximately US within each group.
Mean of scented score population = mean of unscented score pop
Null Hypothesis:_____________________________________________________________________
Mean of scented score population ≠ mean of unscented score pop
Alternative Hypothesis:______________________________________________________________
What significance level will you use to separate "likely" value from "unlikely" values of the test statistic?
Significance Level = _______________________.05_______________________________________
What is the value of the test statistic computed from your data and the p-value?
t = -3.027
p-value = .005
from Equal Variances Assumed line of SPSS output
What is your conclusion?
Do you reject or not reject the null hypothesis?
Reject the null
What are the upper and lower limits of a 95% confidence interval appropriate for the problem? Present
them in a sentence, with standard interpretive language.
Lower Limt = -8.05
Upper Limit = -1.55
We can be 95% sure that the difference in population means is between -8.05 and -1.55.
State the implications of your conclusion for the problem you were asked to solve. That is, relate your
statistical conclusion to the problem.
Recall associated with scents is apparently better than recall with no scent associated with it.
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 6
2/9/2016
Effect size from the conduct of the Independent Groups t-test
Alas, SPSS does not print the effect size, nor does it print a quantity that can be easily converted to the
effect size.
Corty discusses effect size on pages 248-249. He does not straightforwardly present the formula.
I won’t require it, but if you wish to compute the effect size, the formula is
Observed mean 1 – Observed mean 2
d = --------------------------------------------------(N1-1)S12 + (N2-1)S22
----------------------N1-1 + N2-1
s2pooled
Just for kicks, let’s compute it.
(N1-1)S12 + (N2-1)S22
14*4.2562 + 14*4.4272
S2pooled = ----------------------- = -----------------------N1-1 + N2-1
14
+
14
21.4 – 26.2
d = -------------------- =
4.34
Biderman’s P201 Handouts
-4.8
--------------4.34
= 4.34
= -1.10, a huge effect size,
Topic 13: Two Population Tests - 7
2/9/2016
A Second Example of the Independent Groups t-test
Who uses social media more – males or females.
A psychologist is studying the use of social media – such as Facebook – among young adults. She is at the
beginning of her research program, and right now is simply interested in discovering who uses social media
more – males or females. She decides that the population to which she would like to generalize her results
is the population of university students, specifically the university at which she works.
She takes a convenience sample of students eating lunch at the university center and asks them to fill out a
simple questionnaire. One of the questions is, “On average, how many times a day do you check your
favorite ‘social media’ account?”
She obtained the following data . . .
Females
12
32
40
28
54
35
29
40
27
53
37
23
19
18
42
28
35
18
33
29
30
21
17
8
37
Males
23
25
Test the hypothesis that the mean number of times social media is used by the population of males students
is equal to the mean number of times social media is used by the population of female students.
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 8
2/9/2016
Performing the Analysis
1. Enter the data into the computer
SPSS Data Entry
Biderman’s P201 Handouts
Excel Data Entry
Topic 13: Two Population Tests - 9
2/9/2016
Performing the Analysis cont’d . . .
2. Invoke the Independent t procedure
Carrying out the analysis using the SPSS Independent Groups t-test
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 10
2/9/2016
Carrying out the analysis using Excel Two-Sample t Assuming Equal Variances
Note that Excel is much more flexible than SPSS concerning where the data can be located in the
spreadsheet.
SPSS requires that ALL the scores values be in the same column and REQUIRES that you create a second
column to indicate which group each score is in. Excel does not.
But SPSS gives you a LOT more statistical capability than Excel. Small price to pay.
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 11
2/9/2016
3. Examine the Results
SPSS results
T-Test
Group Statistics
sex
uses
N
Mean
Std. Deviation
Std. Error Mean
1
13
33.00
12.159
3.372
0
14
26.00
9.215
2.463
Independent Samples Test
Levene's Test for
Equality of Variances
t-test for Equality of Means
F
95% Confidence
Sig.
uses
t
df
Mean
Std. Error
Interval of the
Sig. (2-
Differenc
Differenc
Difference
tailed)
e
e
Lower
Upper
Equal variances
.671
.420
1.694
25
.103
7.000
4.133
-1.511
15.511
1.676
22.347
.108
7.000
4.176
-1.652
15.652
assumed
Equal variances
not assumed
Excel Results
The key results from the two programs are the same, just presented differently.
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 12
2/9/2016
4. Present the Results
Corty’s Hypothesis Testing Answer Sheet
1. Give the name and the formula of the test statistic that will be employed to test the null hypothesis.
Independent Groups t-test. (Equal population variances assumed.)
2. Do the data meet the assumptions of the test? Provide evidence.
I see no drastic violations of the assumptions.
3. The null and alternative hypotheses.
Null Hypothesis: Population mean use of social media in males and females is equal.
Alternative Hypothesis:_There is a difference in male and female population mean use of social media.
4. What significance level will you use to separate "likely" value from "unlikely" values of the test
statistic?
Significance Level = .05
5. What is the value of the test statistic computed from your data and the p-value?
Test statistic value = 1.69
6. What is your conclusion?
_
p-value = .10
Do you reject or not reject the null hypothesis?
I fail to reject the null. The evidence suggests that the population means are equal.
7. What are the upper and lower limits of a 95% confidence interval appropriate for the problem?
Lower Limit = -1.51
Upper Limit = 15.51
8. State the implications of your conclusion for the problem you were asked to solve. That is, relate your
statistical conclusion to the problem.
Male and female students at this university use social media about equally often.
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 13
2/9/2016
Paired-Samples t Test: Overview Start here on 10/22/15.
If persons in the two conditions are paired so that each person in one condition has a “match” in the other
condition, a different test is required.
Names for this research design . . .
Matched Participants Design
Each person in one condition is matched with a person in the other condition on some test.
Participants as Their Own Controls Design
Each person serves in both conditions, so each person is matched with himself/herself
PrePost Designs
A version of the above in which persons are tested twice – first before a treatment, then after it
Longitudinal Designs
A version of the above in which persons are tested, then after a prespecified period of time, tested
again.
Repeated Measures Designs
Any design in which the same persons are tested more than once
The research designs presented above are more efficient than the independent groups designs.
There are two major differences
1. The two conditions are more likely to be equal because matched or the same people are in both.
This means that any difference between the groups can be attributed to the treatment and not to pre-existing
group differences.
2. If there is a difference due to whatever treatment is being evaluated, it will be more likely to be
detected if a paired-samples design such as these is used.
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 14
2/9/2016
Test Statistic: Paired-Samples t Test
The official formula is presented here for completeness. But we won’t use it.
X1 - X2
t=
S21 + S22 - 2rS1S2
N
Where (more than you ever wanted to know about the correlated groups t formula)
X1
= Mean of the sample from the first population
X2
= Mean of the sample from the second population
S1
= Standard deviation of the sample from the first population.
S2
= Standard deviation of the sample from the second population.
N
= Number of pairs.
r
= Pearson Product Moment Correlation Coefficient between the pairs. Corty, Chapter 13.
The formula we would use for hand computations if we were stranded on a deserted island is
D
D
t = ----------------SD
N
Where
`D
= The mean of the paired difference scores.
SD = The standard deviation of the paired difference scores.
Even the above formula will be difficult to compute by hand for more than a few pairs of scores.
Luckily, our computer program will do it for us.
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 15
2/9/2016
Correlated Groups (also called the Paired-Samples) t-test Example problem.
The data are from Corty, p. 269.
Dr. Kearn wants to examine the effect of humidity on perceived temperature. She obtained six volunteers at
her college and tested them, individually, in a temperature- and humidity-controlled chamber.
Each participant was tested twice so it’s a Participants as Their Own Control Design. The temperature was
set at 76° for both conditions. But in the Control Condition, the humidity was low and in the
Experimental Condition the humidity was high.
Perceived Temperature is the dependent variable in the study.
The data in the form they would be entered into SPSS
Participant
1
2
3
4
5
6
Control Condition
Perceived Temp
76
80
78
72
76
68
Experimental Condition
Perceived Temp
81
90
85
82
82
75
The data as they would be entered in SPSS
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 16
2/9/2016
The menu sequence for the Paired-Samples t Test is
The SPSS Paired-Samples t Test dialog box
A screen shot after the variables to be analyzed have been specified
A screen shot after the order of variables was reversed, so the t will be positive
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 17
2/9/2016
The output of the Paired-Samples t Test
Paired Samples Statistics
Mean
Pair 1
N
Std. Deviation
Std. Error Mean
experimental
82.50
6
4.930
2.012
control
75.00
6
4.336
1.770
Paired Samples Correlations
N
Pair 1
Correlation
experimental & control
6
We have not yet covered
correlations. We’ll do that in a
week or so.
Sig.
.908
.012
Paired Samples Test
Paired Differences
95% Confidence Interval of the
Difference
Std. Error
Mean
Pair 1
Std. Deviation
Mean
Lower
Upper
t
df
Sig. (2-tailed)
experimental 7.500
2.074
.847
5.324
9.676
8.859
5
.000
control
Upper limit of confidence interval for difference in population means = 9.676
Lower limit of confidence interval for difference in population means = 5.324
Same analysis in Excel . . .
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 18
2/9/2016
Results in Excel
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 19
2/9/2016
Filling out the Corty Hypothesis Testing Answer Sheet
1. Give the name and the formula of the test statistic that will be employed to test the null hypothesis.
Paired-Samples t-test
2. Do the data meet the assumptions of the test? Provide evidence.
They do.
3. The null and alternative hypotheses.
Null Hypothesis:
Difference between population means is equal to 0.
Alternative Hypothesis:
Difference between population means is not equal to 0.
4. What significance level will you use to separate "likely" value from "unlikely" values of the test
statistic?
Significance Level =
.05
5. What is the value of the test statistic computed from your data and the p-value?
Test statistic value = 8.859
6. What is your conclusion?
p-value = .0003
Do you reject or not reject the null hypothesis?
I reject the null hypothesis.
7. What are the upper and lower limits of a 95% confidence interval appropriate for the problem?
Lower Limit =
5.324
Upper Limit = 9.676
8. State the implications of your conclusion for the problem you were asked to solve. That is, relate your
statistical conclusion to the problem.
The implication is that high humidity causes temperatures to feel higher than low humidity.
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 20
2/9/2016
Effect Size
In spite of what Corty says, an estimated of Cohen’s d can be computed for these data.
The formula is
ME - MC
d = ---------------------------------S2E + S2C
-----------2
For the above problem,
82.50 – 75.33
d = ---------------------------------- =
4.932 +4.842
-------------2
Biderman’s P201 Handouts
7.17
7.17
------------------------- = ---------------------- = 1.47 (huge)
47.73
--------4.89
2
Topic 13: Two Population Tests - 21
2/9/2016
t-tests example problems
1. A psychologist has devised a new method of teaching a foreign language. She chooses 30 persons who
have never spoken French and then places 15 of them in a regular college-level French class. The other 15
students are taught using her new method. The results are below. Set up and conduct the appropriate test.
The dependent variable is the no. of questions answered correctly on a standardized examination covering
knowledge of French.
Old method
New method
Mean
55.3
54.3
S.D.
12.3
11.2
n
15
15
2. Suppose you have been put in charge of evaluating the design of the packaging for a new product your
firm is marketing. You select 12 persons and have them evaluate both the old design and the new design.
Six persons see the new design first. The other six see the old design first. The products are evaluated on a
variety of measures. Our interest here is on the responses on an overall, summary scale of favorability to
the product. The data are presented below:
Person: 1 2 3 4 5 6 7 8 9 10 11
Old
32 35 44 49 19 23 30 30 28 48 34
New
29 34 38 45 22 21 29 30 22 41 31
12
38
35
3. A company has created an advertisement for its new product. The advertisement is shown to a group of
persons who might be purchasers of the product. They’re asked to rate the advertisement on a 7-point scale.
The responses on the 7 point scale are labeled
1
2
3
4
5
6
7
Very
Somewhat Neither Good Somewhat
Very
Bad
Bad
Bad
nor Bad
Good
Good
Good
The company decides that it will not use the advertisement unless the mean of the responses differs
significantly from the neutral point of 4 and if the sample mean is greater than 4. The data, for 44 persons,
are
1: 3
2: 5
3: 6
4: 7
5: 10
6: 12
7: 3
4. Suppose you're investigating the effects of temperature on performance on the job. You select two work
areas in which employees perform the same tasks. In one of the areas, you arrange to have the ambient
temperature set to 78° F. In the other area, you arrange to have it set equal to 70° F. In both areas, workers
wear fairly heavy protective clothing. The results are as follows. The dependent variable was a measure of
output on a scale of from 0 to 20.
70° F:
78° F:
11 13 15 14 15 18 17 12 13 19 20 13 15
10 9 8 12 15 16 13 14 18 12 13 11 10 6 13
5. In an attempt to assess the effect of placing police cars at key places on the interstate system, a researcher
puts a police car on the highway and records speeds just prior to motorists' seeing the car and just after. The
results are as follows. Before speeds are first.
65-67 63-58 70-60 72-72 55-55 57-56 45-48 79-59 60-57 58-54 59-60 68-62 64-58
Biderman’s P201 Handouts
Topic 13: Two Population Tests - 22
2/9/2016