Chapter 6: t test for dependent samples
****This chapter corresponds to chapter 11 of your book (“t(ea) for Two (Again)”).
What it is: The t test for dependent samples is used to determine whether the means of two
related groups are significantly different. It is called a t test for dependent samples because you
use it when you are comparing the same group of people that was measured twice (i.e., groups
that are related to or dependent on each other).
When to use it: Per the flow chart on page 190 of Salkind (2008), you would use the t test for
dependent samples when:
(1) you are examining differences between groups (as opposed to examining the
relationship between two variables) and
(2) the participants in the study were tested more than one once (as opposed to once) and
(3) You are comparing two groups/time points (as opposed to three or more groups/ time
points)
Questions asked by the t test for dependent samples: Do the means of two groups of
related scores differ from each other?
Examples of research questions that would use a dependent t test:
o
o
o
Do the students in this class know more about SPSS at the beginning or end of
the semester?
Does people’s meaning in life increase after they have a child?
Do people lose weight after attending a weight loss boot camp for 2 months?
Using SPSS to Calculate a dependent t test (Data set: Chapter6_Example1.sav)
Imagine you’ve designed a method that you believe can teach algebra to people in a very short
amount of time (15 minutes). To test this method, you recruit 10 random individuals and give
them an algebra test of 10 problems. You then have them work through your method for 15
minutes and then they complete another algebra test of 10 problems. Now you want to compare
the scores on the pre and post-test to determine if scores improved after participants went
through your training.
To determine whether your method worked or not, follow the famous 8 steps….
1. Statement of the null and research hypotheses
Null H0 : µpretest = µposttest
The average score on the pretest and posttest are equal.
Research H1: X pretest ≠ X posttest
The average score on the pretest and posttest are not equal.
2. Setting the level of risk to p < .05
3. Selection of the appropriate test statistic
Again, using the flowchart on page 190 of Salkind (2008) we see that the t test for
dependent samples is the appropriate test statistic because we are comparing the
means (of test scores) between two dependent groups (the same people tested twice,
once before learning the method and once after.
Open the dataset “Chapter6_example1.sav”. Take a moment to familiarize yourself with the
data. Note how data for this type of analysis should be entered.
1) Each participant has one row in the data
2) One column is used to indicate that participant’s score on the pretest (i.e. the number of
problems out of 10 they answered correctly)
3) Another column indicates each participant’s score on the posttest.
The data should look something like this in SPSS:
If you were to switch to variable view, you would see that more descriptive labels have been
added to the variables. These labels will be what shows up in the output and are helpful to
remind you (the researcher) what the variables represent. In your own data set, you could use
whatever labels you find most helpful.
4. Computation of the test statistic.
We will use SPSS to compute the test statistic for us. To do so, click on the “Analyze” dropdown menu, highlight “Compare Means”, and then click on “Paired Samples t Test”, as pictured
below.
The following pop-up window will appear:
Highlight the name of the variable that represents the pre-test score and click the arrow to place
it in the “paired variables:” box. Next, highlight the name of the variable that designates the posttest score and click the arrow again to also place it in the “paired variables:” box. Your screen
should look like the picture below:
Now, Click OK and navigate to the output window to find your results. The output will look like
this (continued on the next 2 pages):
Paired Samples Statistics
Mean
Pair Pretest Number of Correct Answers
Std.
Std.
Deviatio
Error
n
Mean
N
4.7000
10 1.76698
.55877
5.7000
10 1.63639
.51747
This table tells us the descriptive statistics (mean, N and standard deviation) for both the variables we tested (in our example, pre and posttest scores). These will be helpful in writing up our results. 1
Posttest Number of Correct Answers
Paired Samples Correlations
N
Pair 1
Pretest Number of Correct
Correlation
10
Sig.
.465
.176
Answers & Posttest Number of
Correct Answers
This box tells us the value of the correlation coefficient and the p‐value associated with it for the two variables we tested. A correlation coefficient is a numerical index that tells us about the relationship between the two variables (if you haven’t learned about correlations yet, don’t worry ‐ you will learn about them later). For our purposes right now, we can ignore this box. Paired Samples Test
Paired Differences
95% Confidence Interval
of the Difference
Std.
Mean
Pair 1
Pretest Number of Correct Answers -
-1.00000
Deviation
Std. Error Mean
1.76383
.55777
Lower
-2.26177
Upper
.26177
Sig. (2t
-1.793
df
tailed)
9
.107
Posttest Number of Correct Answers
The box labeled “mean” tells us the difference between the two means (i.e. pretest mean – posttest mean OR 4.7 – 5.7). The box labeled “standard deviation” tells us the standard deviation of the difference scores (think the “Difference” column when you calculate this by hand). This is useful for calculating the effect size. The box labeled “t” tells us the exact obtained value for our test statistic. This is the same number you would get if you calculated t by hand following the method in your text book. The box labeled “df” tells us the degrees of freedom for the test (i.e. n ‐1, where n equals the number of pairs). The box labeled “Sig. (2‐tailed) tells us the exact p‐value associated with this obtained value and number of degrees of freedom. Interpreting the Output
The third piece of the output (the table labeled “paired samples t-test”) includes all of the
information you’ll need to complete the 8 steps. In particular, you want to locate the obtained
value, degrees of freedom, and p-value associated with the analysis you just ran. There is also
information that helps you interpret the t-statistic and some information that you can simply
ignore.
5. Determination of the value needed for rejection of the null hypothesis
If we were doing this example by hand, then this is the point when we would look at the
table of critical values for t (Salkind, 2008, pgs 333-334). Recall that we look up the
critical value to tell us the smallest value of t needed to reject the null hypothesis. The
critical value is the t that corresponds to a p of .05 for a specific number of degrees of
freedom. Because p gets smaller as t gets bigger, we know that if our obtained value is
bigger than that critical value, then the p is less than .05. However, SPSS gives us an
exact p-value! This means we don’t have to find the critical value on our own when we
use SPSS.
6. Comparison of the obtained value and the critical value is made
Because SPSS gives us a p-value, all we have to do now is see whether that p-value
given to us (in the output) is greater or less than .05 (the level of risk we are willing to
take). In this example, the output tells us that the p-value for our test is .107 (which is
greater than our cutoff value of .05). This means that we fail to reject the null hypothesis.
Put another way, this means there is a 10.7% chance that we would obtain this pattern
of results if the null hypothesis was true and there was no effect of the method on
algebra ability. While 10.7% might not seem very likely, recall that we are only willing to
take a 5% chance. This means that we cannot reject that null hypothesis in our example.
7/8. Making a Decision
Because our p-value (.107) is greater than .05, this means we fail to reject the null
hypothesis. In other words, we conclude that there is no significant difference between
the pretest and posttest scores.
Interpretation of the Findings
Now we report our results. Here’s an example of how these results would be reported in a
journal article:
A t test for dependent samples revealed that there was not a significant effect of the
learning method on test scores (t(9) = -1.79, p > .05). Although participants performed
somewhat better after (M = 5.70, SD = 1.63) compared to before the tutorial (M = 4.7,
SD = 1.77), this difference was not statistically significant.
For someone unfamiliar with stats, you might say: The algebra tutorial does not appear
to influence algebra ability.
Effect Sizes
Although not covered in Salkind (2008), it can be helpful to compute an effect size of the
difference between the scores in your data set. For a refresher, see Salkind (2008) pages 178180 for a discussion of why effect sizes are important.
To compute an effect size for dependent t, we can easily plug in some of the information from
the output we already generated into the following formula:
Mean Difference
Standard Deviation of the Difference Scores
If you refer to your output, you’ll see both of these pieces of information in the first part of the
table labeled “paired samples t-test”.
For our example then:
1
1.76
.57
**Notice that we’ve used a 1 rather than the -1 in the output. That’s because we are interested
in the SIZE of the effect, not the direction.
Cohen’s guidelines for interpreting an effect size (See page 180 of Salkind) tell us that this is
actually a large effect size. This is somewhat contradictory with our non-significant results
reported above. While the results of this particular study suggest that the pattern of results likely
occurred by chance, this may also mean that all hope might not be lost for your proposed
method! At this point, you might try running another study with either more participants or a
different methodology. This helps illustrate why it can be important to pay attention to both
statistical significance AND effect sizes. This doesn’t guarantee you’ll find a significant result the
next time, but does suggest it might be worth trying. On the other hand, if the effect size had
also been very small (i.e., close to zero) then you might want to think about developing a new
method…
Practice Problem #1 (for SPSS)
A researcher is interested in changes over time in happiness after a break up. He asks 6 people
to complete a Happiness scale the first day after a break up and then complete the same scale
again 20 days later. Higher numbers represent more happiness. Use SPSS to enter the data
below and answer the questions that follow.
Day 1
Day 20
12.00
14.00
13.00
15.00
13.00
15.00
15.00
18.00
15.00
18.00
13.00
17.00
A. What is the null and research hypothesis (in both words and statistical format)?
B. What is the level of risk associated with the null hypothesis?
C. What is the appropriate test statistic and WHY?
D. What is the obtained value for the t-t test and what is its associated p-value? Is the
difference between the two groups statistically significant?
E. What do you conclude about the effect of video game type on aggression? Write up your
results as you would for a journal article.
F. Write up your results as you would for an intelligent person who doesn’t know stats.
Practice Problem #2 (Hand Calculation)
The data below are from participants’ ratings of how jittery they feel before and after drinking
a Red Bull (higher scores equal more jittery). Use a dependent samples t-test to assess
whether or not the Red Bull increased feelings of jitteriness.
Pretest
3
2
4
2
1
5
6
3
2
3
Posttest
4
7
6
4
5
8
9
8
6
7
A. What is the null and research hypothesis (in both words and statistical format)?
B. What is the level of risk associated with the null hypothesis?
C. What is the appropriate t test statistic and WHY?
D. What is the value needed for rejection of the null hypothesis (i.e. the critical value)?
E. What is the obtained value? Is it greater than the critical value?
F. Can you reject the null hypothesis? What do you conclude about the effectiveness of the
pain relief drug? Write up the results as you would for a journal article. You should
calculate the means of the two groups for this problem, but you do not have to calculate
the standard deviation.
G. Write up your results as you would for an intelligent person who doesn’t know stats.
Practice Problem 3 (SPSS and Hand Calculation)
Below are the number of free throws made (out of 5 attempts) by a group of players. During the
first set of attempts, the crowd was trying to distract the players. During the second set of
attempts, the crowd was quiet. Hand calculate the analysis using the data below and then use
SPSS to check your answer. Also calculate an effect size for the data and include this
information when you write up your results as you would for a journal article.
Time1 Time2
2.00
1.00
3.00
4.00
3.00
3.00
4.00
5.00
4.00
3.00
5.00
3.00